JPH09325962A - Document corrector and program storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the efficiency of correction work by reducing the judgement of a user by preventing erroneous parts in a document from being excessively pointed out concerning a document corrector, with which errors in a document in Japanese, etc., are pointed out, and a program storage medium. SOLUTION: Plural error probability calculation parts 2a, 2b... respectively calculate error probability values through different error probability calculating methods. An error probability preserving part 3 sums up these error probability values and applies the summed-up error probability value to the respective characters or character strings of a source text. An error candidate extracting part 5 extracts the character or character string having the error probability value exceeding a prescribed error probability threshold as a candidate. An error display processing part 6 displays the extracted error candidate in the text for correction with an underline or in a different color.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention proofreads a document by automatically pointing out an erroneous portion in document data input by a user in a document processing apparatus or acquired as an electronic medium. The present invention relates to a document proofreading device capable of reducing work and significantly improving work efficiency related to document proofreading, and a program storage medium for realizing the document proofreading device.

[0002]

2. Description of the Related Art In a conventional document processing apparatus, as a method of pointing out an error candidate, there is a method of pointing out an unregistered word part from the result of morphological analysis and pointing out a word having a homonym. In addition, there is a method in which when a specific word string is detected, it is recognized as an error and pointed out. This is because, for example, using a dictionary for error detection that pre-registers kana usages and notations that are not currently used, it is assumed that the matching one is an error, or that a noun + verb, which is an unusual part of speech, is detected. Alternatively, it is a method such as making an error when there is a single Kanji word. Another method is to arrange Katakana or Kanji character strings in the text in a dictionary order so that words with the same slight fluctuations in the notation are arranged consecutively to make it easier to detect fluctuations in the notation. is there. For example, in the case of "window", "window", and "window", this is a method used to unify the notation.

[0003]

The above-mentioned method of pointing out the unregistered word portion is a method of pointing out that portion because there is a high possibility that there is a misspelled word around the unregistered word portion. However, as the cause of the unregistered word, besides the spelling error, the existence of the original unregistered word, such as a proper noun, which is not mistaken but is not registered in the dictionary, can be mentioned. Therefore, the pointed-out portion cannot be determined to be an error, and the user must judge the correctness one by one.

The above-mentioned method of pointing out a word portion in which a homonym is present is a method of pointing out for confirmation purposes because it is a portion where an operation is likely to be erroneous during kana-kanji conversion. In this case, if there is at least one homonym, it is pointed out even if it is not originally an error word, so there is a problem that the number of parts in which the user has to make a correct judgment becomes very large. is there.

On the other hand, in the method of pointing out as an error when the above-mentioned specific word string is detected, it is necessary to register a part-of-speech string or the like that should be detected as an error in advance, and the object of the error is very limited. In reality,
There is a problem that many errors in sentences cannot be detected.

Further, the method of sorting the katakana words and kanji words and presenting them to the user has a problem that the work efficiency is not improved so much even though the user has to do much work at the time of proofreading.

As described above, when the conventional method is used, the point where the error candidate is pointed out is often not the original error, and there is a problem that there is a tendency for excessive pointing out. As a result, the amount of re-checking by the user becomes large, which may reduce the efficiency of the calibration work.
In addition, there is a problem that proper selection of error locations is often insufficient.

[0008]

In order to solve the above problems, the present invention is provided with each means shown in FIGS. 1 to 5, for example.

FIG. 1 shows a block configuration example of the invention according to claim 1. In FIG. 1, 1 is an error candidate detection unit, 2
a, 2b, ... Are error probability calculation units, 3 is an error probability storage unit,
4 is an error probability threshold value, 5 is an error candidate extraction unit, and 6 is an error display processing unit.

The error probability calculators 2a, 2b, ... Are means for calculating and giving an error probability value to each constituent character or character string of text by different methods. Each of the error probability calculation units 2a, 2b, ... May use the means shown in FIGS. 2 to 4 to be described later singly or in combination. Moreover, the means for detecting an error candidate as described in the prior art may be used. A predetermined error probability may be given to each by using them in combination.

The error probability storage unit 3 includes an error probability calculation unit 2
It is a means for totaling the error probability values obtained from a, 2b, ... And determining the final error probability value for each constituent character. The error candidate extraction unit 5 is a means for extracting a character or a character string having an error probability value higher than this as an error candidate based on a predetermined error probability threshold 4. The error probability threshold 4 is given as an external input or as electronic data stored in advance.

The error display processing unit 6 is means for displaying the error location extracted by the error candidate extraction unit 5. With the above-described configuration, the error probability storage unit 3 aggregates a plurality of error probability values obtained from the respective error probability calculation units 2a, 2b, ... And calculates the respective error probability values for each character or character string. Therefore, it is possible to more accurately extract a candidate with a higher probability of error than before.

Further, since the error display processing unit 6 extracts and displays only those exceeding the predetermined error probability threshold value 4 by the error candidate extraction unit 5, it is possible to prevent excessive error indication.

Furthermore, by changing the error probability threshold value 4, the amount of words and the like extracted and displayed as error candidates can be adjusted, so that the error indication can be made depending on the purpose of the proofreading work and the type and content of the original text. Keep the precision,
Alternatively, it is possible to select whether to improve work efficiency.

FIG. 2 shows an example of the configuration of the error probability calculation unit according to the second aspect of the invention. The error probability calculation unit 20 includes a morpheme analysis unit 21, an unregistered word detection unit 22, and a proper noun pattern detection unit 23.

The morphological analysis unit 21 is a means for performing morphological analysis on text, dividing the text into word strings, and passing the word group to the unregistered word detection unit 22 and proper noun pattern detection unit 23. Since a specific method for morphological analysis has been well known in the related art, detailed description thereof will be omitted here.

The unregistered word detection unit 22 detects a word region that is regarded as an unregistered word from the word group, adds a predetermined error probability value to the word region, and passes it to the error probability storage unit 3 in FIG. Further, the proper noun pattern detection unit 23 receives the word string of the morphological analysis result, recognizes the area where the proper noun is likely to appear by using syntactic features, and recognizes the area where the proper noun is likely to appear. Negative error probability value is added to
To the error probability storage unit 3. Syntactic features are those that can be inferred from the meaning and part-of-speech of known words before and after, and titles such as "President" and "Manager" are often personal names, "city" and This is a characteristic of the pattern in which the words that the kanji string or katakana that precedes the character "state" are often place names are arranged. The reason why a negative error probability value is given to a proper noun is that a proper noun is not an error even if it is an unregistered word. Therefore, it is not necessary to give a negative error probability value to proper nouns that are not unregistered words.

In such a proper noun detection, a proper noun and a specific part-of-speech or word combination is registered in advance as analysis data such as "unregistered word + title", and the proper noun pattern detection unit At 23, the result of the morphological analysis may be matched with these proper noun patterns.

As a result, a word area related to a proper noun that is always pointed out as an error candidate in the conventional method,
It is possible to remove from the point of error. FIG. 3 shows a configuration example of an error probability calculation unit according to the invention of claim 3.

The error probability calculation unit 30 includes a vocabulary extraction unit 31,
A categorization unit 32 and a probability assignment unit 33 are provided. The vocabulary extraction unit 31 is a unit that extracts words from the target text and sends them to the categorization unit 32. The categorization unit 32 is a unit that calculates the degree of similarity in character composition between words for the word group obtained from the vocabulary extraction unit 31 and performs categorization based on the calculated degree of similarity. The probability assignment unit 33 includes the categorization unit 32.
It is a means for giving an error probability value based on the degree of similarity between words and the frequency of appearance of words categorized by.

According to the error probability calculator 30 shown in FIG.
For example, words with a high degree of similarity to each other, such as words that have fluctuations in the notation, are classified into the same category, and which of the notations is legal or incorrect is determined based on the similarity and appearance frequency of the words. Since it can be given as an error probability value, it is possible to reflect the error tendency due to the fluctuation of the notation in the finally displayed error candidate.

FIG. 4 shows an example of the configuration of the error probability calculator according to the invention of claim 4. The error probability calculation unit 40 includes a morpheme analysis unit 41, a region division processing unit 42, and a probability assignment unit 43. The morpheme analysis unit 41 is means for performing morpheme analysis on the target text and dividing it into word strings. Area division processing unit 42
Is a means for dividing the target text into areas composed of the same kind of characters such as Kanji, Hiragana, Katakana, English letters, ....

The probability assigning unit 43 uses the stored average word length for each character type to calculate the predicted number of words in each word region of the same character type in that region and the word group by morphological analysis. It is a means of comparing the calculated actual number of words and giving an error probability value based on the difference for each character or character string. That is, the probability assignment unit 43 considers that there is a high probability that a region that has a large difference between the predicted number of words and the actual number of words includes a word that is not recognized as one word due to a spelling error or the like. To a relatively high error probability value.

Each of the above-mentioned means shown in FIGS. 1 to 4 is realized by using a program for operating a computer. The program can be stored in an appropriate storage medium. FIG.
[Fig. 6] is a diagram showing an example of the block configuration of the invention according to claim 6. In FIG. 5, 51 is an error candidate detection unit, 52 is an extended morphological analysis unit, 53 is a word comparison unit, 54 is a word dictionary, 5
Reference numeral 5 represents an error display processing unit.

The error candidate detecting section 51 is means for detecting a character string having a high error probability from the input text by means similar to the error candidate detecting section 1 shown in FIG. 1, for example. In calculating the error probability, for example, the means shown in FIG. 4 is used. The word comparison unit 53 compares the similarity between the character string obtained as an error candidate from the error candidate detection unit 51 and the word registered in the word dictionary 54, and has a similarity of a predetermined value or more. It is a means to determine that the two match. The extended morpheme analysis unit 52 performs extended morpheme analysis on the character string which is the error candidate obtained from the error candidate detection unit 51, and as a result of the extended morpheme analysis, the evaluation value of the word region is the error candidate detection unit 5
If it is better than the evaluation value obtained from 1, that is, if the overall error probability is small, the word (word group) obtained by the extended morpheme analysis is recognized as the correct word (correct word group).

Here, the extended morphological analysis is not similar to the morphological analysis using only the information in the case of exactly matching the word registered in the word dictionary 54. This means that morphological analysis is also performed using word information.

The error display processing unit 55 displays the correct answer word group obtained from the extended morpheme analysis unit 52 together with the original target text. Each of the above means shown in FIG. 5 is realized by using a program for operating a computer. The program can be stored in an appropriate storage medium.

[0028]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 6 shows the error probability calculator 2 shown in FIG.
It is a figure explaining the example of error probability calculation by 0.

When the original text to be proofread is "Jupe Foreign Minister nodded his opinion", the morphological analysis unit 21 shown in FIG. 2 performs morphological analysis, and as a result, "/ Jupe / Foreign Minister / ha / that The word string "/ opinion / ni / nodazu / ita /" is passed to the unregistered word detection unit 22 and is also passed to the proper noun pattern detection unit 23 including attribute information such as a part of speech.

In the unregistered word detection unit 22, "Jupe",
"Unazu" is detected as an unregistered word. An error probability of 0.5 is given to each of the detected unregistered character strings. This value of 0.5 is a value determined in advance by statistical data.

Further, in the proper noun pattern detecting section 23,
“/ Jupe / foreign minister /” is detected as a match with “unregistered word + title”, which is one of the proper noun patterns registered in advance. Here, in general, the proper title is a proper noun, and the proper noun is usually an unregistered word. Therefore, the word string "/ Jupe / Foreign Minister /" is unlikely to be an error. Become. Therefore, a negative error probability of −0.4 is added to “Jupe” of “/ Jupe / Foreign Minister /”.

From the error probability value from the error probability calculation unit 20, the error probability storage unit 3 sets the error probability value of the character string "Jupe" to 0.1 (= 0.5-0.4), and the character string " The error probability value of "Unazu" is set to 0.5.

From the above results, the same error candidate extraction as the error candidate extraction unit 5 shown in FIG. 1 is performed. If the error probability threshold 4 is set to 0.2, for example, the "Jupe" part is "0.1≤error probability threshold (0.2)".
Therefore, it is excluded from the error candidates, and the part of "unadzu" is
Since “0.5> error probability threshold (0.2)”, it is extracted as an error candidate. The extracted error candidates are sent to the error display processing unit 55. The error display processing unit 55 distinguishes the "nodule" portion as an error indication portion from other portions by underlined display or different color display, and outputs it as proofreading text to a display device or the like.

As described above, by suppressing the error probability value of the unregistered word, which is a proper noun that is not originally an error, "Jupe" is prevented from being displayed as an error candidate. The result of the extended morpheme analysis unit 52 of FIG.
The configuration may be such that it is passed to the error probability storage unit 3 shown in FIG.

FIG. 7 is a diagram for explaining an example of error probability calculation by the error probability calculator 30 shown in FIG. It is assumed that words having the same meaning are represented in three different notations of "inverted", "inverted", and "inverted" in the original text to be proofread. Vocabulary extractor 3
1 calculates the appearance frequency of each word of the original text and passes the information of the word and the appearance frequency to the categorization unit 32.

The categorization unit 32 classifies words having a high degree of similarity in character configuration into the same category. In this categorization process, for example, "ba" and "va" have a very high degree of similarity, "zu" and "zu" have a very high degree of similarity, and there are cases where there is a long-sound "-" Based on the information that the degree of similarity is high, and that the same character with and without the dakuten has a slightly higher degree of similarity, using the value of the degree of similarity determined in advance for the relationship of specific characters,
The word similarity is calculated. As a result, the three words "inverted", "inverted", and "inverted" are classified into the same category.

Next, between the words in the same category, the error probability is calculated based on the highest frequency word and the similarity of the character structure. In this example, the highest frequency of "inverted"
Is recognized as the correct word, and the similarity between the notations "ba" and "va" is high, so the similarity of "inverted" to "inverted" is 10, and the endings of "do" and "to" The similarity of the notation is not so high, and the similarity of "inverted" to "inverted" is 2. Then, the error probability value is determined based on this similarity. Since "Inverted" is recognized as a correct answer word, the error probability is set to 0. "Inverted" having a high similarity to the correct answer word has an error probability of 0.2, and "Inverted" having a low similarity is The error probability is 0.8.

Here, the error probability threshold value 4 shown in FIG.
When set to a high value of 7, the error candidate extraction unit 5 extracts only the character string "inverted" as an error candidate. on the other hand,
If the error probability threshold value 4 is set to a low value of 0.1, the error candidate extraction unit 5 extracts both the character strings “inverted” and “inverted” and recognizes many words that are different from the original word. Is displayed in the proofreading text as an error candidate. Note that this error probability value is an example, and depending on the type of text, it may be better to reduce the error probability of words with a low degree of similarity within the same category.

FIG. 8 is a diagram for explaining an example of the error probability calculation by the error probability calculation unit 40 shown in FIG. 4 and the processing of the extended morpheme analysis unit 52 shown in FIG. When the original text is “keyword file designation”, the morphological analysis unit 41 shown in FIG. 4 outputs “/ key / word / file / of / designation /” as a result of the morphological analysis. Here, since the words "key", "word", and "file" were found in the word dictionary, unregistered words were not detected.

Further, the area division processing unit 42 performs area division processing for each same character type, so that the "keyword file" for the katakana part, the "no" for the hiragana part, and the "designation" for the kanji part.
It is divided into three parts.

The probability assigning unit 43 compares the actual number of words obtained by morphological analysis with respect to "/ key / word / file /" and the number of words predicted by the area division processing. The number of predicted words is calculated from the average word length statistically determined by the character type. For example, if the average word length of the katakana character string is 4, the predicted number of words in the “keyword file” is 2 (≈9 / 2).

The number of words obtained from the morphological analysis of this portion is 3, and thus the actual number of words and the predicted number of words in the "keyword file" will not match.
The probability assignment unit 43 determines that an error may exist in the word area “keyword file” in which the mismatch occurs, and assigns an error probability (eg, 0.1) to each character according to the degree of mismatch. To do.

According to the first aspect of the invention, the error probability value from the error probability calculation unit 40 and the error probability value from another error probability calculation unit are totaled by the error probability storage unit 3 shown in FIG.
The error candidate extraction unit 5 extracts the error candidate, and the error display processing unit 6 displays the proofreading text indicating the error candidate.

According to the sixth aspect of the present invention, the processing for obtaining the correct word string is further performed by using the extended morphological analysis as follows. The extended morpheme analysis unit 52 uses the error candidate detection unit 51.
The character string “keyword file” that is detected by and has an error probability value above a certain value is the target of the extended morpheme analysis. The word comparison unit 53 detects “keyword” and “file” from the words registered in the word dictionary 54 as a word string having a high similarity to the “keyword file”, and the extended morpheme analysis unit 52 detects the extended morpheme analysis. as a result,
“/ Keyword / file /” is extracted.

"/" Obtained as a result of the extended morphological analysis
A certain evaluation value is calculated in order to evaluate whether or not the keyword / file / ”can be regarded as a correct word string. Here, for example, the average word length in the area that is an error candidate is used as the evaluation value. The average word length of “/ key / word / file /” is 3 as a result of the morphological analysis. On the other hand, "/ keyword / file /" according to the result of extended morphological analysis
Has an average word length of 4.5.

Comparing these results, the word string “/ keyword / file / (average word length = 4.5)” of the result of the extended morpheme analysis is more likely to be the word string “/ key” of the result of the normal morpheme analysis. / Word / File / (Average word length =
3) ”, the evaluation value (average word length) is improved because the average word length of the katakana character string is closer to 4.
Therefore, the “keyword” having a portion different from the character string of the original text is recognized as the correct word of the “keyword” of the original text, and the error display processing unit 55 causes the error candidate and the correct word in the proof text. It is displayed in contrast with the part.

According to this method using the extended morphological analysis, for example, in the "keyword file" portion of the original text, the words "key", "word", and "file" are not unregistered words. Even if an error cannot be pointed out, or if the error probability value assigned to the “keyword file” is smaller than the predetermined error probability threshold value, it is not extracted as an actual error candidate and the error candidate and its The effect is that the correct word can be selected.

[0048]

As described above, according to the present invention,
Instead of directly presenting to each user a clue as to whether or not there is an error, it is possible to present only the part with a high possibility of error based on the result of comprehensive judgment based on multiple clues. Becomes

Further, since the probability of each clue is quantified by the error probability value, the user does not have to worry about the adjustment for each clue and widely extracts the error candidates only by adjusting the error probability threshold value. Therefore, it becomes possible to arbitrarily adjust the balance of whether to eliminate the omission of the error indication point or to extract only the obvious error point to improve the efficiency of the calibration work.

[Brief description of drawings]

FIG. 1 is a diagram illustrating an example of a block configuration according to the present invention.

FIG. 2 is a diagram showing a configuration example of an error probability calculation unit according to the invention described in claim 2;

FIG. 3 is a diagram showing a configuration example of an error probability calculation unit according to the invention of claim 3;

FIG. 4 is a diagram showing a configuration example of an error probability calculation unit according to the invention of claim 4;

FIG. 5 is a diagram showing a block configuration example of the invention according to claim 6;

FIG. 6 is a diagram illustrating an example of error probability calculation.

FIG. 7 is a diagram illustrating an example of error probability calculation.

FIG. 8 is a diagram illustrating an example of error probability calculation and extended morphological analysis.

[Explanation of symbols]

 1 error candidate detection unit 2a, 2b, ... Error probability calculation unit 3 error probability storage unit 4 error probability threshold 5 error candidate extraction unit 6 error display processing unit 20 error probability calculation unit 21 morphological analysis unit 22 unregistered word detection unit 23 unique Noun pattern detection unit 30 Error probability calculation unit 31 Vocabulary extraction unit 32 Categorization unit 33 Probability assignment unit 40 Error probability calculation unit 41 Morphological analysis unit 42 Region division processing unit 43 Probability assignment unit 51 Error candidate detection unit 52 Extended morpheme analysis unit 53 Word comparison unit 54 Word dictionary 55 Error display processing unit

Claims (7)

[Claims] 1. A document proofreading device which points out an error in a text document, calculates an error probability value by a predetermined error probability calculation method for each constituent character or character string of the text, and calculates the respective error probability values. A plurality of error probability calculation means to be provided, an error probability storage means for totaling error probability values obtained from the plurality of error probability calculation means, and determining a final error probability value for each constituent character or character string; and the text Error candidate extraction means for extracting an error candidate by comparing the error probability value given to each of the constituent characters or character strings of 1 above with the error probability threshold value set externally or a predetermined error probability threshold value. And an error display processing means for outputting the extracted error candidates. 2. A morpheme analyzing means for morpheme analyzing text to divide it into word strings in the error probability calculating means,
Among undivided word strings, unregistered word detection means that gives a predetermined error probability value to unregistered parts in a prescribed word dictionary, and syntactic features of the divided word strings are used. Specific pattern detecting means for estimating a portion having a high probability of appearance of an unregistered word, and giving a negative error probability value that lowers the error probability value given by the unregistered word detecting means. The document proofreading apparatus according to claim 1. 3. The error probability calculating means includes a vocabulary extracting means for extracting constituent words from text, a categorizing means for categorizing the extracted word groups based on the similarity between words, and the same category. 2. The document proofreading apparatus according to claim 1, further comprising a probability assigning unit that assigns an error probability value based on the similarity between each word in the word and the number of appearances of the word. 4. A morpheme analyzing means for morphologically analyzing text to divide it into word strings in the error probability calculating means,
Area dividing processing means for dividing text into areas composed of characters of the same kind, and an expected value of word length for each character type, and an average word length in each area composed of characters of the same kind and the expected value. 2. The document proofreading apparatus according to claim 1, further comprising a probability assigning unit that assigns an error probability to the area based on the difference between the two. 5. A program storage medium for storing a program for realizing a document proofreading device for pointing out an error in a text document, wherein the error probability is calculated by a predetermined error probability calculation method for each constituent character or character string of the text. A plurality of error probability calculating means for calculating a value and giving each calculated error probability value, and error probability values obtained from the plurality of error probability calculating means are totaled to obtain a final error for each constituent character or character string. An error probability storage unit that determines an error probability value, an error probability value assigned to each constituent character or character string of the text, an error probability threshold value set from the outside, or a predetermined error probability threshold value. A program that realizes an error candidate extraction unit that extracts an error candidate and an error display processing unit that outputs the extracted error candidate is stored. A program storage medium characterized by the following. 6. A document proofreading device for proofreading a text document, error candidate detecting means for detecting an error candidate by an error probability value calculated by a predetermined error probability calculation method for each constituent character or character string of text, A word comparison unit that calculates the similarity between a character string that has been determined as an error candidate by the error candidate detection unit and a word registered in a predetermined word dictionary, and determines a match when the similarity is greater than or equal to a predetermined value. And an extended morpheme that recognizes the word as a correct word for the character string of the error candidate when the word that is determined to match the character string of the error candidate by the word comparison means improves the evaluation as the error candidate. And an error display processing unit for displaying the correct word recognized by the extended morpheme analysis unit together with the text including the character string of the error candidate. Document proofreading device characterized by. 7. A program storage medium for storing a program for realizing a document proofreading device for proofreading a text document, the error probability being calculated by a predetermined error probability calculation method for each constituent character or character string of the text. The error candidate detecting means for detecting an error candidate based on the value and the character string which is the error candidate by the error candidate detecting means are calculated for the degree of similarity with a word registered in a predetermined word dictionary to obtain a value not less than a predetermined value. If the word comparison means that determines that they are similar when they have similarity and the word that is determined to match the character string of the error candidate by the word comparison means improves the evaluation as an error candidate, then that word is selected. The extended morpheme analysis means for recognizing the correct word for the error candidate character string and the extended morpheme analysis means for recognizing the text including the error candidate character string. A program storage medium for storing a program for realizing an error display processing means for displaying a correct answer word.