JP4047895B2 - Document proofing apparatus and program storage medium - Google Patents

Description

  The present invention reduces the work of proofreading a document by automatically pointing out an incorrect portion of document data input by a user or acquired as an electronic medium in a document processing apparatus. The present invention relates to a document proofreading apparatus and a program storage medium for realizing the document proofreading apparatus that can greatly improve the work efficiency.

  In conventional document processing devices, there are two methods for pointing out error candidates: (1) pointing out unregistered words from the results, and (2) pointing out words with homonyms. . In addition, (3) when a specific word string is detected, there are some that are recognized as errors and pointed out. This can be done, for example, by using an error detection dictionary in which kana words or expressions that are not used at present are registered in advance, if a matching part is mistaken, or a part-of-speech sequence such as a noun + verb is detected, Or, it is a method of making an error when there is a single kanji word. As another method, (4) Katakana or Kanji character strings in the text are arranged in the dictionary order so that the same words with few notations are continuously arranged, so that the notation can be easily detected. There is what I did. For example, in the case of “window”, “window”, “window”, etc., it is a method used to unify the notation.

  The method (1) of pointing out an unregistered word portion is a method of pointing out that portion because there is a high possibility that there is a misspelled word around the unregistered word portion. However, the reason why unregistered words are born is the presence of original unregistered words that are not mistaken but not registered in the dictionary, such as proper nouns, in addition to spelling errors. Therefore, the pointed out part cannot be determined to be an error, and the user must judge the correctness one by one.

  The method of pointing out a word location where the homonym exists in (2) above is a method of pointing out for the purpose of confirmation because the operation is easy to be mistaken at the time of kana-kanji conversion. In this case, if there is at least one homonym, it is pointed out even if it is not an error in nature, so there is a problem that the number of parts that the user needs to make correct / incorrect judgments becomes very large. is there.

  On the other hand, in the method of pointing out as an error when a specific word string in (3) is detected, it is necessary to register a part of speech string or the like that should be detected as an error in advance, and the target of the error is very limited. Therefore, in practice, many of the errors in the text cannot be detected.

  In addition, in the method (4) of sorting the katakana words and kanji words and presenting them to the user, there is a problem that the work efficiency is not improved so much even though the work amount that the user himself / herself has to do is large.

  As described above, when the conventional method is used, there are many cases where the point indicated by the error candidate is not the original error and tends to be excessively pointed out. As a result, the amount that the user has to recheck increases, which may reduce the efficiency of the calibration work. In addition, there has been a problem that the appropriate selection of error locations is often insufficient.

  In order to solve the above problems, the present invention includes, for example, each unit shown in FIGS.

  FIG. 1 shows a block configuration example of the present invention. In FIG. 1, 1 is an error candidate detection unit, 2a, 2b,... Are error probability calculation units, 3 is an error probability storage unit, 4 is an error probability threshold, 5 is an error candidate extraction unit, and 6 is an error display processing unit. .

  The error probability calculation units 2a, 2b,... Are means for calculating and giving an error probability value to each constituent character or character string of the text by different methods. Each error probability calculation unit 2a, 2b,... May use means as shown in FIG. 2 to FIG. 4 described later alone or in combination, and means for detecting error candidates as described in the prior art. A predetermined error probability may be given to each in combination.

  The error probability storage unit 3 is a means for collecting the error probability values obtained from the error probability calculation units 2a, 2b,... And determining a final error probability value for each constituent character.

  The error candidate extraction unit 5 is a means for extracting a character or character string having a higher error probability value as an error candidate based on a predetermined error probability threshold 4. The error probability threshold value 4 is given as an external input or as electronic data stored in advance.

  The error display processing unit 6 is a means for displaying the error location extracted by the error candidate extraction unit 5.

  With the above configuration, the error probability storage unit 3 aggregates a plurality of error probability values obtained from the error probability calculation units 2a, 2b,..., And calculates the error probability values for each character or character string. Therefore, candidates with a high probability of being erroneous can be extracted more accurately than before.

  Moreover, since the error display processing unit 6 extracts and displays only those that exceed the predetermined error probability threshold 4 by the error candidate extraction unit 5, it is possible to prevent excessive error indication.

  Furthermore, by changing the error probability threshold 4, it is possible to adjust the amount of words to be extracted and displayed as error candidates, so that the accuracy of error indication is maintained depending on the purpose of the proofreading work and the type and content of the original text. Or to improve work efficiency.

  FIG. 2 shows an example of the configuration of the error probability calculation unit.

  The error probability calculation unit 20 includes a morpheme analysis unit 21, an unregistered word detection unit 22, and a proper noun pattern detection unit 23.

  The morpheme analysis unit 21 is a means for performing morpheme analysis on the text, dividing the text into word strings, and passing the word group to the unregistered word detection unit 22 and the proper noun pattern detection unit 23. Since a specific method for morphological analysis is well known in the art, a detailed description thereof is omitted here.

  The unregistered word detection unit 22 detects a word area that is an unregistered word from the word group, assigns a predetermined error probability value to the word area, and passes it to the error probability storage unit 3 of FIG.

  In addition, the proper noun pattern detection unit 23 receives the word string of the morphological analysis result, recognizes an area where the proper noun is likely to appear by using the syntactic feature, and detects the area where the proper noun is likely to appear. And assigns a negative error probability value to the error probability storage unit 3 in FIG. Syntactic features are features that can be inferred from the meanings and parts of speech of the known words before and after, and in many cases, the titles such as “President” and “Director” are often personal names, “City”, This is a feature of a pattern in which words such as kanji strings or katakana that come before the word “state” are often place names. The reason why a negative error probability value is given to a proper noun is that a proper noun is often not an error even if it is an unregistered word. Therefore, it is not necessary to assign a negative error probability value to proper nouns that are not unregistered words.

  In such a proper noun detection, for example, a combination of a proper noun and a specific part of speech or word is registered as analysis data in advance, such as “unregistered word + title”, and the proper noun pattern detection unit 23 What is necessary is just to collate the result of a morphological analysis with these proper noun patterns.

  As a result, it is possible to remove word regions related to proper nouns that are always pointed out as error candidates in the conventional method from the error indication points.

  FIG. 3 shows another configuration example of the error probability calculation unit.

  The error probability calculation unit 30 includes a vocabulary extraction unit 31, a categorization unit 32, and a probability assignment unit 33. The vocabulary extraction unit 31 is a means for extracting a word from the target text and sending it to the categorization unit 32. The categorization unit 32 is means for calculating the similarity of the character composition between words for the word group obtained from the vocabulary extraction unit 31 and categorizing based on the calculated similarity. The probability assigning unit 33 is a unit that assigns an error probability value based on the similarity between words categorized by the categorizing unit 32, the appearance frequency of words, and the like.

  According to the error probability calculation unit 30 shown in FIG. 3, for example, words having high similarity between words are classified into the same category, such as a word causing fluctuation of notation, and the similarity and appearance frequency of the words are classified. Since it is possible to assign as an error probability value which notation word is legitimate or incorrect, it is possible to reflect an error tendency due to the shaking of the notation in an error candidate to be finally displayed.

  FIG. 4 shows another configuration example of the error probability calculation unit.

  The error probability calculation unit 40 includes a morpheme analysis unit 41, a region division processing unit 42, and a probability assignment unit 43. The morpheme analysis unit 41 is a means for analyzing the target text and dividing it into word strings. The area division processing unit 42 is a means for dividing the target text into areas composed of the same kind of characters such as kanji, hiragana, katakana, alphabets,.

  The probability assigning unit 43 uses the stored average word length for each character type to calculate the number of words predicted for each word region of the same character type and the actual number obtained from the word group by morphological analysis. The error probability value based on the difference is assigned to each character or character string. That is, the probability assigning unit 43 considers that the area where the difference between the predicted word count and the actual word count is large includes a word that is not recognized as one word due to a spelling error or the like. Is assigned a relatively high error probability value.

  Each of the means shown in FIGS. 1 to 4 is realized by using a program for operating a computer. The program can be stored in an appropriate storage medium.

  FIG. 5 is a diagram showing a block configuration example of a technique related to the present invention. In FIG. 5, 51 is an error candidate detection unit, 52 is an extended morpheme analysis unit, 53 is a word comparison unit, 54 is a word dictionary, and 55 is an error display processing unit.

  The error candidate detection unit 51 is a unit that detects a character string having a high error probability from the input text, for example, by means similar to the error candidate detection unit 1 shown in FIG. In calculating the error probability, for example, the means shown in FIG. When the word comparison unit 53 compares the similarity between the character string obtained as the error candidate obtained from the error candidate detection unit 51 and the word registered in the word dictionary 54, the word comparison unit 53 has a similarity greater than or equal to a predetermined value. It is a means to judge that it matches. The extended morpheme analysis unit 52 performs extended morpheme analysis on the character string obtained as the error candidate obtained from the error candidate detection unit 51. As a result of the extended morpheme analysis, the evaluation value of the word region is obtained from the error candidate detection unit 51. When it is better than the obtained evaluation value, that is, when the error probability as a whole becomes small, the word (word group) obtained by the extended morphological analysis is recognized as the correct word (correct word group).

  Here, extended morpheme analysis does not perform morpheme analysis using only information in the case of exactly matching a word registered in the word dictionary 54, but does not exactly match but information on words that are quite similar. It also means that morphological analysis is performed using.

  The error display processing unit 55 displays the correct word group obtained from the extended morpheme analysis unit 52 together with the original target text.

  Each means shown in FIG. 5 is realized by using a program for operating a computer. The program can be stored in an appropriate storage medium.

  According to the present invention, a clue as to whether or not there is an error is not directly presented to the user, but only a portion having a high possibility of error based on a result of comprehensive judgment based on a plurality of clues. Can be presented.

  In addition, since the probability of each clue is quantified by the error probability value, the user can extract error candidates widely only by adjusting the error probability threshold without being bothered by adjustment for each clue. It is possible to arbitrarily adjust the balance between eliminating the omission of the portion or extracting only an obvious error portion to improve the efficiency of the calibration work.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 6 is a diagram for explaining an example of error probability calculation by the error probability calculation unit 20 shown in FIG.

  When the original text to be proofread is “Jupe Foreign Minister nods to the opinion”, the morpheme analysis unit 21 shown in FIG. 2 performs a morpheme analysis, and as a result, “/ Jupe / foreign phase / has / its / opinion / The word string “// Unazu / ita /” is passed to the unregistered word detection unit 22 and is also passed to the proper noun pattern detection unit 23 including attribute information such as part of speech.

  The unregistered word detection unit 22 detects “jupe” and “unazu” as unregistered words. 0.5 is assigned to each detected character string of unregistered words as error probability [1]. This value of 0.5 is a value determined in advance by statistical data.

  In addition, the proper noun pattern detection unit 23 detects “/ jupe / external phase /” as matching with “unregistered word + title” which is one of the proper noun patterns registered in advance. Here, “no title” is generally a proper noun, and since a proper noun is usually an unregistered word, the word string “/ Jupe / foreign /” has a low probability of being an error. Become. Therefore, −0.4 is assigned to “Jupe” of “/ Jupe / External Phase /” as a negative error probability [2].

  Based on the error probability value from the error probability calculation unit 20, the error probability storage unit 3 sets the error probability value of the character string “Jupe” to 0.1 (= 0.5−0.4) and the character string “Unazu”. The error probability value of is assumed to be 0.5.

  From the above results, error candidates are extracted in the same manner as the error candidate extraction unit 5 shown in FIG. Here, if the error probability threshold 4 is set to 0.2, for example, the portion of “Jupe” is excluded from the error candidates because “0.1 ≦ error probability threshold (0.2)”. The part “Unazu” is extracted as an error candidate because “0.5> error probability threshold (0.2)”. The extracted error candidates are sent to the error display processing unit 55. The error display processing unit 55 distinguishes the “unazu” part as an error indication part from other parts by underlined display or different color display, and outputs it to a display device or the like as proofreading text.

  As described above, “Jupe” is prevented from being displayed as an error candidate by keeping the error probability value of an unregistered word, which is a proper noun that is not an error, low. Note that the result of the extended morpheme analysis unit 52 in FIG. 5 may be transferred to the error probability storage unit 3 shown in FIG.

  FIG. 7 is a diagram for explaining an example of error probability calculation by the error probability calculation unit 30 shown in FIG.

  It is assumed that words having the same meaning are represented in three different notations of “inverted”, “inverted”, and “inverted” in the original text to be proofread. The vocabulary extraction unit 31 calculates the appearance frequency for each word of the original text, and passes the word and appearance frequency information to the categorization unit 32.

  The categorizing unit 32 classifies words having high character composition similarity in the same category. In this categorization process, for example, “Ba” and “Va” have very high similarity, “Z” and “ヅ” also have very high similarity, and there is a case where there is a long clef “-”. Based on the information that the similarity is large, the same character with and without dakuten has a slightly high similarity, the value of the word Similarity is calculated. As a result, the three words “inverted”, “inverted”, and “inverted” are classified into the same category.

  Next, the error probability is calculated based on the similarity of the character composition between the words in the same category on the basis of the word with the highest frequency. In this example, “inverted” with the highest frequency is recognized as the correct word, and the similarity between “inverted” and “inverted” is 10. In addition, the similarity between the notation “do” and “to” is not so high, and the similarity of “inverted” to “inverted” is 2. Subsequently, an error probability value is determined based on the similarity. Since “inverted” is recognized as a correct word, the error probability is 0, “inverted” having a high similarity to the correct word has an error probability of 0.2, and “inverted” having a low similarity is The error probability is 0.8.

  If the error probability threshold 4 shown in FIG. 1 is set as high as 0.7, the error candidate extraction unit 5 extracts only the character string “invertet” as an error candidate. On the other hand, if the error probability threshold 4 is lowered to 0.1, the error candidate extraction unit 5 extracts both the character strings “inverted” and “inverted” and is recognized as different from the original word. Are displayed in the proofread text as error candidates. This error probability value is an example, and depending on the type of text, it may be better to reduce the error probability of words with low similarity within the same category.

  FIG. 8 is a diagram for explaining an example of error probability calculation by the error probability calculation unit 40 shown in FIG. 4 and processing by the extended morpheme analysis unit 52 and the like shown in FIG.

  When the original text is “designation of key word file”, the morpheme analysis unit 41 shown in FIG. 4 outputs “/ key / word / file / of / designation /” as a result of the morphological analysis. Here, since there are words “key”, “word”, and “file” in the word dictionary, unregistered words are not detected.

  The area division processing unit 42 performs an area division process for each of the same character type, and divides it into three parts: a “keyword file” for the katakana part, “no” for the hiragana part, and “designation” for the kanji part.

For “/ key / word / file /”, the probability assigning unit 43 compares the actual number of words obtained by morphological analysis with the predicted number of words obtained by the region division processing. The predicted number of words is calculated from the average word length statistically determined by the character type. For example, if the average word length of the katakana character string is 4, the predicted number of words in the “keyword file” is 2 (≈9 ÷ 4 ).

  The number of words obtained from the morphological analysis of this part is 3, and the “keyword file” has a mismatch between the actual number of words and the predicted number of words. The probability assigning unit 43 determines that there is a possibility that an error exists in the word area “keyword file” in which the mismatch occurs, and gives an error probability (for example, 0.1) to each character according to the degree of mismatch. To do.

  In the present embodiment, the error probability values from the error probability calculation unit 40 and the error probability values from other error probability calculation units are tabulated by the error probability storage unit 3 shown in FIG. The error candidates are extracted by the error display processing unit 6 and the proofread text indicating the error candidates is displayed.

  In the technology related to the present invention, processing for obtaining a correct word string is further performed by using extended morphological analysis as follows.

  The extended morpheme analysis unit 52 uses the character string “keyword file” detected by the error candidate detection unit 51 and having an error probability value greater than or equal to a certain value as an object of the extended morpheme analysis. The word comparison unit 53 detects “keyword” and “file” from the words registered in the word dictionary 54 as a word string having a high similarity with the “keyword file”, and the extended morpheme analysis unit 52 performs the extended morpheme analysis. As a result, “/ keyword / file /” is extracted.

  In order to evaluate whether “/ keyword / file /” obtained as a result of the extended morphological analysis can be regarded as a correct word string, a certain evaluation value is calculated. Here, for example, the average word length in the area determined as an error candidate is used as the evaluation value. The average word length of “/ key / word / file /” based on the result of the morphological analysis is 3. On the other hand, the average word length of “/ keyword / file /” based on the result of the expanded morpheme analysis is 4.5.

  Comparing these results, the word string “/ keyword / file / (average word length = 4.5)” of the result of the extended morpheme analysis is more the word string “/ key / word / Since the average word length of the Katakana character string is closer to 4 than “file / (average word length = 3)”, it can be seen that the evaluation value (average word length) is improved. Therefore, the “keyword” having a portion different from the character string of the original text is recognized as the correct word of the “keyword” of the original text, and the error display processing unit 55 causes the error candidate and the correct word to be included in the proofread text. Display in contrast with the part.

  According to this method using extended morpheme analysis, for example, in the “keyword file” portion of the original text, the words “key”, “word”, and “file” are not unregistered words, so an error is pointed out. Even if the error candidate is not extracted as an actual error candidate because the error probability value assigned to the “keyword file” part is smaller than the predetermined error probability threshold, There is an effect that selection becomes possible.

It is a figure which shows the block structural example of this invention. It is a figure which shows one structural example of an error probability calculation part. It is a figure which shows another example of a structure of an error probability calculation part. It is a figure which shows another example of a structure of an error probability calculation part. It is a figure which shows the block structural example of the technique relevant to this invention. It is a figure explaining the example of error probability calculation. It is a figure explaining the example of error probability calculation. It is a figure explaining the example of error probability calculation and extended morphological analysis.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Error candidate detection part 2a, 2b, ... Error probability calculation part 3 Error probability preservation | save part 4 Error probability threshold value 5 Error candidate extraction part 6 Error display process part 20 Error probability calculation part 21 Morphological analysis part 22 Unregistered word detection part 23 Specific Noun pattern detection unit 30 Error probability calculation unit 31 Vocabulary extraction unit 32 Categorization unit 33 Probability assignment unit 40 Error probability calculation unit 41 Morphological analysis unit 42 Region division processing unit 43 Probability assignment unit 51 Error candidate detection unit 52 Extended morpheme analysis unit 53 Word comparison unit 54 Word dictionary 55 Error display processing unit

Claims (2)

In a document proofreader that points out errors in text documents,
A plurality of error probability calculating means for calculating an error probability value for each constituent character or character string of the text by a predetermined different error probability calculation method and assigning the calculated error probability value;
Error probability storage means for totalizing error probability values obtained from the plurality of error probability calculation means and determining a final error probability value for each constituent character or character string;
An error candidate that extracts an error candidate by comparing the error probability value assigned to each constituent character or character string of the text with an error probability threshold value set from the outside or a predetermined error probability threshold value Extraction means;
An error display processing means for outputting the extracted error candidates,
And one error probability calculating means among the plurality of error probability calculating means is:
A morpheme analyzing means for analyzing the text and dividing it into word strings;
Area division processing means for dividing the text into areas composed of the same kind of characters;
It has an expected value of the word length for each character type, and the higher the difference between the average word length and the expected value, the higher the difference between the average word length in each region composed of the same type of characters and the predicted value. proofing, characterized in that the error probability value determined for values of said difference such that the error probability value, an error probability calculation means with a probability assigning means for assigning a probability of error value for the region apparatus. A computer-readable program storage medium storing a program for realizing a document proofreading apparatus that points out an error in a text document by a computer,
A plurality of error probability calculating means for calculating an error probability value for each constituent character or character string of the text by a predetermined different error probability calculation method and assigning the calculated error probability value;
Error probability storage means for totalizing error probability values obtained from the plurality of error probability calculation means and determining a final error probability value for each constituent character or character string;
An error candidate that extracts an error candidate by comparing the error probability value assigned to each constituent character or character string of the text with an error probability threshold value set from the outside or a predetermined error probability threshold value Extraction means;
As an error display processing means for outputting the extracted error candidates,
Make the calculator work,
And one error probability calculating means among the plurality of error probability calculating means is:
A morpheme analyzing means for analyzing the text and dividing it into word strings;
Area division processing means for dividing the text into areas composed of the same kind of characters;
It has an expected value of the word length for each character type, and the higher the difference between the average word length and the expected value, the higher the difference between the average word length in each region composed of the same type of characters and the predicted value. the error probability values defined for values of said difference such that the error probability value, as an error probability calculation means with a probability assigning means for assigning a probability of error value for the area,
A program storage medium storing a document proofreading program for causing the computer to function.

Images