JPS6394364A - Automatic correction device for wrong character in japanese sentence - Google Patents

Abstract

PURPOSE:To attain the automatic correction of a wrong word, by generating a Japanese sentence correction candidate character dictionary, and a character concatenation probability dictionary by using the same kink of document in which no wrong character is included in advance. CONSTITUTION:For an input Japanese sentence data base 3 that is a read result in an input device 1, a wrong character including area is detected by a wrong character detecting part 6 which performs a morphemic analysis such as the extraction of a word candidate, and the qualification of the connection of a part of speech, and furthermore, the position of the wrong character is detected by using the character concatenation probability dictionary 8, and after that, the Japanese sentence correction candidate dictionary 10 is indexed setting a peripheral character other than the character at the position of the wrong character as a key, and a correction candidate character is extracted. Furthermore, the orthographic string probability of a tentative character string in which as correction candidate is inserted at the position of the wrong character in order is found by using the character concatenation dictionary 8, and the correction candidate is squeezed by performing ranking and cutting, and then, the morphemic analysis is applied again on the tentative character string of a clause level inserted to an original sentence for each correction candidate, and grammatical checking is performed, thereby, a grammatically true correction candidate can be selected.

Description

[Detailed description of the invention] (Technical field to which the invention pertains) The present invention is for automatically correcting typographical errors contained in Japanese character strings containing kanji and kana read from an input device in order to create a Japanese document database. The present invention relates to an automatic correcting device for Japanese typographical errors that selects correct candidates from a group of extracted candidate characters and automatically corrects them.

(Conventional technology) When creating a Japanese document database by converting a large amount of Japanese documents such as newspaper articles, publication manuscripts, and scientific and technical papers into electronic files, it is necessary to avoid rejecting characters, misread characters, and typos that are mixed into the reading results. are detected by morphological analysis using a word dictionary and grammar dictionary and checking by a corrector, but in order to correct or automatically correct them, it is necessary to extract candidates with a high percentage of correct candidates and to perform a check based on morphological analysis. Strict Japanese language checking is required.

Examples of traditional candidate extraction and automatic correction are:

Discussing spell checking and its correction in English, J.
, L. Paterson ““Lecture Notes computer
5science Vol, 96Computer
Programs For Spelling
Correction”, Springer-Verl
As described in "Published by Ogsha, 1980", ■ A method of storing correct spellings and incorrect spellings in pairs in a dictionary, and replacing incorrect spellings with correct spellings when detected.

■ Statistically 2 from the correct spelling of English words
There is a method of generating an incorrect spelling list applying the rules of character replacement, one character insertion, one character missing, and one character error, and extracting and correcting the correct spelling when the incorrect spelling is detected.

A method of detecting and automatically correcting typographical errors by converting both correct and incorrect patterns into a dictionary is described in ``Sentence Abnormality Inspection and Correction Apparatus'' disclosed in Japanese Patent Laid-Open No. 1787-1987.

However, these conventional methods have the following problems.

■ In Japanese documents that have many character types and are not separated, the number of incorrect character string patterns used to detect typographical errors is enormous.
It is difficult to extract candidates across clause boundaries.

■ If the error characteristics of the input device are unknown, it is difficult to extract candidates and correct them.

■ When selecting correction candidates by morphological analysis, the processing load increases if a large number of correction candidates are extracted. Furthermore, even if the correction is done manually, it will result in a processing load because it includes many correction candidates that are clearly grammatically incorrect.

(Objective of the Invention) The object of the present invention is to create a Japanese sentence correction candidate character dictionary and a character concatenation probability dictionary using a large number of documents of the same type that do not contain typographical errors in advance, and when a typographical error is detected, to correct the Japanese sentence. By extracting candidates using a candidate character dictionary and narrowing down the candidates by calculating the correct character string probability of the temporary character string created by each of these correction candidates using a character concatenation probability dictionary, the number of character types is 1.
An object of the present invention is to provide an automatic correction device for spelling errors in Japanese characters, which performs candidate extraction and automatic correction with high processing performance independent of bunsetsu boundaries, number of typographical errors, and error characteristics of an input device.

(Structure of the Invention) (Characteristics of the Invention and Differences from the Prior Art) The present invention provides a character string of N characters that is extracted using a large number of documents that do not contain the same type of typographical errors as the document that is the target of automatic typographical correction. Alternatively, extract a character string pattern of specific N characters, N-1 characters, ..., 2 characters selected from these, and extract characters other than the i-th (i = 1 ... N) character in each of these character strings. Based on the Japanese correction candidate character dictionary in which the i-th character is collected as a correction candidate character when the character string patterns of If a character conjunctive probability dictionary for N characters is created in advance, and a typographical error contained in the input Japanese sentence database input from the input device is detected by morphological analysis using a word dictionary and a grammar dictionary, the characters before and after the typographical error are The Japanese sentence correction candidate character dictionary is indexed using surrounding character string patterns as keys, correction candidate characters are extracted, and each correction candidate character is inserted into the incorrect position in the original text.The correct character string probability of the temporary character string is calculated as a character concatenation probability dictionary. Calculate and rank correction candidate characters, narrow down by cutting,
It is characterized by selecting correct candidates from the narrowed-down correction candidates through typographical error detection processing using morphological analysis and automatically correcting them.

The conventional technology is: ■ Candidate extraction is performed using a Japanese sentence correction candidate character dictionary, so it is possible to extract candidates that are independent of the number of character types in the document, the presence or absence of separation, and the error characteristics of the input device.

■ Since the extracted candidates are narrowed down by applying a character concatenation probability dictionary, highly accurate candidate extraction is possible.

■ Typos can be automatically corrected by checking the grammar of narrowed down candidates.

■ Since candidates for correction of obvious grammatical errors are automatically eliminated, the burden of manual correction can be reduced. The difference is that.

(Example) Fig. 1 shows an example of the basic configuration of the present invention, and 1 is a kanji OCR,
Input devices such as Bentouch, tablet, keyboard, etc., 2
3 is an input processing unit that performs input or reading; 3 is an input Japanese database of reading results read by the input device 1 and recorded in the form of character codes in the magnetic device; 4 is a word dictionary; 5 is a grammar dictionary; 6 7 is a typo detection unit that extracts a typo-containing area at the bunsetsu level containing a typo by morphological analysis using the word dictionary 4 and grammar dictionary 5; 8 is a character concatenation probability dictionary; 9 is a correction candidate character extraction unit that indexes the dictionary 10 and extracts correction candidate characters for the typo; lO is a Japanese sentence correction candidate character dictionary; 11 is an index for the dictionary 10. a correction candidate narrowing section that ranks each correction candidate and cuts the candidates;
Reference numeral 2 refers to a correction candidate checking unit that performs morphological analysis to check the provisional character string in which correction candidates are inserted at the misspelled positions in order from the top, and 13 refers to a corrector that uses correction candidates that have been certified by the grammatical check of the correction candidate checker 12. 14 is a correction terminal, 15 is a database of Japanese documents in which typographical errors have been corrected, and 16 is a processing device consisting of a CPU/memory.

In this method, a typographical error detection unit 6 performs morphological analysis such as word candidate extraction and part-of-speech connection test on an input Japanese sentence database 3 that is the reading result read by the input device 1, and detects areas containing typographical errors. After detecting a typographical error position using the conjunction probability dictionary 8, correction candidate characters are extracted by indexing the Japanese sentence correction candidate dictionary 10 using surrounding characters other than the typographical error position as keys.

Furthermore, using the character concatenation probability dictionary 8, calculate the correct character string probability of the temporary character string in which correction candidates are inserted in order at the error positions, and after narrowing down the correction candidates by ranking them,
For each correction candidate, the clause-level temporary character string inserted into the original text is morphologically analyzed again, a grammatical check is performed, and a grammatically correct correction candidate is selected.

If the correction candidates are narrowed down to one at this point, the candidate is replaced with the typographical error and automatic correction is successful. However, if there are still a plurality of candidates, the corrector selects a candidate from the correction terminal 14.

FIG. 2 shows an example in which there are two typographical errors in the extraction of correction candidates and automatic correction after detecting typographical errors in the basic configuration example of FIG. 1.

In this example, 17 is a typo-containing area where a typo was detected, 18 is a typo, 19 is a correct character, 20 is a typo position detected by the typo position detection unit 7 using character concatenation probability, and 21 is a target for a candidate for a typo position. Correction candidates, 22 are correction candidates narrowed down from the correction candidates 21 based on character concatenation probability, 23 and 24 are correction candidates for the typographical position candidate ■ and narrowed down correction candidates, 2
5 is a correction candidate string ranked by the correction candidate narrowing section 11, 26 is a candidate order, 27 is a top two candidates in candidate order among the correction candidate string 25, and 28 is a character string automatically corrected after checking correction candidates.

In this example, a plurality of typo candidates are extracted by detecting typo positions in the typo-containing area 17 using the character concatenation probability dictionary 8, and then correction candidates are extracted for each typo.

Next, each correction candidate at each typo position is inserted into the original string in order, the character concatenation probability of the string is calculated, and candidates with low probability are cut off to obtain correction candidates 22 and 24, and further ranked. Then, a correction candidate sequence 25 is created.

After performing morphological analysis on these correction candidates again and performing a grammatical check on each correction candidate, automatic correction is performed when the number of candidates is 1.

However, if there are two or more candidates and the top two candidates in the candidate order are one typo position, the top one correction candidate is embedded in that position, and the top two candidates in the candidate order are separate typo positions. In this case, it is assumed that there is a two-character error, and both candidates are embedded in each error position.

However, if three or more plurals remain after checking the grammar of the correction candidate column 25, the correction candidate selection unit selects the correction candidate from the correction terminal 14 without selecting the correction candidate with the highest rank in the candidate order.
The corrector is provided with means for the corrector to select these correction candidates from among them, and means for the corrector to make corrections when automatic correction candidate extraction fails.

Because of this structure and operation, compared to conventional techniques, it is possible to extract candidates that are independent of the number of character types, the presence or absence of separation, the number of typographical errors, and the error characteristics of the input device. Candidates are narrowed down based on character concatenation probabilities and a grammar check is performed on these candidates, so the accuracy rate of candidates is high, automatic correction is possible, and the time required for processing can be reduced.

Furthermore, since correction candidates are narrowed down by grammar checking, there has been an improvement in that the load can be reduced even when candidates are selected manually.

(Effect of the invention) As explained above, by using a large number of documents that do not contain the same type of typographical errors as the Japanese document targeted for automatic typographical error correction, a character string of N characters to be extracted or a specific character string selected from these can be extracted. N characters, N-1 characters,...

A Japanese sentence correction candidate character dictionary that extracts a two-character character string and uses it to extract correction candidate characters, and a character conjunctive probability dictionary of N characters calculated based on the appearance frequency of the N characters are created in advance. When a typographical error in the input Japanese sentence database read into the input device is detected by morphological analysis, correction candidates are extracted using the correction candidate character dictionary, candidates are ranked using the character concatenation probability dictionary, and the candidates are narrowed down by cutting, On the other hand, automatic correction is performed by checking the grammar using morphological analysis. ■ Candidates with high accuracy are extracted and narrowed down without depending on the number of character types, the presence or absence of separation, the number of typos, and the error characteristics of the input device. Extraction possible.

■ By performing a grammatical check on a small number of narrowed down candidates, it is possible to automatically correct typos.

■ Since correction candidates for grammatical errors are automatically eliminated, the processing load can be reduced even when correcting by one person.

There is an advantage.

[Brief explanation of the drawing]

FIG. 1 shows an example of the basic configuration of the present invention, and FIG. 2 shows an example of extracting correction candidates and automatically correcting typographical errors. 1...Input device, 2...Input processing unit, 3...
・Input Japanese sentence database, 4...word dictionary, 5...grammar dictionary, 6...
Misprint detection section, 7... Misprint position detection section. 8...Character concatenation probability dictionary, 9...Correction candidate character extraction unit, 10...Japanese sentence correction candidate character dictionary, 11...Correction candidate narrowing down unit. 12...Correction candidate checking section, 13...Correction candidate selection section, 14...Correction terminal, 15...Japanese document database from which typographical errors have been corrected, 16...Processing device, 17... Area containing typographical errors, 18・
... Misprint, 19... Correct character, 20... Misprint position, 21... Correction candidate for misprint position candidate ■, 22...
・Correction candidates narrowed down from 21, 23... Correction candidates for the typographical error position candidate ■, 24... Correction candidates narrowed down from 23, 25... Ranked correction candidate sequence, 26... Candidate order, 27 ...Top 2 candidates in 25, 28...Character string automatically corrected after checking correction candidates. Figure 1 17... agu including chrysanthemum bandits Old knowledge 5 19. Engineering center L8 20 Narrow position 2I...
1 town arc 1 Higashibi ■ 1 Ku Nai Kotoru j1 Gojishiba Watarucho 24
23ri・6autumn QLfj! Ding t, Hou^h “25 Order 1
Kivuijet Kyouoke°zu・126・Sanosaifusute

Claims (1)

[Scope of Claims] For characters that are rejected or misspelled characters caused by input errors or character recognition errors in Japanese sentences in a Japanese document database input from a character input device, misspellings are included by morphological analysis using a word dictionary and a grammar dictionary. A typographical error detection unit extracts a typographical error area at the bunsetsu level, and a typographical position detection unit extracts a character position that is considered to be a typographical error based on the probability of inter-character conjunction from this typographical error area. A character string of N characters extracted using a document that does not contain typographical errors, or a specific N character selected from these, N-1 character,..., the i-th character string (i = 1... - A Japanese correction candidate character dictionary that collects the i-th character when the patterns other than the character N) are the same as a correction candidate character, and the characters around the typo position extracted by the typo position detection unit as keys. A correction candidate character extraction unit that indexes the Japanese sentence correction candidate character dictionary and extracts correction candidate characters for misspellings, and a character concatenation of each N character calculated in advance based on appearance frequency information regarding the pattern of N characters extracted in advance. a character conjunction probability dictionary that holds probability information using each N character as a key; a correction candidate narrowing section that uses the character conjunction probability dictionary to rank and narrow down correction candidates extracted by a correction candidate character extraction section; A correction candidate checking section that inserts corrected correction candidates into the original character string and performs a grammatical check by morphological analysis, and a correction candidate selection section that allows a corrector to select correct characters from the correction candidates that have undergone the grammatical check. The automatic sentence error correction device includes means for automatically correcting correction candidates by extracting correction candidates for detected typos using the candidate dictionary, narrowing them down using the probability dictionary, and further performing a grammar check using morphological analysis. An automatic correction device for Japanese typographical errors.