JP2001229162A - Method and device for automatically proofreading chinese document - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an automatic proofreading device for a Chinese document, etc., which can efficiently detect and correct an error of a document of Chinese. SOLUTION: A character pair read conversion part converts a source document into a reading symbol string. A candidate word generation part cuts syllables out of the reading symbol string to detect possible candidate words and their relative information, deletes candidate characters by using their frequencies of use, and detects possible candidate words and their relative information by using a prescribed mask means for a reading symbol string and a type combination symbol string of successive single-KANJI candidates. A candidate word determination part detects a corresponding reading symbol string for an obtained combination of candidate characters and detects all possible candidate words and their relative information. An optimum candidate character string determination part generates a directed net according to the start positions and end positions of respective candidate words corresponding to the source document character string and uses a cumulative maximum value of prescribed weight to generate an optimum path. A matching part detects and marks mutually different characters by comparing the optimum path character string with the source document character string.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic document proofreading method and an apparatus therefor, and more particularly, to erroneous or missing characters in a Chinese document.
The present invention relates to an automatic proofreading method and apparatus for automatically detecting and correcting surplus characters.

[0002]

2. Description of the Related Art In recent years, with the progress and spread of electronic computers, it has become necessary to increase the Chinese input speed. For this reason, various Chinese input methods have been proposed. Code-based Chinese input methods can be broadly classified into general keyboards and dedicated input devices. An input method using a general keyboard is shown below. (1) Kanji reading input method, (2) Kanji character form input method,
(3) Input method by combination of character shape and reading, (4)
There are an input method using character codes, and (5) an input method using radicals or the number of strokes. The dedicated input device is, for example, a specialized keyboard, OCR (optical character recognition device), voice input, or the like.

[0003] The Chinese input method provides a method that can eliminate the difficulty in writing Chinese characters. However, in a Chinese document file input from a computer, those that are often mistaken in the past, for example,
Although it is possible to solve the problem of guessing characters and the like, it is still impossible to avoid other errors such as homonyms, similar characters, and missed characters. Generally speaking, the causes of errors in a Chinese document file can be divided as follows.

(1) Inability to correctly input phonetic symbols In general, Chinese phonetic symbols can be classified into initials, finals, vowels, and tones. Vowel: b, p, m,… Intermediate: i, u, yu, iu,… Rhyme: a, o, e… Tone: 1 (one voice), 2 (two voices), 3 (three voices), 4 ( 4 voices), 0
(Light voice) For example, the reading symbol of [shape] is [xing2]. Of which
[x] is the initial, [i] is the vowel, [ng] is the final, and [2] is the tone.

[0005] Here, a set of reading symbols that are often mistaken by Chinese is shown below. Initial part: [sh] and [s] or [q] and [x] etc. Intermediate part: [i] and [yu] Final part: [eng] and [en] or [an] and [ang] etc. Part: Frequently mistaken. It is especially difficult for non-Chinese foreigners to pronounce the correct tone. For example, [interest] (xing4qyu4) is often (xing4qi4)
When it is pronounced, it becomes "genital" when input.

(2) Inability to correctly input a combination of character shapes In the case of character shape input, if a wrong combination is input, a correct result cannot be obtained. The following is an example of the Cangjie input method commonly used in Taiwan. For example, the combination symbol string of the shape of [U] is [Tsuki]. However, if you enter a combination symbol string in the shape of [moon moon Shinsui], the character will become [love]. [Receive]
And [Love] are similar, so they are often mistaken when inputting. In the case of character input by an OCR (Optical Character Recognition Apparatus), the reason why a correct result cannot be obtained is that character combinations cannot be recognized correctly.

(3) Homonymous characters There are 1,345 reading symbols for Chinese characters, and the number of Chinese characters often used in computer systems is 13,053 (for example, BIG code). Therefore, there are several kanji candidates for the same reading symbol for a certain part of Chinese characters. Therefore, when inputting a reading symbol, it is necessary to select a correct kanji from the same phonetic character candidates. For example, [approve group] is erroneously input as [approve group].

(4) Error in reference dictionary In any input method, conversion is performed using a reference dictionary. For example, in the input method mainly using the reading symbols, the reference dictionary is a phonetic dictionary, and in the input method mainly using the combination of character forms, the reference dictionary is the character dictionary. If there is an error in the contents of the reference dictionary, the input result will of course be incorrect.

(5) Error in input operation Generally, various document editing software is used to create a document. However, when using editing functions such as [insert] or [delete], if not carefully operated, there is a high possibility that surplus or missing characters in the document will be caused.

In Chinese document files, typographical errors have a significant effect on document quality. Thus, how to efficiently detect and automatically correct errors in a Chinese document file is an important issue.

[0011] A conventional Chinese document automatic proofing method and apparatus are disclosed in, for example, US Patent Publication No. 260772. FIG. 16 is a configuration diagram of this conventional example. In FIG. 1, reference numeral 100 denotes an input device for inputting a Chinese document to be processed. Reference numeral 110 denotes a Chinese document file storing a Chinese document to be processed. Reference numeral 120 denotes a comprehensive similar character replacement device that converts each character in a Chinese document input with reference to the comprehensive similar character collection into similar characters, and combines a plurality of candidate character strings from the converted similar characters. Reference numeral 130 denotes a comprehensive similar character collection that stores characters similar to the character shape, character sound, meaning, input code, and the like of Chinese characters. The following is an example (S:
Character shape similar, P: character sound similar, M: semantic similar, I: input code similar).

Person: Enter S

[Outside 1] Self: Already-S, Otsu-S, Mi-S D: Ama-P, Inui-P, Sen-Si: Go-S-ji: Osamu S 140 gives the best evaluation to each candidate character string evaluated from each candidate character string. Is a language pattern evaluation device that detects a candidate character string with a high score. 150 is an evaluation device,
It comprises (a) a language pattern statistical database and (b) an evaluation device. (A) The language pattern statistics database includes a Chinese knowledge database that records the usage frequency of each language unit, the connection usage frequency between language units, and records the frequency of each word. (B) The evaluation device attaches a score to one character string based on the language unit and the language pattern statistical database. In addition, the points for characters not in the source sentence document file are reduced. Reference numeral 160 denotes a highest-evaluation candidate character string detection device that detects a candidate character string having the highest evaluation by dynamic programming. Reference numeral 170 denotes an erroneous character judging device that matches the highest evaluation candidate character string with a character string in the document file and marks a different character as an erroneous character by a character matching method. Reference numeral 180 denotes a labeling result output device that outputs a character string indicating the end of the mark to a document file after labeling.
Reference numeral 190 denotes a post-marking document file storing a marked character string.

The operation of this conventional example will be described below.

The input device 100 receives a Chinese document to be processed from the Chinese document file 110. The input Chinese document is divided into several processing units according to the position of the delimiter, and the integrated similar character replacement device 120 is used.
Is input to The overall similar character replacement unit 120 extracts all similar characters having similar character shapes, similar character sounds, similar meanings, or similar input codes by referring to the comprehensive similar character collection 130 for each processing unit, and extracting a plurality of candidate characters. After combining the character strings, the candidate character strings are input to the language pattern evaluation device 140. In the language pattern evaluation device 140,
Each candidate character string is evaluated by the statistical language pattern in the evaluation device 150, and the score is reduced by a language pattern evaluation for characters that are not in the source document file. The highest evaluation candidate character string detection device 160 detects the highest evaluation candidate character string by dynamic programming, and
To enter. The typographical error determination device 170 matches the highest-rated candidate character string with the input source document file one by one to mark a mismatched character as an erroneous character. I do. The labeling result output device 180 outputs the marked character string to the document file 190 after labeling.

[0015]

However, the method and apparatus according to the above-mentioned prior art have the following problems.

(1) Characters that are not registered in the comprehensive similar character collection cannot be detected and corrected. Therefore, creating and maintaining a knowledge base requires many man-hours, resources, and expenses. To get a better proofreading rate, it is possible to create a detailed lexicon, but the increase in the number of candidate characters greatly reduces the operational efficiency.

(2) The language pattern evaluation device considers only the frequency of use of each word and the frequency of connection between language units and does not use semantic information or part-of-speech attribute information, so that the detection rate and the correction rate are high. No.

(3) Omissions, surplus characters in Chinese documents,
Problems such as character order errors cannot be solved efficiently.

(4) Since the existing knowledge base, for example, reference lexicons and dictionaries for the input method cannot be used, development resources are consumed, which is not efficient.

[0020]

SUMMARY OF THE INVENTION An automatic Chinese document proofing method according to the present invention is an automatic Chinese document proofing method on an apparatus provided with an input unit, a storage unit, and an output unit. A broken phonetic character section that stores a break character and its corresponding phonetic symbol together with all candidate characters corresponding to the phonetic symbol and the phonetic symbol, a Chinese character and a default phonetic symbol corresponding to the Chinese character A phonetic dictionary containing pronunciation symbols, Chinese pronunciation symbols, all homonyms, words, all homonyms, usage frequency weights, part of speech attribute symbols and meaning codes A dictionary part, a character-shaped dictionary part storing Chinese characters and a corresponding character-shaped combination symbol string, a part-of-speech information part storing learned part-of-speech connection information, and a learned semantic code of an adjacent succeeding word. Destination A creation step of creating a semantic information unit that stores a combination of the meanings code word, comprising the steps of inputting a source document containing a character string from the input unit,
A character-to-reading conversion step of converting the character string of the source document to a reading symbol string with reference to the broken character dictionary section and the syllabary dictionary section, and cutting out a syllable for the reading symbol string, Referring to the syllabic dictionary section using the syllable as a search key, all possible candidate words and related information are detected, and candidate characters whose use frequency is lower than a threshold are deleted by deletion means, and continuous single kanji characters are deleted. Using a similar sound masking means, a reading sound masking means, and a character shape masking means for the reading symbol string corresponding to the candidate string and its character combination symbol string, the character sound dictionary unit, the character shape dictionary unit, With reference to the phonetic dictionary section, a candidate word generating step of detecting the possible candidate words and their related information, and a combination of all the candidate characters obtained by the similar sound masking means and the character shape masking means. Referring to the syllabic dictionary part, the corresponding reading symbol string is detected, and the detected reading symbol string is used as a search key to refer to the phonetic dictionary part, and all possible candidate words and related information are detected. Candidate word determination step, and forming a directed net by connecting each candidate word using the start position and the end position of each candidate word corresponding to the character string of the source document as an index, And the semantic information section, calculate the part of speech weight and the semantic similarity weight of each candidate word, and calculate the part of speech weight, the semantic similarity weight, the use frequency weight, the word length weight, and the source document similarity weight. Using the cumulative maximum value of the evaluation function as an evaluation function and determining the optimal candidate character string to extract the optimal path by dynamic programming, and matching the extracted character string in the optimal path with the character string of the source document Detecting a grayed to phase Idi, a Chinese document automatic calibration method comprising the matching step marking.

The Chinese automatic document proofreading apparatus according to the present invention comprises:
An input unit for inputting a source document including a character string, a storage unit,
An automatic Chinese document proofreading device on a device having an output unit, the method comprising:
A broken character section that stores all the candidate characters corresponding to the reading symbols and their reading symbols, a Chinese character and a default reading symbol corresponding thereto, and a character dictionary section that stores the possible reading symbols. , A phonetic dictionary that stores Chinese phonetic symbols, all corresponding homonyms, words, all homonyms, word usage frequency, part-of-speech attribute symbols and semantic codes, and Chinese characters and A glyph dictionary for storing a corresponding glyph combination symbol string;
A part-of-speech information section that stores Chinese part-of-speech connection information by a two-dimensional matrix composed of “1” or “0”, and stores a combination of the learned semantic code of the adjacent succeeding word and the preceding word that have been learned. A character-to-reading conversion unit that converts the character string of the source document into a reading symbol string with reference to the broken character dictionary unit and the phonetic dictionary unit, and a syllable for the reading symbol string. And referencing the syllabic dictionary as a search key using the cut-off syllables, detecting all conceivable candidate words and their related information, and deleting candidate characters whose use frequency is lower than a threshold value by deletion means. By using a similar sound masking means, a reading sound masking means, and a character shape masking means for a reading symbol string corresponding to a continuous single Kanji candidate string and its character combination symbol string,
By referring to the character dictionary, the character dictionary, and the phonetic dictionary, a candidate word generator for detecting the conceivable candidate word and its related information, a similar sound mask unit and a character mask unit For the combination of all the obtained candidate words, refer to the phonetic dictionary to detect the corresponding phonetic symbol string, and refer to the phonetic dictionary as the search key using the detected phonetic symbol string as a search key. A candidate word determination unit that detects possible candidate words and related information of the candidate words, and connects and directs each candidate word using the start position and the end position of each candidate word corresponding to the character string of the source document as an index. A net is formed, and the calculating means refers to the part-of-speech information section and the semantic information section to calculate the part-of-speech weight and the semantic similarity weight of each candidate word. Frequency weight and word length weight Using a cumulative maximum value of the source document similarity weights as an evaluation function, an optimal candidate character string determining unit that extracts an optimal path by dynamic programming, and a character string and an original document character string in the extracted optimal path. And a matching unit for detecting and marking a different character by matching characters.

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS "Meaning" is the meaning of a morpheme itself (or a meaning code). In the embodiment of the present invention described below, a thesaurus (1) published by Kadokawa Shoten
985) was adopted. In this semantic classification method, all classification information of one morpheme is classified into hexadecimal of a large classification (first digit), a middle classification (second digit), a small classification (third digit), and a fine classification (fourth digit). This is indicated by a four-digit number. The reason why hexadecimal numbers are used here is that hexadecimal (2 bytes) numbers are widely used in computers, and that hexadecimal numbers can sufficiently correspond to each classification with one digit. This thesaurus describes all kanji, words, etc. in "natural", "character", "fluctuation", "action", "heart",
Divide into 10 major categories of "person", "propensity", "society", "gakugei", and "articles". , It is classified more finely. The following is the expression method.

Meaning Code Description 0 (belongs to “natural”) 02 (belongs to “weather” of “natural”) 028 (belongs to “wind” in “weather” of “natural”) 028a (“belongs to“ natural ”) (It belongs to “Strength” in “Wind” in “Meteorology” of “type.”) In such a hierarchical classification code, the meaning range of the upper meaning code is wider than that of the lower meaning code. In other words, the meaning range of the lower-order meaning code is narrow. Therefore, it is only necessary to use the semantic code according to the actual necessity, and there is no necessity of registering each one even though it is not necessary in advance, and the memory can be saved. For example, to represent weather, only “02” may be used. It is not necessary to register “021,” “0222”... In addition, since the meaning code is represented by a number, when the meaning code is processed by a mathematical operation, for example, a set logical product, it is possible to acquire more valuable information generated from the meaning code. For a detailed description of the meaning codes, refer to the Republic of China Patent Publication N0.161.
Since it is described in No. 238, “Machine Translation Apparatus”, its description is omitted.

There are about 1300 types of Chinese kanji readings, and if they are encoded, they can be completed in at most two bytes (in the case of uniting bytes).
There are 22 vowels, 3 vowels, 14 vowels (vowels), and 5 tones. FIG. 1 shows the structure of 2-byte Chinese kanji reading. The first byte is the initial (bit 2 to bit 6)
And the second byte contains the tone (bits 4-6) and the final (bits 0-3). Thus, for example, a character having the same initial, final, and tone is detected by using the AND operation means while masking the first-byte intervening area.

In order to process similar sounds in each area by the masking means, the distance between bit patterns of similar phoneme elements in each area is 1. An example is shown in FIG.

For a detailed description of Chinese read compact codes and similar bit arrangements, refer to the Republic of China Patent Publication No. 08
See No. 9477, Kanji conversion device (1). When it is desired to deal with the problem of surplus or missing characters due to editing mistakes, the masking means in the present embodiment can mask the above-mentioned bits and also functions as character (byte) masking means. For example, “ting2 ting2 * li4” or “ting2 * yu4 li4” (*: represents a masked character.
That is, any character may be used. ) As a search key to refer to the phonetic dictionary, so that "teitei tamadate" (ting2 ting2 y
u4 li4) can be detected.

In this embodiment, a character masking means is introduced in order to deal with a character problem such as a similar character or a hit character. The mask symbol is “＄”. For example, "E ＄
By referring to the glyph dictionary using the combination symbol string of the glyph "I" as a search key, it is possible to detect characters "" and "" which are similar in glyph. In addition, referring to the glyph dictionary using "YJ ＄＄ J" as a search key, characters with similar glyphs such as "style", "ben",
"Ben" and "Ben" can be detected. The character combination of Chinese characters in the present embodiment employs a character combination method used in the Cangjie input method commonly used in the Republic of China.

It should be noted that the Republic of China Patent Publication No. 0894766
Referring to the Kanji conversion device (2), when converting a phonetic symbol string (phonetic symbol string) to a character string (kanji string), the word length (the number of characters in a word) is one of the important evaluation factors. I have. Also in the present embodiment, the word length weight is one of the evaluation functions. The calculation formula is shown below.

Word length weight = (number of characters of candidate word−1) * 2 For example, when the candidate word is “Sun Moon Lake”, the word length weight is (3-1) * 2 = 4.

In addition, in order to efficiently extract an optimal path using character information in the source document, in the present embodiment, the source document similarity weight is used as one of the evaluation functions. The calculation formula is shown below.

Source document similarity weight = (the number of matching characters in the source document and the corresponding candidate word and having the same character) / the number of characters in the candidate word This embodiment also introduces part of speech attribute information. . For example, as shown in FIG. 8, the part-of-speech attribute information of this word is automatically referred to from the large copas after marking, by referring to the part-of-speech attribute type of adjacent words, for example, part-of-speech attribute information such as nouns, adjectives, and verbs. It is a two-dimensional matrix obtained by learning. “0” indicates that there is no possibility that these two types of part of speech attributes are adjacent. "1" indicates that these two types of part of speech attributes may be adjacent. The formula for calculating the part of speech weight in the evaluation function is shown below.

Part of speech weight = part of speech attribute information value of (part of speech attribute of preceding word, own part of speech attribute) * 0.5 In the present embodiment,
The semantic information of the preceding and succeeding words is also introduced. For example,
As shown in FIG. 9, the meaning of this word is obtained by automatically learning from the large-sized copath after marking and referring to the meanings of the preceding and succeeding words of the word. Alternatively, it is obtained by learning from a collection of documents that have been marked for each area. Since the semantic code adopts a hierarchical definition method, the calculation of the semantic similarity between the preceding word and the following word can be obtained by performing a set AND operation. For example, semantic code
The result of the set AND operation of [7140] and [714a] is
[714]. In this case, since the three codes match, the similarity code similarity is 3/4. When all codes match, the semantic similarity is 1; when two codes match, the semantic similarity is 2/4;
If one code matches, the semantic similarity is 1
/ 4, and 0 when there is no match at all.

FIG. 3 is an example showing the structure of the automatic Chinese document proofreading apparatus according to the present embodiment. In the figure,
250 is a Chinese broken character (separate the character according to its meaning,
This is a broken character section in which a character having two or more pronunciations and a corresponding reading symbol are stored together with all candidate characters and the reading symbol corresponding to the reading symbol. Breaking sound dictionary 2
A conceptual diagram of 50 is shown, for example, in FIG. Referring to FIG. 3 again, reference numeral 260 denotes a phonetic dictionary that stores Chinese characters, their corresponding default phonetic symbols, and their possible phonetic symbols. A conceptual diagram of the data structure of the character dictionary unit 260 is shown in FIG. 5, for example. Referring again to FIG. 3, reference numeral 350 stores Chinese phonetic symbols and all corresponding homonyms, words and all homonyms thereof, word usage frequency weights, part of speech attribute symbols, and meaning codes. This is a phonetic dictionary section. A conceptual diagram of the data structure of the phonetic dictionary unit 350 is shown in, for example, FIG. Referring again to FIG.
Numeral 0 is a character dictionary storing Chinese characters and corresponding character combination symbol strings. A conceptual diagram of the data structure of the character dictionary 450 is shown in, for example, FIG. Referring to FIG. 3 again, reference numeral 540 denotes a part-of-speech information unit that stores the part-of-speech connection information in Chinese using a two-dimensional matrix including “1” or “0”. A conceptual diagram of the data structure of the part-of-speech information unit 540 is shown in, for example, FIG. Referring to FIG. 3 again, reference numeral 550 denotes a semantic information section storing a combination of the learned semantic code of the succeeding word and the semantic code of the preceding word that have been learned. A conceptual diagram of the data structure of the semantic information unit 550 is shown in, for example, FIG. Referring again to FIG.
Reference numeral 0 denotes an input unit for inputting a source document by using an input device such as a hard disk and a keyboard. 200
Is a character-to-reading conversion unit that converts a character string in a source document input from the input unit 100 into a reading symbol string with reference to the above-mentioned broken character dictionary unit 250 and character dictionary unit 260.
Reference numeral 300 denotes a candidate word generation unit. The candidate word generation unit 300
A syllable is cut out from the reading symbol string obtained by the character-to-reading conversion unit 200, and all possible candidate words and related information are detected by referring to the syllable dictionary unit 350 using the cut-out syllable as a search key. A similar sound masking means, a reading sound masking means, a reading sound masking means, Using a mask means, and further comprising the character dictionary 260 and the character dictionary 45.
0, a candidate word generation unit for detecting a possible candidate word and its related information by referring to the phonetic dictionary unit 350;
The candidate word generation unit 300 refers to the character dictionary unit 260 for all combinations of candidate characters obtained by the similar sound masking unit and the character shape masking unit, and detects the corresponding reading symbol string. A candidate word determining unit that detects all possible candidate words and their related information by referring to the preceding phonetic dictionary unit 350 using the read symbol string as a search key. 5
00 is formed by connecting the candidate words using the start position and the end position of each candidate word corresponding to the character string of the source document as an index to form a directed net, and then calculating the part-of-speech information unit 540 and the semantic information A part-of-speech weight and a semantic similarity weight of each candidate word are calculated with reference to the unit 550, and an evaluation function is obtained by calculating the maximum sum of the part-of-speech weight, the semantic similarity weight, the use frequency weight, the word length weight, and the source document similarity weight. Is an optimal candidate character string determination unit for extracting an optimal path by dynamic programming. 60
Reference numeral 0 denotes a matching unit that matches the extracted character string in the optimum path with the original document character string, detects a different character, and then marks the character. Reference numeral 700 denotes an output unit that outputs a character string of the source document after being marked as the optimum character string. A buffer 800 temporarily stores the input character string and the intermediate processing result.

Character-to-reading conversion unit 2 in the present embodiment
FIG. 10 shows the operation flow of 00.

Hereinafter, the operation will be described with reference to FIG.

(S 201) A character string of a source document is input by the input unit 100 and then recorded in the buffer 800.

(S205) The character string of the source document input with reference to the syllabic dictionary unit 260 is cut out for each syllable, and then the process proceeds to (S210).

(S 210) Each syllable recorded in the buffer 800 is taken out, and the non-breaking character is converted into a reading symbol by referring to the syllabic dictionary 260.

(S 215) The character having the broken character recorded in the buffer 800 is converted into the corresponding phonetic symbol by referring to the broken character section 250.

(S220) Each pronunciation symbol is corrected based on the Chinese wording. For example, the reading of “Ma” is “ma”
The tone of the second "Ma" in "Ma Ma" is not read in one voice (the highest of four voices), but is read in a light voice (the unique tone is lost due to the continuous syllables). , Pronounced weakly and lightly), and should be read as “ma0”), so the second “ma” reading symbol is corrected.

Thus, the processing of the character-to-reading conversion unit 200 is completed.

[0042] Candidate word generation section 300 in the present embodiment
11 is shown in FIG.

Hereinafter, the operation will be described with reference to FIG.

(S 301) All syllables that are likely to become syllables for the reading symbol with reference to the phonetic dictionary unit 350 with respect to the original character string obtained by the character-to-reading conversion unit 200 and the corresponding reading symbol string. After cutting out
The process enters (S305).

(S305) Using the cut-out syllables as search keys, the syllable dictionary section 350 is referenced to extract all candidate words, their use frequency weights, part-of-speech connection information, and semantic information. The cflags of all the extracted candidate words are set to 0.

(S310) The start values of the variables i, j, and n are set to 1, 1, respectively, and the number of characters in the source character string is set.

(S315) It is determined whether the i-th syllable is a syllable of a single Kanji candidate (that is, not included in any candidate word). If not a single kanji candidate (S320)
Process. Otherwise, go to (S380).

(S320) It is determined whether the value of the variable i is the same as the variable j. If they are not the same, go to (S330). If they are the same, go to (S325).

(S330) The similar candidate word and its related information are detected by the similar sound mask means using the reading symbol string from the j-th syllable to the i-th syllable as a search key. The cflag of all the detected candidate words is set to 1.

(S335) The process of (S340) is started after deleting those whose usage frequency weight is smaller than the threshold value 0.01 and whose cflag is set to 1. (S340) Using the reading symbol string from the j-th syllable to the i-th syllable as a search key, a candidate word and its related information that can be considered by the reading-sound masking means are detected. The cflag of all the detected candidate words is set to 2.

(S350) With reference to the glyph dictionary section 400, a character combination symbol string of a single Kanji in a source character string corresponding to a reading symbol string from the jth syllable to the ith syllable is extracted, and then the character mask means , The similar word candidate word is detected, and the related information is extracted with reference to the character dictionary 260 and the character dictionary 350. Set the cflag of all the detected candidate words to 3.

(S360) The start position and the end position of each candidate word are determined by referring to the reading of the start and end of each candidate word, or by referring to the reading or character of the source document for the start character and the end character. I do.

(S370) The value obtained by adding 1 to the value of the variable i is set to the value of the variable j.

(S380) When it is determined in the above (S315) that the ith syllable is a syllable of a single Kanji candidate, it is determined whether the value of the variable i is the same as n. If they are not the same, the process proceeds to (S390). Otherwise, (S3
It returns to the process of 20).

(S390) After adding the value of the variable i to 1, the process returns to (S315).

If the value of the variable i is the same as the variable j in the above (S320), the process goes to (S325).

(S325) It is determined whether the variable i has the same value as the variable n. If not the same value as n, (S
337). Otherwise, the process ends.

(S337) After setting the value of the variable j to the variable i, the process returns to (S315).

[0059] Candidate word determination section 400 in the present embodiment
12 is shown in FIG.

Hereinafter, the operation will be described with reference to FIG.

(S401) The value of each of the variables i and n is set to 1 and the number of characters of the character string to be processed.

(S405) The start position is i and cflag
After extracting candidate words whose value is 1 or 3, A [i,
j], j = 0,..., k.

(S410) After the variable m is set to 1, the process proceeds to (S415).

(S415) Start position is m + 1 and cflag
After extracting candidate words whose value is 0, 1, or 3,
B [m, p], p = 0, ..., q.

(S420) It is determined whether B [m] is an empty set. If it is not an empty set, the process proceeds to (S440). Otherwise, go to (S425).

(S440) After adding the value of the variable m to 1, the process returns to (S415).

(S425) The candidate words having the same character string are referred to by referring to the phonetic dictionary section 350 using the character strings of all the connectable combinations in A [i] and B [m] and their reading symbol strings as search keys. After extracting the relevant information, the start position and the end position of the extracted candidate word are calculated with reference to the source document sequence. Then, the process enters (S430).

(S430) The value of the variable i is added to 1.

(S435) It is determined whether the value of the variable i is larger than n. If it is smaller than n, the process returns to (S405). Otherwise, the process ends.

FIG. 13 shows an operation flow of the optimum candidate character string determining section 500 in the present embodiment.

Hereinafter, the operation will be described with reference to FIG.

(S501) After inputting all the candidate words, their related information, the start position, and the end position from the candidate word determination unit 400, the process goes to (S505).

(S505) A directed net of the candidate word is created using the start and end positions of each candidate word as an index.

(S510) The character string of the source document is extracted from the buffer, and the similarity weight and word length weight of the source document are calculated using the start position and end position of each candidate word as an index, and then go to (S515). .

(S515) According to the order from left to right,
The part-of-speech weight and the semantic similarity weight are calculated by referring to the part-of-speech information part 540 and the semantic information part 550 using the part-of-speech attribute symbol and the meaning code of the candidate word of the adjacent connection in the net as an index.

(S520) An optimal path is extracted by dynamic programming using the cumulative maximum value of the part of speech weight, the meaning similarity weight, the use frequency weight, the word length weight, and the source document similarity weight as an evaluation function.

(S525) After the candidate words on the optimal route are extracted and recorded in the buffer 800, the process ends.

Matching section 600 in the present embodiment
14 is shown in FIG.

Hereinafter, the operation will be described with reference to FIG.

(S 601) The character string A on the optimum route is extracted from the buffer 800.

(S605) After fetching the original character string B recorded in the buffer 800, the process proceeds to (S610).

(S610) The character string A and the character string B are matched by the matching means, the erroneous characters and words in the source document are marked, and then the process of (S615) is started.

(S615) The character string of the source document after the marking and the character string A in the optimum path are output to the output unit 70.
Sent to 0.

The operation of the present embodiment configured as described above will be specifically described below by taking as an example a case where "non-compliant person" is input. Input unit 100
When a source document “non-compliant person” is input, the character-to-speech conversion unit 200 refers to the syllabic dictionary unit 260 and converts the non-breaking character part in the input source document into Like reading sign "fuzuen1shou3zhe3i3 law jiou4b
ian4 ", and then referencing the broken character dictionary 250 in the order from the rear to the front, because there is no broken character corresponding to" disobey "," lesser law "," legal " ,
Default reading sound "bu4" for "un" and "law" respectively
And "fa3". Therefore, the reading symbol string in the above character string is converted as follows.

"Bu4 zuen1 shou3 zhe3 i3 fa3 jiou4 bi
an4 "In addition, the source character string is recorded in the buffer 800. Next, the process enters the candidate word generation unit 300. The candidate word generation unit 300 cuts out all syllables that are likely to be syllables for the above reading symbols according to the operation flow shown in FIG. As shown in Table 1 below, all possible candidate words and their related information are detected by referring to the phonetic dictionary unit 350 using the cut-out syllable as a search key.

[Table 1]

Subsequently, candidate words as shown in Table 2 below are obtained for all the candidate words by the similar sound masking means (candidate characters having a usage frequency of less than 0.1 have already been deleted).

[Table 2]

For all the candidate words shown in Table 1, candidate words as shown in Table 3 below are obtained by the reading sound masking means (candidate characters less than 0.1 in use frequency are already used). Has been removed).

[Table 3]

Similarly, candidate words as shown in Table 4 below are obtained by the character shape masking means for all the candidate words shown in Table 1 (candidate characters less than 0.1 in use frequency are already deleted). Has been).

[Table 4]

Next, the candidate word determination unit 400 determines that cflag is 0,
The candidate words shown in Table 5 below are obtained by referring to the phonetic dictionary unit 350 using the reading symbol string of the combination of the candidate words of 1, 3 as a search key.

[Table 5]

Then, the process of the optimum candidate character string determining section 500 is started. First, the optimal candidate character string determination unit 500 creates a directed net as shown in FIG. 15 by connecting each candidate word using the start position and the end position of each candidate word as an index. Then, using a dynamic programming method with the cumulative maximum value of usage frequency weight + primary document similarity weight + word length weight + part of speech weight + semantic similarity weight as an evaluation function, an optimal path “non-observance” as shown in FIG. ―Person-Dependent Character Model 'can be detected. Then, the process of the matching unit 600 is started. Matching unit 6
00 matches the character string of the source document recorded in the buffer 800 with the character string in the optimal path. As shown in Table 6 below, different parts found in the matching are marked with a mark symbol (*: guess letter or erroneous letter).

[Table 6]

Finally, as shown in Table 7 below, the output unit 700 outputs the character string that has been marked as the optimal path character string.

[Table 7]

Although the present invention has been described based on the embodiments, the present invention is not limited to the embodiments, and may be modified as appropriate without departing from the scope of the invention. It is. That is, for example, a character dictionary may be constructed by another character combination method. Also, other reading symbol structures may be used.

[0093]

As described above, according to the method and the apparatus for automatically correcting a Chinese document of the present invention, the conventional problems can be solved and the following effects can be obtained.

(1) Error detection and correction can be efficiently performed on Chinese documents. According to the present invention, the miss detection rate and the correction rate are at least 89% or more.

(2) There is no need to provide a language pattern and a special knowledge database. As a result, many man-hours can be deleted for collection and maintenance of the knowledge database,
Expenses can be deleted.

(3) Applicable to pre-processing and post-processing in the Chinese input method, character recognition device, machine translation device, etc. According to the present invention, it is possible to increase the accuracy of use of the above-described system and apparatus.

As is clear from the above, the utility of the present invention is very high.

[Brief description of the drawings]

FIG. 1 is a structural diagram showing the structure of reading two-byte Chinese kanji.

FIG. 2 is an explanatory diagram illustrating a distance between bit patterns of similar phonemic elements.

FIG. 3 is a block diagram illustrating a configuration of a Chinese document automatic proofreading device according to the present invention.

FIG. 4 is a conceptual diagram of a broken character dictionary of a Chinese document automatic proofreading device according to the present invention.

FIG. 5 is a conceptual diagram of a data structure of a character dictionary section of the automatic Chinese document proofreading device according to the present invention.

FIG. 6 is a conceptual diagram of a data structure of a phonetic dictionary part of the automatic Chinese document proofreading device according to the present invention.

FIG. 7 is a conceptual diagram of a data structure of a character dictionary part of the automatic Chinese document proofreading apparatus according to the present invention.

FIG. 8 is a conceptual diagram of a data structure of a part-of-speech information unit of the automatic Chinese-document proofreading device according to the present invention.

FIG. 9 is a conceptual diagram of a data structure of a semantic information section of the automatic Chinese document proofreading apparatus according to the present invention.

FIG. 10 is an operation flow of a character-to-reading conversion unit of the Chinese document automatic proofreading device according to the present invention.

FIG. 11 is an operation flow of a candidate word generation unit of the Chinese document automatic proofreading device according to the present invention.

FIG. 12 is an operation flow of a candidate word determination unit of the automatic Chinese document proofreading apparatus according to the present invention.

FIG. 13 is an operation flow of an optimal candidate character string determination unit of the Chinese document automatic proofreading device according to the present invention.

FIG. 14 is an operation flow of a matching unit of the automatic Chinese document proofreading apparatus according to the present invention.

FIG. 15 is a diagram illustrating an optimal sutra detected by a process of an optimal candidate character string determining unit.

FIG. 16 is a block diagram showing a configuration of a conventional Chinese document automatic proofing method and apparatus.

[Explanation of symbols]

 Reference Signs List 100 Input unit 200 Character-to-reading conversion unit 250 Breaking character dictionary unit 260 Character dictionary unit 300 Candidate word generation unit 350 Phonetic dictionary unit 400 Candidate word determination unit 450 Character shape dictionary unit 500 Optimal candidate character string determination unit 540 Part-of-speech information unit 550 Meaning information Unit 600 matching unit 700 output unit 800 buffer

Claims (2)

[Claims] 1. A method for automatically proofreading a Chinese language on an apparatus including an input unit, a storage unit, and an output unit, wherein a Chinese abbreviation and a corresponding reading symbol are included in the reading symbol. A broken phonetic dictionary containing all the corresponding candidate characters and their pronunciations, a phonetic dictionary containing Chinese characters and their corresponding default and possible pronunciations, and a Chinese pronunciation symbol , All corresponding homonyms, words, all homonyms, usage frequency weights of words, part-of-speech attribute symbols and semantic codes, and Chinese character strings and corresponding glyph combinations And a part-of-speech information part storing learned part-of-speech connection information, and a semantic information part storing a combination of a semantic code of a learned adjacent succeeding word and a semantic code of a preceding word. A creating step of creating; a step of inputting a source document including a character string from the input unit; and converting the character string of the source document into a reading symbol string with reference to the broken sound dictionary and the phonetic dictionary. Character-to-reading conversion step, and cut out a syllable for the reading symbol string, refer to the syllable dictionary section as a search key using the cut-out syllable, detect all possible candidate words and related information thereof, A candidate character whose use frequency is lower than a threshold value is deleted by a deletion unit, and a similar sound masking unit and a reading sound masking unit for a reading symbol sequence corresponding to a continuous single Kanji candidate sequence and a character combination symbol sequence thereof. A candidate word generating step of detecting the possible candidate word and its related information by referring to the character dictionary, the character dictionary, and the phonetic dictionary using a character mask unit. For all combinations of candidate characters obtained by the similar sound masking means and the character shape masking means, refer to the character sound dictionary section to detect a corresponding reading symbol string, and detect the detected reading symbol string. Referring to the phonetic dictionary section as a search key,
A candidate word determining step of detecting all possible candidate words and their related information; and connecting each candidate word by using the start position and the end position of each candidate word corresponding to the character string of the source document as an index. A directional net is formed, and the calculating unit refers to the part-of-speech information part and the semantic information part to calculate the part-of-speech weight and the semantic similarity weight of each candidate word. An optimal candidate character string determining step of extracting an optimal path by dynamic programming using a cumulative maximum value of a use frequency weight, a word length weight, and a source document similarity weight as an evaluation function; Matching a character string with the character string of the source document to detect a different character,
And a matching step of marking. 2. An automatic Chinese document proofreading apparatus on an apparatus having an input unit for inputting a source document including a character string, a storage unit, and an output unit, the method comprising: A phonetic alphabet that stores the reading symbols together with all the candidate characters and the reading symbols corresponding to the reading symbols, a Chinese character, a default reading symbol corresponding thereto, and its possible reading symbols. Character dictionary, and Chinese phonetic symbols, all corresponding homonyms,
A phonetic dictionary section that stores words, all homonyms thereof, frequency of use of words, part-of-speech attribute symbols, and semantic codes; a character shape dictionary section that stores Chinese characters and corresponding character combination symbol strings; A part-of-speech information unit that stores Chinese part-of-speech connection information in a two-dimensional matrix composed of "1" or "0", and stores a combination of the learned semantic code of the adjacent succeeding word and the semantic code of the preceding word. A semantic information unit, a character-to-reading conversion unit that converts the character string of the source document into a reading symbol string with reference to the broken character dictionary unit and the phonetic dictionary unit, and a syllable for the reading symbol string. With reference to the syllabic dictionary section as a search key using the cut-out syllable as a search key, all possible candidate words and related information are detected, and candidate characters whose use frequency is lower than a threshold value are deleted by a deletion unit. The similar character masking means, the reading sound masking means and the character shape masking means are used for the reading symbol string corresponding to the continuous single Kanji candidate string and its character combination symbol string, and the character dictionary section is deleted. A candidate word generation unit that detects the possible candidate words and related information with reference to the glyph dictionary unit and the phonetic dictionary unit; With respect to the combination of candidate words, the corresponding phonetic dictionary portion is detected by referring to the phonetic dictionary portion, and the detected phonetic dictionary portion is referred to as the search key using the detected phonetic symbol sequence,
A candidate word determining unit that detects all possible candidate words and their related information; and a concatenation of each candidate word using the start position and the end position of each candidate word corresponding to the character string of the source document as an index. A directional net is formed, and the calculating unit refers to the part-of-speech information part and the semantic information part to calculate the part-of-speech weight and the semantic similarity weight of each candidate word. An optimal candidate character string determining unit for extracting an optimal path by dynamic programming using a cumulative maximum value of the usage frequency weight, the word length weight, and the source document similarity weight as an evaluation function; An automatic Chinese document proofreading apparatus comprising: a matching unit that matches a character string with a source document character string to detect and mark a different character.