JPH01183795A - Dictionary consulting system in post-processing for document reader - Google Patents

Abstract

PURPOSE:To reduce the number of times of retrieving a word dictionary and to improve processing speed by retrieving all words starting at a character on all the characters of an inputted candidate character table from the word dictionary by one consulting with the dictionary. CONSTITUTION:In the word dictionary 12, word information such as a word notation, part of speech information, connecting information with other word is stored. The dictionary 12 is sorted in the sequence of the dictionary and the word starting at the same leading character is present at one place in the dictionary. A dictionary retrieving means 111 retrieves all the words starting at the character of all the characters of the inputted candidate character table from the dictionary by one consulting with the dictionary. On the retrieved word, only the word in which all the characters are present in the candidate character table is selected. Thereby, the quantity of a processing and a time required for collating the character word in a post-processing can be reduced.

Description

[Detailed Description of the Invention] [Summary] This invention relates to a post-processing device for a document reader that reads and converts Japanese documents into a machine-readable format, and improves processing speed by reducing the number of word dictionary searches. With the aim of In the post-processing device that uniquely determines the candidate characters, the input candidate characters are entered into the word matching unit that matches the word formed by combining the characters in the candidate character table with the word dictionary that stores word information such as word notation and part of speech information. A dictionary search means that searches for all words starting with that letter from a word dictionary in one dictionary lookup for all the letters in the table, and selects only those words for which all the letters of the searched words are in the candidate character list. The device is configured to include word selection means.

[Industrial application field]

The present invention relates to a document reading recognition device (document reader device), and particularly to a document reader post-processing device for Japanese text.

Document reader devices are in increasing demand as devices that convert large amounts of printed matter, publications, documents, handwritten documents, etc. written in human-readable character format into machine-readable format. Although it depends on the amount of documents to be processed by this device, it is desired that the processing speed is as high as possible.

The document reader post-processing device uses word information, grammar information, context information, related word information, etc. from the candidate character string output from the character recognition device of the document reader to identify characters that could not be uniquely determined by the character recognition device. It has the function of uniquely determining.

The present invention relates to a method for quickly comparing word information with a word dictionary, which is necessary when using word information among various types of information used in a document reader post-processing device.

[Conventional technology]

Word matching in conventional document reader post-processing devices involves checking all words created by combining candidate characters obtained from character recognition devices against a word dictionary to see if they exist in the word dictionary. there were.

[Problem to be solved by the invention]

In the document reader post-processing device, words formed by combining candidate characters obtained from the character recognition device are checked against a word dictionary. At this time, the candidate characters obtained from the character recognition device are usually -position candidates, 3rd place candidate, -1nth place candidate (n
is often plural, such as any positive integer). The number of words that can be created by simply combining candidate characters is as follows.

Here, k is the number of characters in the word.

n1+n2+...+nk However, it is practically impossible to make k take on an infinitely large number, so in general, the breaks between character types (hiragana, katakana, symbols, mathematical formulas, kanji) are used to separate words. The maximum value of k is determined when determining word candidates based on the experience that there are many words.

For example, as shown in Figure 6, when six kanji characters are determined by the above separation and four candidates are obtained for each character, conventionally the dictionary checks whether all combinations are in the word dictionary or not. It was pulling. That is, as one-letter words, price, lang, sarashi, whether or not there is a version, and as two-letter words, price, price, price, price, price chair, name, name, name, name, etc.
・Whether there is a three-letter word such as price pair, price index, price turn, price chair, price index pair, etc., 4
Whether there is price vs. performance, price vs. performance, etc. as a letter word, price vs. performance, price vs. performance, etc. as a 5 letter word.
. . . 6-letter words such as price vs. sex bear ratio, price vs. sex bear this, . . . are searched. The number of times is 4' + 42 + 4' + 4' + 4'
+4'=5460.

As described above, if the maximum number of characters in a word is large and the number of candidate characters is large, the number of words that can be combined increases, resulting in a problem of poor post-processing throughput.

Also, from the experience that the breaks in the character types mentioned above often serve as word breaks, for example, ``filtration'', ``help'', etc.
Since there are exceptions such as "Class A", there is a problem in that the word matching itself may not go well in such cases.

The problem to be solved by the present invention is to provide a dictionary lookup method in document reader post-processing that eliminates the above-mentioned conventional problems.

[Means to solve the problem]

FIG. 1 shows a block diagram of the principle of a dictionary lookup method in document reader post-processing of the present invention.

In the figure, reference numeral 11 denotes a word matching unit, which matches words formed by combining characters in the input candidate character list with a word dictionary.

A word dictionary 12 stores word information such as word notation, part of speech information, and connection information with other words.

Reference numeral 111 denotes a dictionary search means, which searches the word dictionary 12 for all words starting with all characters in the input candidate character table in one dictionary lookup.

Reference numeral 112 denotes a word selection means, which selects searched words only from words in which all the characters are in the candidate character list.

[For production]

According to the configuration of the present invention, all words starting from a character candidate are searched in one dictionary lookup, thereby reducing access to the word dictionary. The word dictionary is sorted in dictionary order, so
Words that start with the same first letter are grouped together in one place in the dictionary, so the dictionary to be searched is stored in secondary memory (
Even on a magnetic disk device (such as a magnetic disk drive), the number of input/output operations can be significantly reduced compared to conventional methods.

This can significantly reduce the amount of processing and time required for word matching in post-processing.

〔Example〕

The present invention will be explained in more detail below with reference to embodiments shown in FIGS. 2 to 5.

FIG. 2 shows the configuration of a document reader device as a system configuration of an embodiment of the present invention.

In the figure, 6 is an image scanner that reads general printed documents and handwritten documents as images.

Reference numeral 5 denotes a character block cutting device, which cuts out a character block in which a sentence is written from an image obtained from the image scanner 30.

4 is a line cutting device which cuts out line blocks from character blocks.

3 is a character cutting device, which cuts one character from a line block.
Cut out to the letter position.

2 is a character recognition device that performs character recognition and assigns a distance value (an amount for weighting, the smaller the value is judged to be closer to the correct answer) to each candidate character group corresponding to each character. , − candidate, third candidate, ..., n candidate (n is any numerical value).

Reference numeral 1 denotes a document reader post-processing device having a dictionary lookup method according to the present invention, which determines characters that are considered to be correct from candidate character strings obtained from the character recognition device 2.

FIG. 3 is a diagram showing the configuration of a document reader post-processing device according to an embodiment of the present invention.

In the figure, reference numeral 16 inputs the candidate character string output from the character recognition device 2 and stores it in the work area of the document reader post-processing main body 15. This is called a candidate character table.

15 is a post-processing main body, which is a main control unit for post-processing functions.

Reference numeral 17 denotes a post-processing section of the post-processing device, which performs post-processing on words that could not be determined by the post-processing main body 15.

Reference numeral 11 is a word matching unit that employs the dictionary lookup method of the present invention, and 11 and 12 are words formed by combining characters in the candidate character list.
Check against the word dictionary. The word dictionary 12 stores word notations, part-of-speech information, and part-of-speech information of words that can be adjacent to this word (referred to as adjacent information).

Reference numeral 13 denotes a grammar checking unit, which checks whether or not words can be adjacent by referring to the grammar dictionary 14 based on the adjacency information obtained from the word checking unit 11. The grammar dictionary 14 stores part-of-speech information of words that can be adjacent to each other on the part-of-speech side.

FIG. 4 is a flowchart showing the processing of the word matching section of the document reader post-processing device according to an embodiment of the present invention.

■First, reset the word list.

(2) Processes (2) and (2) are performed for all candidate characters at the currently determined position of the candidate character table (the position on the candidate character table of the word to be processed next).

(2) Search the word dictionary based on the key character, that is, one candidate character to be processed, and search for all words starting with the key character. That is, one block (usually 512 to 4096 bytes) of the area on the dictionary containing the word starting from the first character is read.

For all the words searched by ■■, only those words in which all the characters are in the candidate character list are selected, added to the word list, and output.

The word list obtained by this word matching is further narrowed down by checking by the grammar matching section 13.

FIG. 5 is a diagram illustrating an example of a word matching process according to an embodiment of the present invention.

This example is a word matching process in which six Kanji characters are determined as a word by character type separation, etc., four candidates are obtained for each character, and these are input as a candidate character table.

First, words whose first character is "valence", which is the first candidate for character position 1, are searched at once. As a result, four items were obtained: value, price, value, and valuation. Among the words starting with the third-place candidate, ``ro'', only one word for ``corridor'' was found. Among the words starting with the third-place candidate "sarashi", two words were searched: "sarashi" and "sarashi".

Among the words whose first character is "ban" as a circulation candidate, six words are searched: han, hanshita, print, copyright, publisher, and edition number, and a total of 13 candidates are found for words whose first character is a candidate at character position 1. was searched.

Similarly, for words whose first character is the candidate character ``kaku, haku, maki, chair'' in character position 2, a total of 15 candidates, such as kaku and kaku, were searched.

Similarly, 80 candidates such as tai, tai-so, etc. were searched for the word whose first character is the candidate character ``tai, hou, mura, material'' in character position 3.

Similarly, 30 candidates such as gender, personality, etc. were searched for the word whose first character is the candidate character "sex, residence, service, sacrifice" in character position 4.

Similarly, 6 candidates were searched for words whose first character was the candidate character at character position 5, and 40 candidates were searched for words whose first character was the candidate character at character position 6, for a total of 184 candidates.

The above search is for a general word dictionary containing about 70,000 words.

This is a significant reduction compared to the 5,640 candidates in the conventional example shown in Figure 6, and since all words starting with one letter are retrieved in one access, the number of candidates is reduced to secondary memory. The number of accesses is only 16, which is a significant reduction.

In this example, the candidate characters to be input into the document reader post-processing are limited to the top rankings, but if this is made more lenient, the difference is expected to widen further.

Next, for all the searched words, only those words in which all the characters are in the candidate character list are selected. The ones that were rejected as a result are those marked with an X, and the ones that remained are O.
This is indicated by a mark. Next, the word ○ is connected in order from character position 1 to character position 2 to character position 3. If the word searched at character position 1 includes a candidate character at character position 2, it is connected to the word searched at character position 3.

In this way, relatively high-ranking character candidates are used, and the final word, which is underlined in the diagram, is a relatively long word that connects with the candidate character in the next character position. is output as a word list.

〔Effect of the invention〕

As described above, according to the present invention, the number of dictionary lookups can be significantly reduced in the word matching section of the document reader post-processing device, and word matching can be performed at high speed and efficiently. The effect of contributing to the improvement of abilities is extremely large.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of the principle of the present invention; FIG. 2 is a diagram showing a system configuration of an embodiment of the present invention; FIG. 3 is a diagram showing the configuration of a document reader post-processing device of an embodiment of the present invention; FIG. 4 is a flowchart showing a process according to an embodiment of the present invention, FIG. 5 is a diagram showing an example of a word matching process according to an embodiment of the present invention, and FIG. 6 is a diagram showing an example of a word dictionary lookup according to a conventional example. be. In the drawings, 1 is a post-processing device, 2 is a character recognition device, 3 is a character segmentation device, 4 is a line segmentation device, 5 is a character block segmentation device, 6 is an image scanner, 11 is a word matching unit, and 111 is a dictionary search means. , 112 is a word selection means, 12 is a word dictionary, 13 is a grammar matching unit, 14 is a grammar dictionary,
Reference numeral 15 indicates a post-processing main body, 16 indicates a candidate character string input section, and 17 indicates a post-processing post-processing section. Candidate character table Figure 1 is a block diagram of the principle of the present invention.Flowchart illustrating the processing according to an embodiment of the present invention
4 Candidate character table Figure 5

Claims (1)

[Claims] From the candidate character table output from the character recognition device of the document reader, the characters that could not be uniquely determined by the character recognition device are determined by using word information, grammar information, context information, related word information, etc. In the post-processing device that uniquely determines the character, there is a word dictionary (12) that stores words formed by combining the characters in the candidate character list, and word information such as word notation and part-of-speech information.
The word matching unit (11) performs matching with the word dictionary (11), which searches the word dictionary (12) for all words starting with the characters in the input candidate character list in one dictionary lookup. ), and a word selection means (112) for selecting only those words in which all the characters of the searched words are in the candidate character list.