JPH11102414A - Method and device for correcting optical character recognition by using bitmap selection and computer-readable record medium record with series of instructions to correct ocr output error - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user without having to look at the document of an original paper at the time of correcting an error that occurs, while an original text is converted into a text of an OCR output. SOLUTION: An document image 210 that is a source of optical character reader(OCR) output is displayed. A recognition likelihood parameter is decided with respect to an area of a document image that corresponds to a work in the OCR output. The areas are displayed so that they represent respective recognition likelihood parameters (for instance, emphasized by various colors). It is preferable that a user be able to select an area where the displayed document image exits. When the area is selected, a text of the OCR output which corresponds to the selected area is shown in a pop-up menu.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to optical character recognition, and more particularly, to a method and apparatus for finding errors in optical character recognition output, and recording a series of instructions for finding errors in OCR output. Computer-readable recording medium.

[0002]

BACKGROUND OF THE INVENTION Obtaining text and graphics from paper documents is a significant problem for many industries. For example, a publisher may print hundreds or thousands of scholarly articles throughout the year. Often a publisher starts with a paper document that must be entered into the publisher's computing device. One conventional approach is to employ a keyboard enthusiast to read a paper document and type the document into a computer system. However, entering a document is time consuming and costly.

[0003] Optical character recognition (hereinafter referred to as OCR)
Is a technology that guarantees benefits to the publishing and other industries. The reason is that the input processing speed of the OCR device is much higher than the input speed of the keyboard input person. Thus, publisher employees often begin work with scanned documents. The document has been converted by the OCR device into a computer readable text format, for example, ASCII.

[0004] However, even with the high recognition rates (often over 95%) possible with modern OCR devices,
It is not enough for industries such as the publishing industry that require high accuracy. Thus, publishers typically employ a proofreader to manually modify the OCR output.

[0005]

SUMMARY OF THE INVENTION However, OCR
Calibrating the output manually is very time consuming,
It is also difficult for humans to do. The proofreader must compare the original paper document with the printed or screen display of the OCR output and compare them word by word. Even if the recognition rate is high, when a person calibrates the OCR output, one person tends to miss and miss an error.

Another conventional option is to spell check the resulting computer readable text. However, it does not recognize all misspelled words. In addition, the entered words may be very distorted, so the proofreader must refer back to the paper text throughout the entire spell check. Once the proofreader sees the text on the paper, he decides the correct word, and the correct word is OCR
Hit the key on the text of the output and enter it. This approach has been found to be time consuming and somewhat error prone.

It is an object of the present invention to eliminate the need for a user to view an original paper document in correcting errors made while converting the original text to OCR output text. is there.

[0008]

SUMMARY OF THE INVENTION There is a need to facilitate human calibration of OCR output. Especially OC
There is a need to reduce the time spent configuring the R output.

[0009] These and other needs are successfully addressed by the present invention. The characters of the document image obtained from the original paper document are recognized (e.g., via OCR) to generate the document text. The regions of the document image corresponding to the words of the document text are determined, and recognition accuracy parameters are determined for each region. The regions of the document image can be displayed in a manner that indicates the respective recognition parameters. As the user progresses through the document text, the document image is displayed in the same way.

[0010] Preferably, the user can select a location in the document image. The word selected is determined by the document area containing the selected location in the document image and the display, for example, in a pop-up menu. In addition, the recognition accuracy parameter may be compared to one or more thresholds and displayed in a color corresponding to the exceeded threshold.

[0011] Additional objects, advantages and novel features of the invention will become apparent in the detailed description which follows, and in which part will be apparent upon inspection, or will be learned by practice of the invention. . The objects and advantages of the invention will be realized and attained by the methods and devices pointed out in the appended claims.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A method and apparatus for correcting optical character recognition using a heat map according to the present invention and a series of instructions for detecting an error in an OCR output are described below. A computer-readable recording medium will be described with reference to specific examples illustrated in the accompanying drawings, but the present invention is not limited thereto. In addition, in the drawings, components denoted by the same reference numerals represent similar components.

A method and apparatus for finding an error in an OCR output will be described. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It is apparent, however, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

FIG. 1 is a block diagram of a computer system 100 on which an example of the present invention can be implemented. Computer system 100 includes a bus 110 or other communication means for communicating information, and a processor 112 connected to bus 110 for processing information.
Further, computer system 100 may include a random access memory (RAM) or other dynamic storage device 11.
4 (shown as main memory),
The main memory 114 stores information and the processor 11
2 to store instructions to be executed by
10 is connected. Main memory 114 may also be used to store temporary variables and other intermediate information while processor 112 is executing instructions. Computer system 100 also includes one or both of read-only memory (ROM) and other static storage devices 116, which are connected to bus 110 and store static information and instructions for processor 112. Data storage device 118 is, for example, a drive for a magnetic or optical disk and the equivalent disk, and may be connected to bus 110 for storing information and instructions.

An input / output device can be connected to the computer system 100 via a bus 110. For example, the computer system 100 uses a display device 120, such as a cathode ray tube (CRT), to display information to a computer user. Further, the computer system 100 uses a keyboard 122 and cursor control means 124 such as a mouse. In addition, computer system 100 may use a scanner 126 to convert paper documents into a computer readable format. Furthermore, the computer system 100 may store the document image generated by the scanner 126 or the main memory 114 or the storage device 118
The OCR device 128 can be used to recognize characters in a document image stored in the OCR. Or O
The function of the CR device 128 is performed by executing the instructions stored in the main memory 114 by the processor 112.
It can be implemented in software. In yet another example, the scanner 126 and the OCR device 128 can be combined into a single device designed to scan a paper document and recognize the characters there.

The present invention relates to using computer system 100 to find errors in OCR output. According to one embodiment, finding errors in the OCR output is performed by computer system 100 in response to processor 112 executing a sequence of instructions stored in main memory 114. Such instructions may be read into main memory 114 from another computer-readable medium, such as data storage device 118. By executing a series of instructions stored in the main memory 114, the processor 11
2 performs the processing steps described below. In another example, hardwired circuitry may be used in place of, or in conjunction with, software instructions to implement the present invention. Accordingly, the present invention is not limited to any particular combination of hardware circuits and software.

(Synthesized Document Architecture) A synthesized document is
It has multiple representations of a document and treats the multiple representations as a logical whole. The composite document 200 is shown in FIG.
As shown in FIG. 3, the information is stored in a memory such as the main memory 114 or the storage device 118 of the computer system 100.

The composite document 200 includes a document image 210, which is a bitmap representation of a document of the document (eg, a TIFF file generated from the scanner 126). For example, a copy of the United States Constitution may be read and scanned by scanner 126 to produce an image of the United States Constitution in the form of a document image 210.

A bitmap representation is a row of pixels,
Monochrome (eg black and white) or multicolor (eg red, blue,
Green, etc.). Therefore, the position of the rectangular area of the document image 210 can be specified, for example, by combining the upper left corner and the lower right corner of the rectangle. In the example of reading and scanning the United States Constitution, the first letter of the word "form" in the preceding sentence may be located within a rectangle whose upper left corner is (16,110) and lower right is (31,119). Good.
Therefore, the last character of the same word is (16,14)
The lower right of the coordinates of (0) can be arranged in the rectangle of the coordinates of (31,149).

The composite document 200 is composed of the document text 22
0 and a correlation table 230, which may be generated by the method shown in the flowchart of FIG. The document text 220 is encoded ASCII, E
It is made up of a series of 8-bit or 16-bit bytes that encode characters in BCDIC or Unicode. Thus, characters in the document text 220 may be located within the document text 220 by offset. In the above example, the first character of the word “form” in the preamble may be located at offset 57 and the last character of the same word may be located at offset 60, as represented in the offset column of the correlation table 230.

Referring to FIG. 3, step S25 will be described.
0, the characters in the document image 210 are
8 or equivalent and is stored in the document text 220 in step S252. Also O
The CR device 128 is designed to output the coordinates of the recognized character in the document image in step S250. Thus, characters recognized at known offsets in the document text 220 may be associated with regions of the document image 210. In the above example, the character at offset 57 is associated with the area defined by coordinates (16, 110) and (31, 119).

In addition, some embodiments of the OCR device 128 are designed to output a recognition accuracy parameter that indicates the likelihood that the recognition result is correct, as is well known in the art. For example, in one font, the character "rn" in the document image 210 may be recognized as a character "m" that has the potential to be determined. In this case, the OCR device 128 may output a 60% recognition accuracy parameter for the character pair, for example.

In step S254, the document text 220
Is specified by, for example, interpreting characters between spaces as words. In step S254, the area of the document image 210 corresponding to the character of the word is merged into one area corresponding to the complete word of the document text 220. In one embodiment, the document area is specified as a rectangle having the upper left coordinates and the lower right coordinates of the coordinates of the area corresponding to each character. For example, the area corresponding to the word “form” in the previous sentence is represented by coordinates (1
6,110) and (31,149). Alternatively, especially for documents having characters of various sizes, the coordinate list for all basic characters may be saved.

When a word is specified, a recognition accuracy parameter for the word is calculated from the recognition accuracy parameters of all characters or character pairs. Preferably, the recognition accuracy parameters for the words are calculated by multiplying the recognition accuracy parameters based on the individual characters.
When recognizing the word "form", the letters "f" and "o" have very high recognition accuracy parameters (eg, 95% and 90%), but the letter pair "rm" has a 60% It may only have the recognition accuracy parameter. When these recognition accuracy parameters are multiplied, the overall recognition accuracy parameter is 51.3%.
Alternatively, in another calculation, for example, a minimum recognition accuracy parameter (eg, 60%) for the word may be used.

In step S256, the document text 220
Is stored in the correlation table 230 so that regions of the document image 210 can be associated with words of the document text 220. In particular, the correlation table 23
0 stores a coordinate pair 232 for specifying an area in the document image 210, an offset pair 234 for specifying the position of a word in the document text 220, and a recognition accuracy parameter 236 for the word. In the above example, "fo
rm "are (16,110) and (31,1
49) has a coordinate pair 232, an offset pair 234 of 57 and 60, and a recognition accuracy parameter 236 of 51.3%.

Using the correlation table 230, the offset 234 in the document text 220 corresponds to the area of the document image 210 specified by coordinates 232, and vice versa. For example, given the coordinates (23,127), the column of coordinates 232 in the correlation table 230 is examined to determine that the given coordinates are included in the word located at offset 57-60. obtain. The word located at that offset in the document text 220, in the example above, the word "form" may be derived.

For the other, the correlation table 230 is consulted for a given offset (eg, 58), and as a result, (16, 110) and (3)
A rectangle having the coordinates of (1,149) is specified. Thus, the composite document architecture described herein uses word placement in document text 220 as document image 210.
Provides a way to relate to the corresponding region of

(Indicating Words with Potential Misrecognition) To reduce the time associated with referencing the original paper document, the scanned image of the original paper document (ie, document image 210) is proofread. Displayed to those who do.
In an example of reading and scanning the United States Constitution, the read and scanned image of the preamble may be displayed in image display 300 as shown in FIG.

In the image display 300, the word most likely to be erroneously recognized is displayed in a different state. For example, lightening in different colors, changing fonts, shimmering, underlining, etc. These words may be determined by comparing the corresponding recognition accuracy parameter 236 to a defined threshold. For example, the recognition accuracy parameter 236
Words with less than or equal to 60% are displayed in red, and can draw the user's attention to words that may be wrong.

In the above example, the original word "form" is 51.
It is assumed that an incorrect recognition as “fonn” is made with a recognition accuracy parameter 236 of 3%. In this case, the black pixels in the area of the image display 300 corresponding to the word "fonn" are displayed as red pixels. In a preferred example, the color of the background pixels surrounding the image of a character is changed instead of the colors of the pixels constituting the character image.

In a preferred example, the recognition accuracy parameter 236 further determines the individual display colors appropriate to the area of the document image 210 and provides a "heat map" of the recognized word.
Is compared to a plurality of thresholds to form The heat map is a chart showing values of parameters (for example, frequency, temperature, or recognition accuracy) at various points in the spectrum using a plurality of colors. The resulting "heat map"
Helps guide the user to the most problematic parts of the document image for OCR output.

Referring to FIG. 5, step S31 will be described.
When the document image 210 is displayed on the image display 300 by the loop controlled by 0, a heat map is generated. Step S310 forms a loop for all the areas to be displayed on the image display 300. In step S320, the correlation table 230 is searched to find the recognition accuracy parameter 236 corresponding to the displayed area.
Is examined. This parameter 236 is then subsequently compared to a plurality of thresholds, for example, 60%, 80% and 90%.

Steps S322 to S334 show a process of generating a heat map display when the thresholds are set to 60%, 80% and 90%, for example. First, the lowest threshold of 60% is used as a threshold for comparison. If the recognition accuracy parameter 236 is lower than the threshold, the color of the display area is set to red (step S324). In the above example, the word “form” is displayed in red because its recognition accuracy parameter 236 is 51.3%. FIG.
Other words set to red in "general" and "Constituti
on ".

Next, in step S326, the next lower threshold value of 80% is used as a comparison threshold value. If the recognition accuracy parameter 236 is lower than the threshold, the color of the display area is set to green (step S32).
8). In the above example, the word "Union" has a recognition accuracy parameter 236 of 75% and is therefore displayed in green. Other words set to green in FIG. 4 are "ensure" and "secure".

In step S330, the last threshold value of 9
0% is used as a threshold for comparison. If the recognition accuracy parameter 236 is lower than the threshold, the color of the display area is set to blue (step S332). The words set in blue in FIG. 4 are "more", "Tranquility" (partially hidden by pop-up menu 304), and "establish". On the other hand, the recognition accuracy parameter 23
If 6 is higher than all thresholds, the color of the display area is
It becomes the same black as the default color (step S334).
When the color is set, the area is displayed in that color (step S336).

It will be appreciated that the numbers and colors for the thresholds may vary depending on the embodiment without departing from the spirit of the invention. For example, one threshold, 2
One, three, or even ten. As another example, the color selection may change (e.g., red, orange,
yellow). Actually, display attributes other than the display color such as blinking and underlining may be employed. It is also understood that the thresholds and display colors or other display attributes may be entered in one table and continually examined in one loop without strictly organizing the branches as shown in the flowchart of FIG. Can be done.

The user may place the cursor 302 over the highlighted word in the document image 210 and cause the corresponding recognized text in the document text 220 to be displayed nearby (eg, a pop-up menu display) to correct the error. Corrections can be even easier. For example, when the user positions the cursor 302 over the red word "form" in the document image 210, a pop-up menu 304 is displayed so that the user can know that the word has been incorrectly recognized as "fonn". You may.
When the user corrects the word, the recognition accuracy parameter 236 of the corrected word is reset to 100%, and the display of the area of the document image 210 corresponding to the corrected word is returned to black.

After executing the loop controlled by step S310, the document image 210 is displayed on the display device 120 such as a high-resolution monitor by the image display 300.
Will be displayed as In addition, the cursor 302 is displayed on the image display 300. Then, the user may position the cursor 302 on any part of the image display 300 using the cursor control means 124 such as a mouse, a trackball, or a joystick.

In step S340, the error detection device receives an input for selecting a position on the image display 300. This input is performed when the cursor 302 is displayed on the image display 300.
It may be automatically generated by the cursor control means 124 whenever it is placed on it or only when the user operates a button. In the latter case, when the user operates the button, the cursor control means 124 sends the current position of the cursor 302 as an input.

The location associated with the input received in step S340 is converted from the coordinate system of image display 300 to the coordinate system of document image 210 by mapping techniques well known in the art. Coordinate transformations are often required because the document image 210 of a large document does not fit within the smaller image display 300. In the example shown in FIG. 4, the position of the cursor 302 on the image display 300 corresponds to the coordinates (23, 127) of the document image 210.

In step S342, step S340
The correlation table 230 is consulted for a description that specifies an area that includes the coordinates 232 obtained from the input received at. In the above example, the coordinates (23, 127) are (16,
110)-(31,149) are included in the area determined by the coordinate range. The offset pair 234 for the document text 220 is taken from the entry in the correlation table 230 and used to determine the selected word in the document text 220. In the above example, the corresponding offset pair is 57-60. This offset pair is used to extract the string placed in the document text 220 at an offset within the offset pair 234.
In the above example, it was assumed that the original word "form" in the preceding sentence was misrecognized as "fonn", in which case the word selected within the offset pair 234 would be "fonn".

In step S344, the selected word is displayed in pop-up menu 304 near cursor 302, so that the user can easily determine what the recognized word is. Therefore, in the above example,
Pop-up menu 304 displays the selected word "fonn" so that when the pop-up menu is displayed, the user can only look at image display 300 of document image 210 and determine that the selected word is incorrect. can do.

According to one embodiment, when the cursor 302 is over a word in the image display 300, the position of the cursor is automatically entered, so that the pop-up menu 304 is automatically displayed. . Thus, the user can move the cursor 302 over the displayed column of text in the image display 300 and compare the selected text automatically displayed at the reference position in the pop-up menu 304. Thus, the user does not have to spend the time required to look at the original paper to determine whether a character has been misrecognized by the OCR device 128. If the word is different, the user can correct the text as described above.

Although the invention has been particularly described and illustrated with reference to specific embodiments, variations in form or detail may be made in the above description or figures without departing from the spirit or scope of the invention. It will be understood by those skilled in the art that

[0045]

As described above, a method and apparatus for correcting optical character recognition using a heat map according to the present invention, and a series of instructions for finding an error in an OCR output are recorded. According to the computer-readable recording medium described above, when correcting an error that occurred while converting the original text to the text of the OCR output,
This has the effect that the user does not have to look at the original paper document.

[Brief description of the drawings]

FIG. 1 is a high-level block diagram illustrating a computer system on which the present invention can be implemented.

FIG. 2 is a block diagram illustrating a composite document architecture.

FIG. 3 is a flowchart illustrating a process of generating a composite document.

FIG. 4 is a diagram showing an example of a screen display according to an embodiment of the present invention.

FIG. 5 is a flowchart showing a process of finding and correcting an error in an OCR output according to one embodiment.

[Explanation of symbols]

112 processor 118 data storage device (computer-readable recording medium) 120 display device 124 cursor control means 128 optical character recognition device 210 document image 220 document text 236 recognition accuracy parameter 304 pop-up menu

Claims (12)

[Claims] 1. A method for finding errors in an OCR output, the method comprising: recognizing characters in a document image to generate document text; and determining a region of the document image corresponding to words in the document text. Determining a recognition accuracy parameter for the region of the document image; and displaying the region of the document image so as to represent the respective recognition accuracy parameters. How to make corrections. Receiving an input to select a location in the document image; determining a selected word corresponding to a document area containing the location in the document image; The method for correcting optical character recognition using a heat map according to claim 1, further comprising: displaying a word. 3. The method according to claim 2, wherein displaying the selected word comprises displaying the selected word in a pop-up menu. To make corrections. 4. The method according to claim 1, wherein the step of displaying the area of the document image comprises: comparing the recognition accuracy parameter with a plurality of recognition accuracy thresholds; 2. The method according to claim 1, further comprising: setting a color of the area; and displaying the area of the document image with each set color. Way to do. 5. An apparatus for finding an error in an OCR output, comprising: an OCR apparatus for recognizing a character in a document image to generate a document text; and determining an area of the document image corresponding to a word of the document text. Means for determining a recognition accuracy parameter for the area of the document image, and a display device for displaying the area of the document image so as to represent the respective recognition accuracy parameters, using a heat map characterized by comprising: A device for correcting optical character recognition. 6. A cursor control means for receiving an input for selecting a position in a document image, and means for determining a selected word corresponding to a document area including the position in the document image. The apparatus for correcting optical character recognition using a heat map according to claim 5, wherein the display device displays the selected word. 7. The apparatus for correcting optical character recognition using a heat map according to claim 6, wherein the display device displays the selected word in a pop-up menu. 8. A comparator for comparing a recognition accuracy parameter with a plurality of recognition accuracy thresholds, and means for setting a color of the region based on the recognition accuracy threshold exceeded by the recognition accuracy parameter for each region. The device for correcting optical character recognition using a heat map according to claim 5, further comprising: a display device for displaying a region of the document image in each set color. . 9. A computer-readable recording medium having recorded thereon a series of instructions for finding an error in an OCR output, the series of instructions recognizing a character in a document image to generate a document text. Determining the area of the document image corresponding to the word of the document text, determining the recognition accuracy parameter for the area of the document image, and displaying the area of the document image to represent each recognition accuracy parameter And a series of instructions for performing the following steps. 10. The series of instructions comprising: receiving input selecting a location in a document image; and determining a selected word corresponding to a document region including the location in the document image. The computer-readable medium of claim 9, further comprising: a series of instructions for performing: and displaying the selected word. 11. The computer-readable medium of claim 10, wherein displaying the selected word comprises displaying the selected word in a pop-up menu. 12. The step of displaying an area of the document image, the step of comparing the recognition accuracy parameter with a plurality of recognition accuracy thresholds, and the step of displaying based on the recognition accuracy threshold that is exceeded by the recognition accuracy parameter for each area. 10. The computer-readable recording medium according to claim 9, comprising: setting a color of the area; and displaying an area of the document image with each set color.