JPH08335248A - Document reader - Google Patents

Abstract

PURPOSE: To provide a document reader which does not perform unnecessary rerecognition. CONSTITUTION: A layout data confirmation and correction part 10a sets all the recognition flags of layout data ON when its confirmation and correction are performed for the 1st time and sets only a corrected part ON when not. A recognition part 5 recognizes only parts where recognition flags are ON. A recognition data confirmation and correction part 10b instructs the layout data confirmation and correction part 10a to conform and correct the layout data when the layout data need to be corrected after the recognition processing. Therefore, only parts of the layout data which are corrected are recognized for the 2nd and succeeding times and the parts which are already recognized correctly are not recognized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document reading device for reading characters and images (graphics other than characters, photographs, ruled lines, etc.) recorded on a document.

[0002]

2. Description of the Related Art Conventionally, a reading device for a document in which characters and images are mixed has been developed. These devices perform the following processing. That is, the layout of the position on the document of the data area to be recognized is analyzed, and the operator confirms / corrects the result. Then, character recognition or the like is performed according to the confirmed / corrected layout data, and the operator further confirms / corrects the result. If an error in the layout analysis result is found during the confirmation of the recognition result, the process returns to the layout analysis confirmation / correction process, the layout data is corrected, and the recognition is performed again based on the layout data (for example, Japanese Patent Publication 4-731
92, etc.).

[0003]

However, in the above-described conventional document reading apparatus, when re-recognition is performed, the layout data that has not changed since the previous recognition and the corrected layout data cannot be distinguished. For this reason, at the time of re-recognition, the same recognition as before is performed on the portion that is not corrected, and redundant processing is performed. In addition, there is a problem that the previously corrected recognition result is recognized again and returns to the state before the correction, and the same recognition result as before needs to be corrected.

From this point of view, it has been desired to realize a document reading apparatus which does not perform unnecessary processing when re-recognizing.

[0005]

The present invention adopts the following configurations in order to solve the above problems. That is, the document reading apparatus of the present invention analyzes the layout of the position on the document of the data area to be recognized from the image data on the document,
A layout analysis unit that obtains the layout data of the document, the layout data acquired by the layout analysis unit, and the image data on the document are overlapped and displayed on the display unit,
When the confirmation / correction of the layout data is accepted, the recognition flag corresponding to each area is provided, and when the confirmation / correction processing is performed twice or more, the recognition flag of the area is changed to
Based on the layout data confirmation / correction unit that outputs the layout data to which these recognition flags are added, as a state different from the area where the confirmation / correction processing has not been performed, the first In recognition, all areas including character recognition are recognized,
In the subsequent recognition, a recognition unit for recognizing only the area corrected by the layout data confirmation / correction unit, and a display unit for instructing whether or not the layout data after the recognition processing by the recognition unit needs to be corrected. And a recognition data confirmation / correction unit for confirming / correcting layout data to the layout confirmation / correction unit when a correction is designated. It is a thing.

[0006]

In the document reading apparatus of the present invention, the layout data confirmation / correction unit performs the confirmation / correction processing of the layout data acquired by the layout analysis unit. At this time, if the confirmation / correction processing is performed for the second time or more, the recognition flag of the area is set to a state different from the area in which the confirmation / correction processing is not performed, and the layout data including these recognition flags is output. To do. The recognition unit, based on the recognition flag,
In the first recognition, all areas are recognized, and in the second and subsequent recognitions, only the areas corrected by the layout data confirmation / correction unit are recognized. The recognition data confirmation / correction section
A selection instruction is made as to whether or not the layout data after recognition needs to be corrected, and when the correction is designated, a confirmation / correction instruction is given to the layout data confirmation / correction unit. As a result, in the second recognition process, only the part in which the layout data has been corrected is recognized, and the part that has already been recognized correctly is not recognized.

[0007]

Embodiments of the present invention will now be described in detail with reference to the drawings. First, the first embodiment will be described.

<< Structure of First Embodiment >> FIG. 1 is a block diagram of a document reading apparatus according to the present invention. The apparatus shown in FIG.
The image memory 2, the layout analysis unit 3, the layout memory 4, the recognition unit 5, the recognition memory 6, the output unit 7, the display unit 8, the operation unit 9, and the control unit 10. Further, the image input unit 1 to the result output unit 7 and the control unit 10 are respectively connected to the data bus 1
The image input unit 1, the layout analysis unit 3, the recognition unit 5, the output unit 7, and the control unit 10 are connected via a control bus 12.

The image input unit 1 is composed of an image scanner or the like, optically scans a document to be read, converts characters and images recorded on the document into an image signal by photoelectric conversion, and further converts the image signal. It has a function of converting into binary image data. The image memory 2 is a memory that stores binary image data output from the image input unit 1. The layout analysis unit 3 divides the characters and images from the image data in the image memory 2 into regions by a method such as using the peripheral distribution of black pixels, and the regions are defined as characters according to the geometrical characteristics of each region. Identify in the image. Then, with respect to the character area, a line is cut out by a method of utilizing the peripheral distribution of black pixels, and a character is cut out from each line. Further, it has a function of analyzing the reading order of the character area, line and character. The layout data is made up of the above areas, lines, characters, their reading order, and the like.

The layout memory 4 is a memory for storing layout data output from the layout analysis section 3. The recognition unit 5 includes a recognition dictionary and a word dictionary that record standard character recognition features, reads the layout data of the layout memory 4, and performs character matching by pattern matching using the recognition dictionary according to the layout data. It has a function of performing recognition, and further performing post-processing such as using a word dictionary and grammar rules to correctly recognize as a document. The recognition data includes a character code, a candidate character code, a certainty factor of each, a character string as a post-processing result, a candidate word and its certainty factor, and the like. Recognition memory 6
Is a memory for storing the recognition data output from the recognition unit 5.

The output unit 7 reads the image data from the image memory 2, the layout data from the layout memory 4, and the recognition data from the recognition memory 6, and the character string and the image data in the recognition data are read according to the layout data.
Convert to document data of any format, etc.,
It has a function of outputting to an output device in the output section. still,
The output device is a magnetic disk device that stores document data, a printer that prints document data, or the like. Display 8
Is a CRT or the like, and the operation unit 9 is a keyboard, a mouse, or the like. The display unit 8 and the operation unit 9 display processing start instructions and processing results (image data, layout data, and recognition data) in the image input unit 1, the layout analysis unit 3, and the recognition unit 5, and confirm / check them. It has a function as an input / output device for interfacing with an operator for correction and the like.

The control unit 10 has a function of controlling the overall operation of each unit and each memory, and confirms layout data.
A correction unit 10a and a recognition data confirmation / correction unit 10b are provided. The layout data confirmation / correction unit 10a causes the display unit 8 to display the layout data acquired by the layout analysis unit 3 and the image data from the image input unit 1 in an overlapping manner.
When the operator inputs the confirmation / correction content from the operation unit 9, the input is accepted and the corresponding processing is performed, and if the confirmation / correction processing is the second or subsequent processing, It has a function of performing confirmation / correction based on the layout data, and adding and outputting a recognition flag for identifying whether or not the correction is made again.

The recognition data confirmation / correction unit 10b instructs the display unit 8 to select whether or not the layout data after the recognition processing by the recognition unit 5 needs to be corrected.
When the operation unit 9 specifies that the correction is necessary,
The layout data confirmation / correction unit 10a has a function of instructing layout data confirmation / correction.
The layout data confirmation / correction unit 10a and the recognition data confirmation / correction unit 10b are composed of a dedicated processor or program.

<< Operation of First Embodiment >> FIG. 2 is a flowchart showing the operation of the document reading apparatus of the present invention. First, an image is input (step S1). That is, the image input unit 1 scans a document to be read, converts characters and images recorded on the document into image signals by photoelectric conversion, and further converts the image signals into binary image data. And
The image data is stored in the image memory 2 and displayed on the display unit 8.

When image input is completed, layout analysis is performed (step S2). In the layout analysis, first, the area is analyzed. This area analysis is performed as follows. The layout analysis unit 3 extracts the circumscribing frame (region frame) of the character and image regions from the image data in the image memory 2 by using the peripheral distribution of black pixels, and further, the geometrical characteristics of each region. To distinguish between characters and images. Then, regarding the character area, it is determined whether the writing is vertical writing or horizontal writing using the peripheral distribution of the black pixels. If there is much vertical writing as a result of this determination, the reading order is assigned to the character areas from the upper right to the lower left. On the other hand, when there is a large amount of horizontal writing, the reading order is given to the character areas from the upper left to the lower right. The area data is configured by these area frames, area types, vertical / horizontal writing of character areas, and reading order.

Next, of the area data, line cutting is performed for the character area. For line segmentation, first, a circumscribing frame (line frame) of a line is extracted using a peripheral distribution of black pixels in a character area. Then, according to the vertical writing / horizontal writing of the character area determined by the area analysis, the reading order of the lines is given from right to left in the case of vertical writing and from top to bottom in the case of horizontal writing. The row data is composed of this row frame and the reading order. Finally, character cutting is performed on this line data. Also in the character cutout, first, the circumscribing frame (character frame) of the character is extracted using the peripheral distribution of the black pixels in the row. Then, according to the vertical writing / horizontal writing of the character area determined by the area analysis, the reading order is given from the top to the bottom in the case of the vertical writing and from the left to the right in the case of the horizontal writing. Character data is constituted by this character frame and the reading order. Further, the layout data is composed of the above area, line, and character data.

When the line and character segmentation of all character regions is completed, the layout data thus obtained is stored in the layout memory 4.

When the layout analysis is completed, the layout data is confirmed / corrected (step S3). In this layout data confirmation / correction processing, the layout data confirmation / correction unit 10a reads the image data from the image memory 2 and the layout data from the layout memory 4, and displays the image data and the layout data on the display unit 8. Overlap and display. The operator looks at this display and
The portion of the layout data that needs to be corrected is corrected using the operation unit 9.

FIG. 3 is an explanatory diagram of the layout data confirmation / correction processing. In FIG. 3, the image data of the document to be read is G1. Further, the area data confirmation / correction screen is G2. In the drawing, in order to avoid overlapping of the reading order G24 and the vertical writing / horizontal writing section G26 of the character area such as "1 horizontal" and the character string in the character area frame G21 such as "document", Although the area frame is shown slightly larger than the character string, it is actually a frame substantially outside the character string.

First, the character area frames G21 and G22, the image area frame G23 (the character area frame and the image area frame are distinguished by changing the display color) overlaid on the image data G1, the reading order G24, G25 of the character area, and Vertical writing of character area /
Horizontal writing sections G26 and G27 are displayed. The operator
Using this display, the position, size, type, reading order, vertical writing / horizontal writing, etc. of the area frame are confirmed and corrected using the operation unit 9. At this time, if the character area data is modified, the line and character cutout data of this character area will be different from the one before correction.Therefore, the line and character cutout will be performed for the corrected character area, and the layout after correction and recutout ( The area, line, character) data is stored in the layout memory 4.

When the confirmation / correction of the area data is completed, the screen G3 is displayed to confirm / correct the row data. This screen G3 shows a confirmation / correction screen for line data in the character area G21. Here, the line frames G31 and G32 and the reading order G33 and G34 are displayed so as to be superimposed on the image data of the character area G21. The operator sees this display and confirms / corrects the position, size, reading order, etc. of the line frame using the operation unit 9. At this time, if the line data is modified, the character cutout data of this line will be different from that before correction, so character cutout will be performed for the corrected line, and layout (line, character) data after correction and recutout will be laid out. Stored in the memory 4. The above line data confirmation / correction is similarly performed for the character area G22.

When the confirmation / correction of the line data is completed, the screen G4 is displayed to confirm / correct the character data. This screen G4 shows a confirmation / correction screen for the character data in the line frame G31. Here, the character frames G41, G42, G43, G44, the reading order G45, are superimposed on the image data of the line frame G31.
G46, G47 and G48 are displayed. The operator sees this display and confirms / corrects the position, size, reading order, etc. of the character frame using the operation unit 9. Then, the corrected layout (character) data is stored in the layout memory 4. The above-described confirmation / correction of the character data is similarly performed for the line frame G32 and the line of the character area G22.

In the above-described layout data confirmation / correction processing, the processing target is the entire hierarchy of the area, line, and character layout analysis. However, if there is almost no error in character segmentation, the confirmation / correction processing is performed for the area and line. You can also finish. Similarly, the confirmation / correction process can be terminated in the area. In this way, only the layers that require confirmation / correction processing are processed, and the confirmation / correction work can be reduced.

Further, when the process proceeds from the confirmation / correction of the area data to the confirmation / correction of the line data, the operator does not need to perform all the operations in the reading order as described above, but a character area in which the operator needs to confirm / correct the line data. There is also a method to specify and proceed. Confirmation of character data from confirmation / correction of line data
The same applies to the case of proceeding with the correction. In this way, only the parts that need confirmation / correction are processed, and the work of confirmation / correction can be reduced.

FIG. 4 is an explanatory diagram showing a configuration example of layout data and recognition data. First, the layout data is composed of page attribute / reading frame → region attribute / region frame → line attribute / line frame → character attribute / character corresponding to region analysis → line segmentation → character segmentation of the hierarchical structure of layout analysis processing. It becomes a frame. Each attribute is, for example, the type of area, the reading order, the vertical writing / horizontal writing distinction, the recognition flag, etc., although it differs depending on the hierarchy. Each frame is coordinate data representing a circumscribing frame. Also,
The recognition flags are all turned on at the time of checking / correcting the first layout data and stored in the layout memory 4 in this state (step S4 → step S5). On the other hand, when the layout data is checked / corrected again after the recognition, only the corrected part is turned on, and the other uncorrected parts are turned off and stored (step S4 → step S6).

When the recognition / correction of the layout data is completed, the characters of the image data of the part where the recognition flag is turned on are recognized according to these layout data (step S).
7). In this recognition processing, the recognition unit 5 first performs character matching data for each character (candidate character code, respectively) on the characters of the image data according to the layout data by pattern matching using the recognition dictionary in the recognition unit 5. Confidence). Next, the obtained character recognition data is post-processed using a word dictionary and grammar rules in the recognition unit 5, and is correctly recognized as a sentence, and post-processed data (a character string that is a result of the post-processing, Candidate words and their certainty)
Create Here, the character recognition data and the post-processing data are collectively referred to as recognition data. Then, when the post-processing is completed, the recognition data is stored in the recognition memory 6.

When the recognition is completed, the recognition data confirmation / correction unit 10b first turns off the recognition flag (step S
8) Next, confirm / correct the recognition data (step S)
9). In this confirmation / correction processing, the recognition data from the recognition memory 6, the layout data from the layout memory 4, and the image data from the image memory 2 are read out, and a confirmation / correction screen of the recognition data is displayed on the display unit 8.

FIG. 5 is a confirmation / correction screen for this recognition data. In FIG. 5, R1 is a recognition data correction / correction screen, R2 is a text screen, and the post-processed character string R21
Is displayed, and the character to be checked / corrected is displayed with the mouse cursor R22.
, And then the highlighted character is highlighted. In the figure, "reading" surrounded by a frame indicates an inverted display state. R3 is a character image screen, which displays a character image R31 of the character designated by the mouse cursor R22, and the character frame R3
Display 2 on top of each other. Further, R4 is a candidate screen for displaying a candidate character (or candidate word) R41 of the character designated by the mouse cursor R22. The operator looks at this display, compares the character string R21 with the character image R31, confirms the character string, and if there is an error, replaces it with the candidate character R41, inputs the correct character from the operation unit 9, etc. Checking and correcting the recognition data.

Next, the recognition data confirmation / correction unit 10b
It is determined whether the layout data needs to be modified (step S10). FIG. 6 is an explanatory diagram of the layout data correction determination. Note that in FIG. 6 as well, the character area frame is shown slightly larger than the character string in order to avoid overlapping of characters such as "1 landscape" and the character string in the character area frame such as "document". However, it is actually the outer frame of the character string. For example, as shown in FIG. 6B, the character area is erroneously analyzed as an image area (the area F23, which is a character area, is identified as an image area). In some cases, as shown in FIG. 6C, two lines are erroneously cut out into one line (a part of two lines is cut out as one line F31), and the line is recognized as it is.

In such a case, since the work of correcting the recognition data is too much, the operator judges whether or not there is a portion that needs to be re-recognized by correcting the layout data. That is, the recognition data confirmation / correction unit 10b causes the display unit 8 to display a selection screen as to whether or not the layout data needs to be corrected, so that the operator makes a designation from the operation unit 9. For example, when the character area is identified as the image area, the correction of the recognition data requires a work of inputting the character in the area from the operation unit 9, and when the number of characters in the character area is large, a great deal of labor is required. And it will take time. Therefore, in such a case, the layout data itself is corrected.

When it is specified in step S10 that the correction is necessary, the process returns to step S3, and the layout data confirmation / correction unit 10a corrects the layout data.

7 to 9 are views for explaining the contents of the layout data at the time of determining the correction of the layout data. 7 to 9 are the contents of the layout data shown in FIG. FIG. 7 shows layout data before it is determined that the layout data needs to be corrected in step S10 in FIG. 2 and the process returns to step S3 to correct the layout data. At this time, all areas, lines, and characters have been recognized, so the recognition flag is "off" by the processing in step S9.

Next, when the image area F23 is changed to a character area and lines and characters are cut out for this character area by modifying the layout data of FIG. 6B, the layout data is changed from FIG. 7 to FIG. To fix. Further, in this case, since the confirmation / correction of the layout data is not the first, the process proceeds to step S6, and the recognition flag of only the corrected portion L1, the area, the line and the character is turned on. Further, by modifying the layout data of FIG.
When the line is changed from 1 line to 2 lines and character cutting is performed on these 2 lines, the layout data is corrected to FIG. 8 → FIG. 9. Similarly, in the modified portion L2, the line and character recognition flags are set to "ON".

After that, the recognition processing by the recognition unit 5, the recognition data confirmation / correction processing by the recognition data confirmation / correction unit 10b, and the necessity / unnecessity of the layout data correction are determined. If it is determined in step S10 that the correction is unnecessary, the process proceeds to step S11, and the output unit 7 converts and outputs the recognition, layout, and image data. The output unit 7 outputs image data from the image memory 2,
The layout data is read from the layout memory 4 and the character string is read from the recognition memory 6, the character string and the image data are converted into document data of an arbitrary format according to the layout data, and a magnetic disk device or a printer in the output unit 7 is used. Output to the output device.

<< Effects of First Embodiment >> As described above, in the first embodiment, the recognition flags are provided for the area attribute, the line attribute, and the character attribute, and when the layout data is recorrected, the recognition flag of the corrected portion is set to " By setting the recognition flag of the portion that is not corrected to "off" to "off", the presence or absence of the correction can be distinguished. Then, since only the character "on" of the recognition flag is re-recognized, redundant recognition (the character of the recognition flag "off" becomes the same as the previous recognition result even if it is re-recognized), Efficient processing is possible. In addition, since the correction process is not performed for the part that does not require the correction of the layout data due to the recognition and correction of the previous recognition data, the recognition data does not return to the state before the correction, and the recognition data must be corrected again. Processing can be prevented.

Next, a second embodiment will be described. << Structure of Second Embodiment >> In the second embodiment, the structure on the drawing is the same as that of the first embodiment shown in FIG. 1, but the function of the recognition data confirmation / correction unit 10b is different. That is, in the second embodiment, when the recognition data confirmation / correction unit 10b displays the layout data to be recognized, the recognition data confirmation / correction unit 10b performs control so as to distinguish and display the area for re-recognition and the area for no recognition. There is. Further, the other configurations are similar to those of the first embodiment, and thus the description thereof is omitted here.

<< Operation of Second Embodiment >> In the operation of the second embodiment, a part different from that of the first embodiment will be described below. The parts not described below are the same as in the first embodiment. In the recognition / correction processing of the recognition data again in step S8 of FIG.
The portion re-recognized in step S7 and the portion not re-recognized are displayed separately. For example, the part that is not re-recognized is shaded (other than shaded, it is highlighted, the display color is changed, blinking,
(Underline, bold line, etc. may be displayed), indicating that the processing has been performed by the recognition / correction of the previous recognition data. On the other hand, the portion that has been re-recognized is displayed as a normal display to show that the confirmation data is recognized and corrected this time.

FIG. 10 shows a confirmation / correction screen for this recognition data. FIG. 10A shows a confirmation / correction screen of the recognition data corresponding to the corrected portion of the layout data shown in FIG.
FIG. 10B is a confirmation / correction screen corresponding to the correction of the layout data of FIG. In addition, in the figure, "area" and "acknowledgement" surrounded by a frame indicate an inverted display state.

<Effect of Second Embodiment> As described above, according to the second embodiment, at the time of confirming / correcting the re-recognition result, whether the recognition result is based on the corrected layout data or the previously confirmed recognition result. Since it is possible to distinguish whether or not
It is possible to reduce the burden on the operator in the correction work.

In each of the above embodiments, the image input unit 1 is
Although a device such as a scanner that optically reads a document to be read is used, other configurations may be adopted in which image data is input via a communication line. Further, the output unit 7 may be configured not only as a magnetic disk device or a printer but also as an external output to a communication line.

[0041]

As described above, according to the document reading apparatus of the present invention, in the re-recognition process, only the portion in which the layout data is re-corrected is recognized, and therefore redundant recognition is performed. Therefore, the processing efficiency can be improved. In addition, since the recognition processing is not performed for the portion of the layout data that does not need to be corrected, it is possible to avoid a situation in which the recognition data returns to the state before correction, and as a result, it is possible to prevent unnecessary load on the operator. can do.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a document reading apparatus of the present invention.

FIG. 2 is an operation flowchart of the document reading device of the present invention.

FIG. 3 is an explanatory diagram of layout data confirmation / correction processing in the document reading apparatus of the present invention.

FIG. 4 is an explanatory diagram showing a configuration example of layout data and recognition data in the document reading device of the present invention.

FIG. 5 is a confirmation / correction screen of recognition data in the document reading apparatus of the present invention.

FIG. 6 is an explanatory diagram of layout data correction determination in the document reading apparatus of the present invention.

FIG. 7 is a diagram (part 1) for explaining the contents of layout data when determining whether to correct the layout data in the document reading apparatus of the present invention.

FIG. 8 is a diagram (part 2) for explaining the contents of layout data at the time of determining whether to correct the layout data in the document reading device of the present invention.

FIG. 9 is a content explanatory view (No. 3) of the layout data at the time of determining whether to correct the layout data in the document reading apparatus of the present invention.

FIG. 10 is an explanatory diagram of a recognition data confirmation / correction screen in the second embodiment of the document reading apparatus of the present invention.

[Explanation of symbols]

 1 Image Input Section 3 Layout Analysis Section 5 Recognition Section 8 Display Section 9 Operation Section 10 Control Section 10a Layout Data Confirmation / Correction Section 10b Recognition Data Confirmation / Correction Section

Claims (2)

[Claims] 1. A layout analysis unit that analyzes a layout of a position of a document in a data area to be recognized from image data on the document to obtain layout data of the document, and layout data acquired by the layout analysis unit. When,
The image data is displayed on the display unit in an overlapping manner to accept confirmation / correction of the layout data, and a recognition flag corresponding to each area is provided.
If it has been done more than once, check the recognition flag of the area.
The layout data confirmation / correction unit that outputs the layout data to which these recognition flags are added is set as a state different from the area in which the correction process is not performed, and the first recognition of the target document is performed based on the recognition flag of the layout data. , A recognition unit for recognizing all regions including character recognition, and for subsequent recognition, recognizing only a region corrected by the layout data confirmation / correction unit, and a layout after recognition processing in the recognition unit. The display unit is instructed to select whether or not the data needs to be corrected, and when the correction is designated, the layout confirmation / correction unit confirms the layout data.
A document reading device, comprising: a recognition data confirmation / correction unit for issuing a correction instruction. 2. The document reading device according to claim 1, wherein when the layout data to be recognized is displayed, the layout data confirmation / correction is performed so that the area to be re-recognized and the area not to be recognized are displayed separately. A document reading device comprising a section.