JPH01258089A - Document processor - Google Patents

Abstract

PURPOSE:To quickly correct the significant structure of a document by providing a display device and instructing the erroneous portion of the document being displayed on the display device, and then, processing the portion by means of an auxiliary switch. CONSTITUTION:Character rows of document pictures inputted to a picture input device 1 are extracted by mans of a character row extracting section and the extracted information is displayed on a display device 3 in a state capable of identification. The display device 3 displays each row by a circumscribing polygon and each significant group of each row by means of a color or a polygon surrounding the group. Then the connecting relations among the groups are displayed by numbers affixed to the side of each row. A user instructs a point or area of the character row information displayed on the display device 3 through an instructing device 5. When the user finds the erroneous portion, he designates the erroneous portion from the device 5 and corrects the erroneous portion by using an auxiliary switch 6. Therefore, the erroneous portion in significant structure of the document can be corrected quickly.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial Application Field) The present invention relates to a character reading device, and more particularly to a document processing device by analyzing the geometric structure of a document.

(Prior Art) A character recognition device that inputs a document image, reads the characters in the same order as a human reads the document, and outputs the results is effective as an input means to an automatic translation device, etc. In order to perform character recognition, it is necessary to have knowledge about how words and phrases in a document are connected in meaning, that is, the semantic structure of the document. In order to deal with a wide variety of documents, it is desirable to obtain knowledge about this semantic structure by analyzing the document. One way to do this is to recognize the meaning of each word after recognizing each character in the document. The most reliable method is to determine the semantic structure by analyzing continuity and grammatical consistency. However, this process was complicated and unfeasible.

On the other hand, in addition to the semantic structure, a single document has a geometric structure as seen in the layout of the document. Geometric structure often reflects semantic structure.

Therefore, by analyzing the geometric structure, it is possible to accurately determine the semantic structure to some extent.

Conventionally, as an approach to this, there has been a method of automatically extracting character lines by processing a document image, and analyzing the layout of the character lines to infer semantic connections between the character lines. However, if the quality of the input image is poor or the structure of the document is too complex, there are problems in that the extraction of character lines may be incorrect or the inference of semantic connections may fail.

(Problems to be Solved by the Invention) As described above, conventional character recognition devices that infer semantic structure from geometric structure have the problem that they may make mistakes in semantic structure analysis based on the layout of the document. Ta. If a correction function for correcting the above-mentioned errors is added to the device, the device can be applied more widely. Since the ease of use of this correction function is an issue, it is desirable that it can be used interactively with the user.

It is an object of the present invention to provide a corrective function to solve the above problems.

[Structure of the invention]

(Means for Solving the Problem) In order to achieve the above object, the present invention provides a display device that allows a user to view information on character lines extracted from a document image, and a display device using the display device. An indicating device that allows the user to specify a point, figure, or area, and an auxiliary switch attached to or separate from the indicating device, and also has a function of displaying character lines as line circumscribing polygons and the order of the lines during the event. A function to display a number next to the polygon, a function to distinguish articles by the color of the polygon or a polygon surrounding the article, a function to display the chain relationship of articles by a number beside the article area, and To provide a document processing device which has a function of displaying a pattern as necessary and a function of interactively performing correction work by making full use of the display function, the instruction device and the auxiliary switch.

(Function) According to the present invention, the user can find the error on the display, and can correct the error by indicating the error on the display using the indicating device and specifying the process using the auxiliary switch. In particular, by displaying lines and articles as numbered polygons, users can easily determine the current combination of text lines and articles within the system and the display range that should be specified when specifying a text line. and can be accurately grasped. Therefore, correction work can be carried out quickly.

(Example) An example of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of a character recognition device according to the present invention.

The image input device 1 is a means for inputting images. The control device 2 is a means for controlling the entire system. The display device 3 is a means for displaying information. The storage device 4 is a means for storing software, image data, and other data. The instruction device 5 and the auxiliary switch 6 are means for receiving processing commands, cursor movement commands, etc. The flow of processing in the above device will be explained below.

FIG. 2 shows the overall processing flow in the apparatus of this embodiment. For the image cuff, input an image that includes text lines. Line extraction 8 extracts character lines from the input image. In structure extraction 9, the extracted character lines are integrated into an order that is considered to be semantically continuous by evaluating the layout of the character lines. The merged text lines are treated as one article.

In modification 10, extraction errors, omissions, and errors in the order of character lines are interactively corrected for the character lines and articles automatically extracted by the above process, and article information is obtained in which the character lines are integrated in the order of their semantic connections. . In the recognition step 11, character recognition is performed in the order of the semantic connections obtained through the above procedure, and character strings that are connected in meaning throughout the article are output.

Next, this correction function will be explained.

The interpretation of the semantic structure of the input document automatically extracted through structural analysis is interactively modified by this modification function.

The problem here is how to specify lines and articles using the indicating device. Three methods of specification in this embodiment will be explained below.

Point designation is to designate a point position on the display using a pointing device and designate that point using an auxiliary switch. Point designation is a designation method used for three purposes: menu selection, point position designation, and designation of an area extracted by the system, such as a row circumscribed rectangle. The actual operation is performed by moving the pointing point to the desired position using the pointing device and pressing the auxiliary switch.

Line specification is to specify a locus that can be drawn by moving the indicated point on the display. The line designation is a designation method used to consecutively designate multiple areas extracted by the system and to designate a cutting line in a line or article division operation. The actual operation is performed by first moving the pointing point to the starting position using the pointing device, then moving the pointing point along the desired trajectory while pressing the auxiliary switch, and releasing the auxiliary switch at the end point.

Two-point designation is to designate the end points of a certain line segment. Two-point designation is a designation method used for two purposes: character pattern string designation and cutting line designation when dividing multiple lines. Actual operation is performed by moving the pointing point to two target points using the pointing device and briefly pressing the auxiliary switch twice at each position.

FIG. 3 is a block diagram showing the configuration of a processing system that implements this function. Each block will be explained below.

The main menu operation 34 is a function for selecting a target process from a group of processing modules. The user can select the desired process from the displayed menu by specifying a point. The above menu is always displayed by 30 during the correction work.

The auxiliary menu operation 37 is a function for selecting detailed processing in each processing module. The user can select the desired process from the displayed menu by specifying a point. The contents of the auxiliary menu vary depending on the processing module.

Furthermore, by brightening currently selectable items and darkening currently unselectable items among the options included in the menu, the user is informed of the process to be specified in an easy-to-understand manner according to the situation. The above menu is displayed by 30.

The line information/article information memory 32 has a function of storing information regarding character lines and articles. The line information includes each line label, the character pattern string in the line, the position of the line on the document image, etc., and the article information includes the label of each article, line chain relationship information in the article, article chain relationship information, etc.

The image memory 33 has a function of storing images excluding patterns extracted as character lines from an input document image and other images.

Layout image operation 35 is a function that generates a layout image by combining the article and line information stored in 32 with the input document image stored in 33 excluding character line patterns, and stores it in 33 again. be.

The above image information is displayed by 30.

The pattern image operation 36 extracts one to several lines of the character pattern string stored in 32 as necessary, enlarges or reduces the pattern string to generate a line pattern image of a size that is easy to see, and stores it in 33. This is a function to The above image information is displayed by 30.

Add line 20 is a function for registering a pattern string in an image as a character line. An example of this operation is shown in FIG. 4. It is assumed that a pattern string 42 is a character pattern string that has not been extracted as a character line. In this case, the chain relationship of lines is (→41
→43→). When you specify line addition processing, the system displays a submenu. The auxiliary menu includes ``Specify pattern'', ``Front attachment'', ``Retro attachment'' and ``
If you specify 6 points with 4 items of ``Independent Article'', select ``Independent Article'' on the menu.
Select "Pattern Designation" and designate the diagonal of the rectangle 44 that includes the processing target pattern row 42 by designating two points. The system extracts only the pattern included in the rectangle 44, records it as a character line, and displays its circumscribed rectangle. There are three types of processing after specifying a pattern: "front matter", "post matter", and "independent article".
Or you can specify whether to register it as an independent article 0
For example, if you select "Retrofit" on the menu and specify the character line 41 to which you want to connect the character lines by specifying a point, the system will insert 42 after 41 and write the line chain relationship information (→41→
42→43→). If you want to interrupt the above process and respecify the pattern, just specify the line addition process on the main menu. At this time, changes that have already been made will not be undone. If you wish to continue with another process after the line addition process is complete, you can specify another process item on the main menu.

The line deletion 21 is a function for deleting a pattern string in an image registered as a character line from the it8. An example of this operation is shown in FIG. Assume that line 52 is a pattern string that is not a character line, but is mistakenly determined to be a character string. In this case, the chain relationship of lines is (→51→52
→53→). When you specify line deletion processing, the system displays an auxiliary menu. The auxiliary menu has one item, ``Specify 1 line.'' After specifying a point, select "line specification" on the menu, and then specifying a point again to specify the outer periphery or inside of the rectangle of the line 52 to be processed. The system deletes line 52 from the character line registration and updates the line chain relationship information (→
51→53→). If you want to interrupt the above process and respecify the pattern, just specify the line deletion process on the main menu. At this time, changes that have already been made will not be undone. Furthermore, if you wish to continue with other processing after completing line deletion, you can specify another processing item on the main menu.

Line division 22 is a function for dividing a character line into a plurality of lines. An example of this operation is shown in FIG. 6, where it is assumed that a one-line circumscribed rectangle 62 is a character line that should originally be divided into two. At this time, it is assumed that the chain relationship of lines is interpreted as (→61→62→63→). Systems that specify line-splitting processing display an auxiliary menu. The auxiliary menu has four items: 1 line specification, ``left/right split,'' ``vertical split,'' and ``free split.''

Point designation After selecting "line designation" on the C menu, point designation is again used to designate the outer periphery or inside of the rectangle of the process target row 62.

The system enlarges and displays the character pattern string in row 62 to an appropriate size so that the details of the pattern can be clearly seen. Next, select "Left and Right Split" on the menu and specify the point to 64 points.
When instructed, the system divides the character pattern string left and right from point 64, generates and registers two character lines 65 and 66, and sets the line chain relationship information to (→61→65→66→63→). change. Similarly, specify the dividing point for "vertical division". "Free division" specifies a cutting line and divides along this line.

For example, in the case of horizontal writing, the chain relationship of dividing lines is automatically changed so that the upper left side is placed first. Hereinafter, the interruption of processing and the designation of other processing are the same as in each of the above processes.

Line merging 23 is a function for merging multiple character lines into one line. An example of this operation is shown in FIG. Text lines 72 and 7
3 is originally one line, but it is extracted by being divided into two lines. In this case, the chain relationship of lines is (→71→72
→73→74→). When you specify row merging processing, the system displays an auxiliary menu. The auxiliary menu has two items: 1 line specification and 1 line specification end. Select "line designation" on the menu by specifying a point, and then specify lines 72 and 73 in succession by specifying a point again. When you have finished specifying 0 lines, which can be specified further, specify "End line specification" on the auxiliary menu. The system generates and registers a line 75 corresponding to a character pattern string that is a combination of two lines, and changes the line chain relationship information to (→71→75→74→). Hereinafter, the interruption of processing and the designation of other processing are the same as in each of the above processes.

The article generation 24 is a function for separating one or more lines from the article to which each line belongs and registering them all as one separate article. An example of this operation is shown in FIG. 8, where it is assumed that single character lines 85 and 88 do not originally belong to articles 81 and 82, but are lines that should constitute another article. At this time, suppose that the line chain relationship information is interpreted as [81=(→83→84→85)] and (82=(→86→87→88)).The system for specifying the article generation process is is displayed.The auxiliary menu has two items: 1 line specification'' and ``End line specification.'' Select ``Line specification'' on the menu by specifying a point, and then continue to lines 85 and 88 by specifying a point. Specify.You can also specify it continuously.Also, you can specify multiple consecutive lines at once by drawing a trajectory passing through the line area by specifying a line.Each line in the generated article will be Sf the chain relationship in order, and when you finish specifying one line to be recorded, specify "End line specification" on the auxiliary menu.The system generates and registers a new article 89 and registers lines 85 and 88 in the article. Then, the row chain relationship information is set to [81=(→83→84)] and (82=(→86
→87)] and (89=(85→88)).The interruption of the process and the designation of other processes are the same as those described above.

The article division 25 is a function for dividing an article into a plurality of articles line by line. An example of this operation is shown in FIG. Article 9
1 is an article to be split between rows 93 and 94. In this case, the chain relationship of lines is [91:(→92→93→
94→95→)].

When article division processing is specified, the system displays an auxiliary menu. The auxiliary menu includes "Split left and right" and "Unsplit"
There are three items of "free division". If you select "Vertical Split" on the menu by specifying a point and then specify point 96 again by specifying a point, the system will divide article 91 vertically between lines 93 and 94 and generate articles 97 and 98. Register the row chain relationship information (97 = (→92 → 93)) and (98 = (94
→95→)]. In addition, "left and right division"
The "free division" can also be operated in the same way as the line division 22. Hereinafter, the interruption of processing and the designation of other processing are the same as in each of the above processes.

Article merging 26 is a function for merging multiple articles into one article. An example of this operation is shown in FIG. Article 1
It is assumed that 01 and 102 are originally one article. In this case, the chain relationship of lines is [101=(→103→104)) and (
102=(105→106→)]. When article fusion processing is specified, the system displays an auxiliary menu. The auxiliary menu has one item: "line specification". Select "Specify one line on the menu by specifying a point," and then specify lines 104 and 105 in succession by specifying a point again. The system connects rows 104 and 105, generates and registers article 107 that includes articles 101 and 102, and stores row chain relationship information (107=(→103→104→105→10
6→)]. Hereinafter, the interruption of processing and the designation of other processing are the same as in each of the above processes.

The line chain relationship change 27 is a function for respecifying the order of character lines within an article. An example of this operation is shown in FIG. 11, where eight rows 114 that should follow row 111 are connected to row 113. At this time, the row chain relationship information is (111→1
12→113→114→115). When you specify row chain relationship change processing, the system displays a submenu. The auxiliary menu has three items: 1 line designation, front attachment, and back attachment. Select "Specify one line by specifying a point" on the menu, then specify the line whose connection you want to change by specifying a point or line. In this case 114 and 1
Specify 15.

Next, select "Retrofit" on the auxiliary menu and then line 11
If you specify 1, the system connects rows 114 and 115 after row 111 and creates row chain relationship information (111→114→
115→112→113). below,
Interruption of processing and designation of other processing are the same as each processing described above.

The position information calculation 28 is a function for calculating position information from input from the pointing device.

The command interpretation 29 is a function of interpreting the input from the auxiliary switch and the position information from the above 28 to determine the processing.

The display 30 is a function of displaying line information, article information, operation instructions, menus, etc. to the user.

The control module 31 has a function of controlling each of the above blocks according to the process determined in step 29 above.

The following example will be given as a modification of this embodiment.

By adding a key-powered device such as a keyboard as a component and assigning a number to each item in the menu, the user can input the item number or the number of the line or article to be processed from the key-powered device. good.

Further, the present invention may be applied to a document recognition device that uses a method of analyzing grammar, meaning, etc. and extracting a semantic structure.

The original image may be displayed in addition to the rectangle surrounding the example character. Numbers may be displayed adjacent to lines and articles to indicate the connection relationship and order of lines and articles.

Further, connection relationships may be indicated by displaying arrows between lines or between articles.

In short, the present invention can be modified in various ways without departing from its gist.

〔Effect of the invention〕

According to the present invention, a user can quickly modify the interpretation of the semantic structure of an automatically extracted document.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of the entire apparatus in an embodiment according to the present invention. FIG. 2 is a diagram showing the flow of processing in an apparatus according to an embodiment of the present invention. FIG. 3 is a configuration diagram of a correction processing system in an apparatus according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of a row adding operation in an apparatus according to an embodiment of the present invention. FIG. 5 is a diagram showing an example of a line deletion operation in an apparatus according to an embodiment of the present invention. FIG. 6 is a diagram showing an example of a line dividing operation in an apparatus according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an example of a row merging operation in an apparatus according to an embodiment of the present invention. FIG. 8 is a diagram showing an example of an article generation operation in an apparatus according to an embodiment of the present invention. FIG. 9 is a diagram showing an example of an article dividing operation in an apparatus according to an embodiment of the present invention. FIG. 10 is a diagram showing an example of an article merging operation in an apparatus according to an embodiment of the present invention. FIG. 11 is a diagram showing an example of a row chain relationship changing operation in an apparatus according to an embodiment of the present invention. 1... Image input device, 2... Control device. 3...Display device, 4...Storage device. 5... Indication device, 6... Auxiliary switch agent Patent attorney Nori Ken Chika Hikaru Matsuyama Figure 1 Figure 2 Figure 3 Explanation■■■-■11-8 Figure 4 Roninji 2==]-53 Figure 5 I ( Figure 6 Figure 7 Figure 9

Claims (1)

[Claims] (1) A character line extraction unit that extracts character lines from a document image;
a display section that displays information on the character line extracted by the character line extraction section in an identifiable shape; an instruction section for indicating a point or area of the character line information displayed on the display section; A document processing device characterized by comprising means for modifying the structure of the corresponding character line information according to instructions from the instruction unit (2) The display unit displays character lines as circumscribed polygons and also displays semantic groups of lines. 2. The document processing apparatus according to claim 1, wherein the document processing apparatus is configured to display the data in the color of a circumscribed polygon or a polygon surrounding the group, and to indicate the chain relationship in the group by a number beside the line.