JPH10207981A - Document recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To process even documents somewhat different with respect to physical layouts in the case of documents of the same logical layout by using logical layout information to fix frame areas corresponding to item frames and entry frames on a document picture to be processed. SOLUTION: A form consists of a title 301 and plural item frames as frames surrounded with ruled lines, and item name frames 302 to 304 where item names are preliminarily printed and entry frames 305 to 307 where item information is to be written for individual document by manual operation or the like are provided as item frames. As shown by (b), the title 'XX application' of the form exists n the first row, and the item name frame 302 of 'applicant' and the entry frame 305 for 'name' are arranged under it in a line in order in the second row, and the item name frame 303 of 'address' and the entry frame 306 for 'address' are arranged under them in a line in order in entry frame 307 for 'telephone number' are arranged under them in a line in order in the fourth. Thus, the flexibility for difference in physical layout is improved.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the field of processing technology for forms or similar tabular documents (these documents are collectively referred to as "forms" in the present specification). It relates to the technology for recognizing the entered contents.

[0002]

2. Description of the Related Art When reading contents written on a form,
It is necessary to identify which part of the form is the entry frame to be recognized. Various techniques have been known for such form recognition. Conventionally, however, any of the conventional techniques uses the physical layout information of a form prepared in advance, more specifically, the coordinates of the form using the coordinates of the form. Extracted (for example, Japanese Patent Publication No. Hei 7-109610).

[0003]

The method of using the physical layout information of a form requires precise correction of the displacement and magnification of the image of the input form, which complicates the processing and increases the speed of the processing. difficult. Further, even if the logical layout of the form (frame type, relative positional relationship of the frame, the attribute of the content to be entered in the entry frame, etc.) is the same, even if the physical layout is slightly different, it is not possible to cope with the difference. There is no flexibility for

[0004] An object of the present invention is to provide a form recognition method that can solve such problems of the prior art.

[0005]

The gist of the present invention is to prepare logical layout information in which the layout of the item name frame and the entry frame of a form is defined as a relative positional relationship between them, and based on the input form image. By extracting a frame region and comparing the relative positional relationship of the extracted frame region with the logical layout information, a frame region corresponding to the item name frame and the entry frame of the input form image is identified, and at least the entry frame (1) is to perform character recognition in a frame area corresponding to (1).

Preferably, the logical layout information includes the item name of the item name frame, and when identifying a frame region corresponding to the item name frame of the input form image, character recognition of the frame region is performed. A check is made to determine whether the character recognition result matches the item name of the corresponding item name box in the logical layout information (claim 2). The logical layout information also includes the attribute of the content of the entry frame, and the attribute of the corresponding entry frame in the logical layout information is used for character recognition of a frame area corresponding to the entry frame of the input form image. (Claim 3).

[0007]

Embodiments of the present invention will be described below with reference to the drawings.

FIGS. 1 and 2 are a block diagram showing the configuration of a form recognition apparatus according to an embodiment of the present invention and a flowchart showing the flow of the processing, respectively. 3 and 4 are diagrams respectively showing an example of a form and an example of their logical layout information.

In FIG. 1, reference numeral 101 denotes an image input unit for reading and inputting a form as a binary image, 102 an image memory for storing an image of the input form, and 103 temporarily storing data relating to processing. Memory for storing,
104 is a character recognition unit for performing character recognition processing; 109 is an attribute-based knowledge database storing knowledge such as words for each attribute used in the character recognition processing; 105 is a ruled line extraction unit for performing vertical and horizontal ruled line extraction processing And 106, a frame region extracting unit for extracting a frame region surrounded by ruled lines on four sides from the ruled line data.
Reference numeral 7 denotes a logical layout processing unit for performing a process related to a logical layout of a form; 108, a logical layout structure database storing logical layout information of various forms;
0 is an output data editing unit that edits output data, 111 is an output unit that outputs edited output data, and 112 is a control unit that controls the operation of the entire apparatus.

It should be noted that the form recognition apparatus 10 having such a configuration is used.
Obviously, 0 can be realized by software using a general computer system. In this case, for example, 104, 105, 106, 10
The functional units such as 7, 110 are described as programs, and these programs are stored in a recording medium such as a hard disk, read into a memory of the system as needed, and executed by the CPU. The logical layout structure database 108 and the attribute-specific knowledge database 109 will be stored in a storage medium such as a hard disk.

Here, a method of defining logical layout information of a form will be described with reference to FIGS.

The form shown in FIG. 3A has a title 301.
And a plurality of item frames, which are frames surrounded by ruled lines. As these item frames, an item name frame 3 in which item names are printed in advance
02, 303, 304, and entry frames 305, in which item information is entered by hand or mechanically for each form.
306 and 307. Although not shown,
At a specific position in the blank portion of the form, for example, at the lower right corner, a mark indicating the type of the form (a character or the like capable of character recognition) is printed in advance.

FIG. 3B schematically shows the logical layout information defined for the form so that the logical layout information can be easily compared with the logical layout of the form. Tytle on the first line
{xx application form} indicates that the title of the form is “xx application form”. 2nd to 4th line item {applicant}, item
{Address} and item {Phone} are the item name box 302, respectively.
303 and 304 are shown. item {xxxx} means an entry frame, and xxxx following {} in {} represents an attribute of information to be entered in the entry frame. These attributes include name (personal name), address (address), tel (telephone number), number (number), money (money), and count (quantity) and ignore (ignore) shown in FIG. ). Each of the plurality of items {} connected by & represents each of the plurality of item frames arranged in a line from left to right in the order of description. The uppermost item frame row on the form is described first, and the same item frame row is described in order from the leftmost item frame.

Therefore, in FIG. 3B, the first line means that there is a title of “XX application form” on the form, and the second line has an item name box 302 of “applicant” below the title. And the entry boxes 305 of “personal names” are arranged in order in a horizontal line, and the third line is an item name box 303 of “address” and an entry box 30 of “address” below.
6 means that there is an order in the horizontal line, and the fourth line has an item name box 304 of “telephone” and an entry box 307 of “telephone number” in the lower line in order. Means that.

The form shown in FIG.
And item name boxes 402 to 406, and entry boxes for each item. A mark such as a number indicating the form type is also printed, for example, in the lower right corner of the form. The logical layout information of this form is defined as shown in FIG. It should be noted that 7 in FIG.
From the line, item {ignore} & item {ignore} describes the entry frames 407 and 408 located at the bottom of the form, and since the attribute is "ignore", the contents thereof can be ignored.

It should be noted that the form handled in the present embodiment does not have a title surrounded by a frame as in the example of the form shown above, and the item frame at the upper left position is always an item name frame.

Next, the flow of processing in the form recognition apparatus 100 of this embodiment will be described with reference to the flowchart shown in FIG.

First, a form image is input by the image input unit 101 and stored in the image memory 102 (step 201 in FIG. 2). It should be noted that, unlike the related art, the correction of the position shift and the scaling of the form image is not performed. This is because the processing for the following item frames is performed based only on the logical layout information, so that there is no problem even if there is a slight displacement of the image or a difference in the physical layout of the item frames due to scaling.

When the form image is input, the character recognition unit 10
In step 4, a character image such as a number indicating the form type is cut out from the lower right corner of the form image in the image memory 102 and character recognition is performed (step 203). The logical layout processing unit 107 identifies the type of the input form from the recognition result, and selects the logical layout information of the relevant form type in the logical layout structure database 108 (step 2).
05). The selected logical layout information is used for processing the input form. When this selection cannot be made, that is, when an image of a form of which the logical layout information is not registered in the logical layout structure database 108 is input, it is determined that there is an abnormality (step 20).
7) The control unit 112 abnormally terminates the process by issuing an error warning to the user. It is also possible to adopt a method in which the user specifies the form type and selects the logical layout information according to the specified type.

When the logical layout information corresponding to the type of the input form is normally selected, the ruled line extracting section 105 first extracts horizontal and vertical lines from the input form image in order to extract a frame area from the input form. Is extracted (step 209). This ruled line extraction is performed, for example, by detecting horizontal and vertical black runs having a length equal to or more than a predetermined value from an input form image and integrating extracted black runs within a certain distance. . Then, the frame region extracting unit 107 extracts the logical region of the ruled lines in the horizontal direction and the vertical direction, thereby extracting the region of the frame surrounded by the ruled lines (step 211). The coordinates of the extracted frame area, for example, the coordinates of two opposing vertices are stored in the memory 103.

When the frame area is extracted in this manner, the logical layout processing unit 107 sets one of the extracted frame areas located at the uppermost and leftmost as the current frame area, and sets its coordinates as the current frame area. Is designated to the character recognizing unit 104, and character recognition is performed inside the frame area (step 215). The logical layout processing unit 107 compares the recognition result with the first item in the selected logical layout information.
Check for a match with the item name in {} of {} (Step 2
15). The first item frame is always an item name frame, and if the frame area extraction is normal, it should be possible to obtain a match by this comparison.
For example, in the case of the form shown in FIG. 3A, the current frame area corresponds to the item name frame 302, and the recognition result should match the item name “applicant”. If a match is found, step 2
The process proceeds to 21 and subsequent processes.

If they do not match, it is possible to immediately terminate the processing as an error, but in this embodiment, in order to attempt to recover from this abnormal state, the coordinates of the frame area adjacent to the current frame area are Is designated to the character recognizing unit 104, and the character recognition is performed (step 217), and it is checked whether the recognition result matches the head item name (step 21).
9). If they do not match, it is determined that an error has occurred, and the control unit 112 abnormally ends the processing. However, when a match is found in any adjacent frame area, the matched frame area is set as the current frame area, and the process proceeds to step 221 and subsequent steps.

Note that the first frame area can be identified as the first item name frame without character recognition based only on its relative positional relationship, that is, only at the upper left position. is there. However, the risk of identification errors increases. This is the same for other item name frames (steps 229 and 231 described later are omitted, and only the matching between the relative positional relationship of the frame area and the relative positional relationship described in the logical layout information is used as a basis. , It is also possible to identify the item name frame immediately).

When the first item name frame can be confirmed in this way, the logical layout processing unit 107 refers to the selected logical layout information and sets another item frame to the right of the item frame corresponding to the current frame area. Check if it exists (Step 2
21). In the case of the form shown in FIG. 3A, since item {applicant} has an item {name} connected with & next to the item {applicant} in the second line of FIG. 305). In this case, referring to the coordinate data of the frame area in the memory 103,
It is checked whether a frame area corresponding to the item frame on the right side exists (step 225). If the frame area on the right side does not exist, the extraction of the frame area is abnormal, so it is determined that an error has occurred, and the processing is abnormally terminated.

If the frame area exists, the frame area is set as the current frame area, and the description of the corresponding item frame in the logical layout information is referred to, to determine whether the item frame is an item name frame or an entry frame. Check (step 227). In the case of the entry frame, the attribute and the coordinates of the current frame area are designated to the character recognizing unit 104 to perform the character recognition (step 23).
5). At this time, the knowledge in the attribute-specific knowledge database 109 corresponding to the given attribute is used to improve the accuracy of the recognition result. In the case of the form shown in FIG. 3A, the current frame area corresponds to the entry frame 305, and the logical layout information includes item @
Since name @ is described, the attribute of the content is "person name", and the knowledge of "person name" is used for character recognition. The recognition result is stored in the memory 103.

When the step 235 is completed, the logical layout processing unit 107 returns to the step 221 and refers to the logical layout information to check whether there is an item frame on the right side of the item frame corresponding to the current frame area. In the case of the form shown in FIG. 3A, there is no item frame connected by & to item {name} on the second line in FIG. 3B. Then, it is checked whether there is an item frame below (step 203). Since item {address} is on the third line in FIG. 3B, an item name box 303 of “address” is shown below.
It turns out that there is. Therefore, in step 225,
It is checked whether there is a frame area corresponding to the item name frame 303 (step 225). If the frame area extraction is normal, there is a corresponding frame area, so that it is set as the current frame area, and since it is an item name frame, the flow proceeds to step 229. In this step 229, the coordinates of the current frame area are designated to the character recognizing unit 104, and character recognition is performed (step 22).
9). The recognition result and the item name “address” in the item name box
Is checked (step 231). If they do not match, the process is terminated abnormally as an error, but if they match, the process is continued from step 221.

The form shown in FIG. 4A is similarly processed. In the case of an entry frame having an attribute of "ignore" such as the entry frames 407 and 408, it is meaningless to perform the character recognition. Therefore, the step 235 is skipped, and a space code or the like is specified as a recognition result. The code of is output.

In this manner, up to the last item frame, FIG.
In the form (a), if the processing is normally performed up to the entry box 305, the output data
The output data is edited based on the recognition result obtained in step 3 (step 237). The format of the output data is appropriately selected according to the purpose of use. For example, if only the contents entered in the entry boxes are desired, the output data can be output in a format in which information indicating the form type and the recognition results of the entry boxes are arranged in order. Then, the edited output data is output to the output unit 111 (step 239).

[0029]

According to the first to third aspects of the present invention, the layout of the item name frame and the entry frame of the form is processed using the logical layout information defined by the relative positional relationship. Since a frame area corresponding to an item frame and an entry frame on a form image is identified, flexibility with respect to a difference in physical layout is high, and a form having a slightly different physical layout can be processed if the logical layout is the same. Yes,
In addition, it is not necessary to perform a process for correcting a position shift or scaling of a form image to be processed, and the process can be simplified and speeded up. Since the logical layout information is prepared more easily than the physical layout information, it can easily cope with a change in the layout of a form or the addition of a new form. According to the second aspect of the present invention, it is possible to avoid an identification error of the item name frame and perform more reliable processing. According to the third aspect of the present invention, the character recognition accuracy can be improved by using the attribute of the entry frame.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a form recognition device according to one embodiment of the present invention.

FIG. 2 is a flowchart showing a processing flow of a form recognition device according to an embodiment of the present invention.

FIG. 3 is a diagram showing an example of a form and its logical layout information.

FIG. 4 is a diagram showing another example of a form and its logical layout information.

[Explanation of symbols]

 101 Image Input Unit 102 Image Memory 103 Memory 104 Character Recognition Unit 105 Rule Line Extraction Unit 106 Frame Area Extraction Unit 107 Logical Layout Processing Unit 108 Logical Layout Structure Database 109 Attribute Knowledge Database 110 Output Data Editing Unit 111 Output Unit 112 Control Unit 301 Form Title 302-304 Item name frame 305-307 Entry frame 401 Form title 402-406 Item name frame 407, 408 Entry frame for "ignore" attribute

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yukio Uchiyama 1-3-6 Nakamagome, Ota-ku, Tokyo Stock inside Ricoh Company (72) Inventor Fumihiro Hasegawa 1-3-6 Nakamagome, Ota-ku, Tokyo Stock Inside the company Ricoh

Claims (3)

[Claims] 1. A logical layout information which defines the layout of an item name frame and an entry frame of a form as a relative positional relationship is prepared, and a frame region is extracted from an input form image, and the extracted frame is extracted. By comparing the relative positional relationship between the regions with the logical layout information, a frame region corresponding to the item name frame and the entry frame of the input form image is identified, and at least character recognition of the frame region corresponding to the entry frame is performed. A form recognition method characterized in that: 2. The logical layout information includes an item name of an item name frame. When identifying a frame region corresponding to the item name frame of the input form image, character recognition of the frame region is performed. 2. The form recognition method according to claim 1, wherein a match between the character recognition result and an item name of a corresponding item name box in the logical layout information is checked. 3. The logical layout information also includes an attribute of the content of the entry frame, and the character recognition of the frame area corresponding to the entry frame of the input form image is performed by the corresponding entry frame in the logical layout information. 3. The form recognition method according to claim 1, wherein the attribute is used.