JPH07141462A - Document system - Google Patents

Abstract

PURPOSE:To obtain a document system capable of coping with a quasiroutine format document by extracting vertical and horizontal ruled lines from a document image, performing a sheet discrimination by the positional information and the length information of the ruled lines and the discrimination information for every form of a discrimination dictionary and performing a reading based on the information on the relative reading item area for every form of the discrimination dictionary. CONSTITUTION:Data composed of the starting coordinate of the ruled line for every document form and the coordinates of the length of the ruled line and a reading item area is stored in discrimination dictionary 6. The image input 2 of a document 1 is performed and the extractions 3 of a horizontal ruled line and a vertical ruled line are performed. Next, a sheet discrimination 4 performs the matching of the extracted ruled lines and the discrimination dictionary 6, determines a reading item area coordinate, performs a character recognition 8 by extracting the characters of a reading item area coordinate range from a document image, and performs a reading item registration or parforms the operations of the reading item registrastion 9 and an image registration 10. Thus, a correction which is easy to be used by a user can be performed by coping with the document of a quasi-routine format and performing the integral display of images and the recognition result window of the reading item.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document system for automatically reading, registering and storing documents having different styles and formats, correcting read items such as titles, and searching.

[0002]

2. Description of the Related Art The format of a document defines the description items and the order of the description items, but does not specify the absolute position coordinates and dimensions. The document format not only defines the description items and the order of the items, but also the absolute position coordinates and dimensions thereof. A document whose style is defined is called a semi-standard format document, and a document whose format is defined is called a standard format document.

In the conventional optical character recognition device, a standard format document can be read by presetting the format of the document as format data. However, a document in which only the style is specified, such as a legal document, can be read. The standard format document could not be read.

The document system related to this type is
As described in Japanese Unexamined Patent Publication No. 56-11573, a mark writing area for instructing a reading item to be input in advance is provided in a document, and a mark written in the area is read so that a predetermined area on the document is read. The method of reading is proposed.

Further, as described in Japanese Patent Application Laid-Open No. 4-15775, there is a method for detecting a table structure by numbering search areas with a predetermined length by a histograph.

Further, the conventional correction of the reading result is described in the 44th national conference of the Information Processing Society of Japan, "Consideration on the erroneous recognition correction interface in the handwritten character recognition system", and the image and the recognition result cut out by the machine for character recognition. Is proposed to display.

[0007]

The above-mentioned prior art has a problem that since the mark writing area is added, the area occupied by the writing area on the paper surface is increased and the number of read items cannot be increased. In addition, since it is necessary to newly prepare a mark writing area, there is also a problem that various existing documents cannot be read as they are. Furthermore, in the case of a document whose format is determined by laws and regulations, it is not permitted to provide a mark entry area as in the past because the format is legally prescribed, and the format document without a mark entry area must be read. There was a problem that I could not.

If the search area is numbered with a predetermined length by a histograph as a method for detecting the table structure, the number is changed because it is easily affected by noise, blurring of the ruled line, and expansion / contraction of the image. There was a problem of causing false recognition.

Since the machine automatically displays the image cut out and the recognition result for the character recognition, the layout as a document disappears and the machine is not convenient for the user.

A first object of the present invention is to provide a document system which can deal with a semi-standard format document by performing sheet identification by position information of ruled lines and coding of ruled lines.

A second object of the present invention is to provide a document system capable of providing a user-friendly display by integrally displaying an input document image whose layout is known and a recognition result.

[0012]

In order to achieve the first object, the document system according to the present invention extracts vertical and horizontal ruled lines from a document image, and the position information and length information of the ruled lines and an identification dictionary. Sheet identification is performed based on the identification information for each style, and reading is performed based on the information of the relative read item area for each style in the identification dictionary. In addition, vertical and horizontal ruled lines are extracted from the document image, and sheet identification is performed based on the information in which the ruled lines are coded according to the positional relationship of the ruled lines and the identification information for each style in the identification dictionary, and the ruled lines for each style in the identification dictionary. The characters inside the read item area are read based on the information of the read item area that is represented by the code.

In order to achieve the second object, an input document having a recognition error in reading the document system according to the present invention is determined, and the location of the recognized read item and the correction information such as the font size and the recognized character are memorized. The input document image that has been recognized as a recognition error in the input document is temporarily registered, and when correcting the document that has been determined as a recognition error, a correction display layout is generated from the input document image and the correction information. I made a correction.

[0014]

The principle and operation of the present invention will be described below.

FIG. 1 shows a state in which a document image is input to perform sheet identification and read items are registered or read items and image data are registered. First, input image 2 of document 1,
The horizontal ruled lines and the vertical ruled lines are extracted 3. Next, the sheet identification 4 performs matching 5 with the extracted ruled line, the identification dictionary 6 and the extraction 7 of the read item area to obtain the read item area coordinates, extracts the characters in the read item area coordinate range from the document image, and performs the character recognition 8 for reading. Item registration 9 is performed. Alternatively, the operations of reading item registration 9 and image registration 10 are performed.

FIG. 2 shows how the sheet is identified by the starting point coordinates and the length of the ruled line. Document 1 performs image input 2 and extracts ruled lines from the input document. The extracted ruled lines are vertical ruled lines,
After the division into horizontal ruled lines, ruled line starting point length data 23 including the starting points 21 and the lengths 22 of the respective ruled lines is obtained. Then, matching 27 is carried out with the identification format data 26 consisting of the starting point 24 and the length 25 of the ruled line of the identification dictionary 28 in which the identification data of the sheets of various styles are registered in advance, and the identification consisting of the style number and the read item area data is carried out. The result 29 is extracted from the identification dictionary 28.

FIG. 3 shows the sheet identification matching 27 of FIG.
A method of reading item area extraction 7 in FIG. 1 will be described.

First, the input ruled line length data 23 and the identification format data 26 of the identification dictionary 28 are compared with each other to perform the identification dictionary registration number error extraction 31. The identification dictionary registration number error extraction 31 is based on the length of the ruled line starting point length data 23 and the identification dictionary 2
8 to obtain the sum of the lengths of the identification format data 26, and compare the values with the length error extraction 32, the starting point coordinates of the ruled line starting point length data 23 and the starting point coordinates of the identification format data 26 in the identification dictionary 28. Starting point coordinate error 1 extraction 33 for comparing the respective X coordinates, starting point coordinate error 2 extraction 34 for comparing the respective starting Y-coordinates of the ruled line starting point length data 23 and the starting point coordinates of the identification format data 26 in the identification dictionary 28. , The difference between the ruled line start point length data 23 and the identification format data 2 is calculated by obtaining the value of the X coordinate difference between the start point coordinates of the adjacent ruled lines in each data of the ruled line start point length data 23 and the identification format data 26 in the identification dictionary 28.
6 Extraction of coordinate error 3 extraction 35 for comparing difference values, ruled line starting point length data 23, and Y format difference value of the starting point coordinates of adjacent ruled lines in each data of the identification format data 26 in the identification dictionary 28. A starting point coordinate error 4 extraction 36 for comparing the difference between the starting point length data 23 and the difference value of the identification format data 26 is performed.

As a result, the identification format data with the minimum error is obtained from the ruled line starting point length data 23 and the identification dictionary 2 is obtained.
8, the operation of extracting the format number and reading position data 37 is performed.

FIG. 4 shows how ruled lines are coded from ruled line start and end point data for coding a ruled line to identify a sheet and to obtain ruled line related data. Hierarchical coding 4 based on the ruled line data 42 consisting of the start and end coordinates of the input ruled line 41
3 is performed to obtain ruled line code data 45 in which a code is assigned to each ruled line like the coded ruled line 44. As a result, the code for each ruled line is as follows. First, regarding the vertical ruled line, ID number 1 is y1, ID number 2 is y1-1, I
The D number 3 becomes y1-1 + 1 and the ID number 4 becomes y1-2. For horizontal ruled lines, ID number 1 is x1, ID number 2
Becomes x2 and the ID number 3 becomes x2 + 1. Then, from the ruled line data 42 and the ruled line code data 45, the connection relation of the start point coordinates and the end point coordinates of the horizontal ruled line on each vertical ruled line and the start point coordinates and the end point of the vertical ruled line on each horizontal ruled line The connection relationship of the coordinates is obtained, and as a result, the starting point relationship of the vertical ruled line is x1, x2, x2 + 1 with respect to y1, and the ending point relationship is y1-1.
X1 and y1−2, the relational expressions of x2 and x2 + 1 are extracted, and the starting point relation of the horizontal ruled line is y1 with respect to x1,
y1-1, y1-1 + 1 and x2 become y1-2, and the end point relation is x1-1, y1-1 + 1 for x2,
The operation of creating the ruled line relational data 46 for extracting the relational expressions of y1 and y1-2 with respect to x2 + 1 is performed.

FIG. 5 illustrates the method of encoding 43 of FIG. First, numbering 51 is performed for each ruled line of the first layer that does not overlap, based on the coordinates of the start point and the end point of the ruled line. Next, for the ruled lines of the second layer and below, search for the ruled line of the upper layer, if the line is longer than that rule, search for the ruled line of the upper layer, if the same, add a + sign to number, and if it is short, add a sign. Numbering 52 is performed. This overlap confirmation and layering will be described by dividing the ruled line 41 into vertical and horizontal ruled lines. The vertical ruled lines are seen in the direction of the arrow 53 (X direction) and overlap confirmation and layering are performed. The horizontal ruled line is the direction of arrow 54 (Y direction)
See and check overlap and layer. As a result, the first layer becomes the vertical ruled line 55 and the horizontal ruled line 56, and the second and subsequent layers become the vertical ruled line 57 and the horizontal ruled line 58.

FIG. 6 shows how the ruled line relation data 46 obtained in FIG. 4 is matched with the identification dictionary. The dictionary A63 and the dictionary B64 of the identification dictionary 62 including the ruled line relation data 46, the frame structure number, Don't Care data, the ruled line relation data, and the reading frame code.
Are matched by the error calculation method 65. Specifically, the error calculation method 65 gives an error of 10 in the relational number and the relational expression, and an error of 1 in the equal part of the relational expression. As a result, dictionary A
63 = error = 0, dictionary B64 =
The error value of 51 is extracted. As a result, the structure number 1 of the dictionary A63 having the smallest error and the read item code data 66 are output, and the read item code data 66 is the target ruled line extraction 6
7. The read item area extraction 68 is performed by a linear method.

FIG. 7 shows how Don't Care is used for matching. In the ruled line start point / end point data 41 of FIG. 4, if the blur is one, 71 is the ruled line related data, and if it is one noise, 73 is the ruled line related data 74. Then, from D = y1-1 * in the dictionary A63, y
When matching is done with Don't Care for items less than 1-1 and y1-1, the underlined parts of the ruled line related data 72, 74 are ignored and matching is performed, resulting in an error of 0 in either case. Do.

FIG. 8 shows a state in which the read item of the input document is automatically recognized and the correction is performed when there is a recognition error. When the document 1 is input and the characters of the read item are recognized by the automatic recognition 81 and there is a recognition error from the recognition error judgment 82, the recognition error image data, the recognized character of the read item, the font size, and the read item area coordinates are used. The recognition error information is recognized and the recognition error image data / recognition error information temporary storage 83 is operated. When a document having a reading error is to be corrected, a correction screen is created by the read item correction display layout creation 84, and the correction work is performed by the image / read item integrated display correction 85, and the image data / read item information registration 86 is performed. Also, the recognition error determination 82
If there is no more recognition error, the image data / read item information registration 86 is performed in the storage device 86.

FIG. 9 shows a state of the display method at the time of correction of FIG. The recognition result window 92 is integrally displayed on the document image 91 around the reading position. Specifically, the recognition result window 97 of the reading item 1 is displayed below the document image portion 96 of the reading item 1, and the recognition result window 9 of the reading item 2 is displayed below the document image portion 98 of the reading item 2.
The integrated display a93 in which 9 is displayed changes as follows according to an instruction from the user or the like. In the case of a recognition error of item 2, an integrated display b94 in which only the recognition result window 99 of the read item 2 is displayed below the document image portion 98 of the read item 2. Alternatively, in order to make the document image portion 98 of the read item 2 easy to see, the recognition result window 97 of the read item 1 is displayed on the document image portion 96 of the read item 1, and the read item 2 is displayed.
The operation of changing to the integrated display c95 in which the recognition result window 99 of the reading item 2 is displayed under the document image portion 98 of is displayed.

[0026]

EXAMPLES Examples of the present invention will be described in detail below. FIG. 10 is a diagram showing the configuration of the document system according to the present invention. The present system includes an image input device 100 for inputting a document 79 as image data, an automatic recognition device 600 for recognizing read items of an input document, a storage device 200 for registering image data and read items of a document, image data and characters. It comprises a display device 300 for displaying a window, an external instruction input device 400 for inputting an instruction from a user, a search device 700 for searching a registered document, and a control device 500 for controlling each of these devices.

FIG. 11 is a block diagram of an automatic recognition system using the length of the starting point of a ruled line. The constitution is a run data extraction device 601 which is a ruled line information extraction means for extracting continuous black dots of a document image, that is, run data, and a ruled line data extraction for extracting the ruled line data, that is, the start point and end point coordinates of the ruled line from relatively long run data. A device 602, a ruled line data normalization device 603 that corrects the extracted ruled line data as a pre-process for performing collation with an identification dictionary, and a ruled line start point length that extracts ruled line start point data from the normalized ruled line data. Data extraction device 604, sheet identification device 605 that performs sheet identification based on the ruled line starting point length data of the input document and the identification dictionary in identification dictionary device 606, read item area data of identification dictionary device 606 based on the identified result A read item area extraction device 607 for extracting the read item area coordinates based on the Band consists of character recognition character recognition apparatus 608.

Next, referring to FIG. 12 and FIGS.
An automatic recognition processing flow using the starting point length of the ruled line will be described using 0 and 11. First, in the image input A1 of FIG. 12, the document 79 is input from the image input device 100 shown in FIG. 10, and the document image input by the control device 500 is transferred to the automatic recognition device 600. In the run data extraction A2, run data is created by obtaining the start and end coordinates of the connected black dots from the document image, and the run data whose run length is longer than a preset value is extracted.

This is executed by the run extraction device 601 shown in FIG. The ruled line data extraction A3 searches the connection of the run based on the run data and defines the connected run as one ruled line (an outline is shown in the ruled line extraction 3 in FIG. 1). The slope of the ruled line is calculated using the coordinate values, and the ruled line data (details are shown in the ruled line data 23 in FIG. 2) including the start point end point coordinates and the slope are extracted. This is processed by the ruled line data extraction device 602 shown in FIG. The ruled line data normalization A4 of FIG. 2 is performed as a pre-process for identifying the document format using the ruled line starting point length data 23 of FIG. 2, that is, for consistency with the identification dictionary device 604 of FIG. Based on the normalized data in the identification dictionary 28,
Correct the parallel movement. This is performed by the ruled line data normalization device 602 in FIG. Specifically, the inclination correction of the ruled line data is performed by correcting the start point and end point coordinates of each ruled line data using the inclination obtained in the ruled line data extraction A3.

Further, for the expansion / contraction and parallel movement correction, a reference line having the same ruled line as the ruled line registered in the normalized data is extracted from the ruled line data, and the ruled line data reference line and the normalized data reference line are compared. , Calculate the expansion / contraction rate and the amount of parallel movement and correct it.
In the ruled line starting point length data extraction A5 of FIG. 12, the starting point coordinates and length of the ruled line are obtained from the normalized starting point and ending point coordinates of the ruled line data. This is performed by the ruled line starting point length data extraction device 604 of FIG.

In the sheet identification A6, the normalized ruled line data and the identification dictionary 2 in the identification dictionary device 606 (FIG. 11) are used.
8 (FIG. 2) and sheet matching 27 are performed (FIG. 2). This processing is executed by the sheet identification device 605 shown in FIG.

Details of this matching method will be described with reference to FIG. First, ruled line data whose start point coordinates are close to each other is selected from the identification format data 26 registered in the identification dictionary 28, and in the length error extraction 32, the ruled line starting point length data 23 and the length of the identification format data 26 are respectively selected. Calculate the sum of and compare the values.

Next, in the start point coordinate error 1 extraction 33, the start point coordinates of the ruled line start point length data 23 and the start point coordinates of the identification format data 26 in the identification dictionary 28 are compared in the horizontal direction (referred to as the X coordinate). To do. In the start point coordinate error 2 extraction 34, the start point coordinates of the ruled line start point length data 23 and the start point coordinates of the identification format data 26 in the identification dictionary 28 are compared in the vertical direction (referred to as Y coordinates). In the starting point coordinate error 3 extraction 35, for each of the ruled line starting point length data 23 and the identification format data 26 in the identification dictionary 28, the difference regarding the X coordinate of the starting point coordinates of the adjacent ruled lines is obtained, and the ruled line starting point length data 23 is obtained. The difference is compared with the difference in the identification format data 26.

Next, in the start point coordinate error 4 extraction 36, for each of the ruled line start point length data 23 and the identification format data 26 in the identification dictionary 28, the difference in the Y coordinate of the start point coordinates of the adjacent ruled lines is obtained, and the ruled line start point is calculated. Length data 2
The difference in 3 and the difference in the identification format data 26 are compared.

As a result of the above, reading position extraction A7 in FIG.
Then, it is possible to obtain the identification format data having the minimum error with respect to the ruled line starting point length data 23. By referring to the identification dictionary 28, the format number F1 of the format number / read item area data 29 and the read item area Starting point X
Coordinate RX1, starting point Y coordinate RX2, length R from starting point coordinate
L, the width RW from the starting point coordinates is obtained, and the read item area coordinates on the document image are extracted. The read position extraction processing (7 in FIG. 1) is performed by the read item area extraction device 60 in FIG.
It is executed in 7.

Next, in the read item character recognition A8 of FIG. 12, the character of the read item area coordinates obtained in A7 is cut out from the document image and recognized. This is performed by the character recognition device 608 of FIG. In registration A9 of FIG. 12, the character information of the read item, or the image data and the read item are registered in the storage device 200 (FIG. 10).

Next, another embodiment of the present invention will be described with reference to FIG.

FIG. 13 is a block diagram of an automatic recognition system using ruled line coding. The configuration is a run data extraction device 611 that extracts start point / end point coordinate run data of continuous black dots of a document image, and a ruled line data extraction device 61 that extracts ruled line data consisting of start point / end point coordinates from long run data.
2. A frame block extraction device 613 for extracting frame block data consisting of the minimum and maximum coordinate areas of the frame from the run data,
Each frame block ruled line data extraction device 614 that selects ruled line data for each frame block for ruled line coding, ruled line data inclination correction device 615 that corrects and corrects the extracted ruled line data, and the extracted ruled line data as an identification dictionary. Ruled line coding device 61 for extracting coded data of ruled lines based on the positional relationship of the ruled lines as a pre-process for matching
6. Ruled line relation extraction device 617 for extracting ruled line relation data based on ruled line data and ruled line code data 617 Based on ruled line relation data of an inputted document, ruled line code data and identification dictionary device 61
Sheet identification device 618 for performing sheet identification based on the identification dictionary in FIG. 9, read item area extraction device 620 for extracting read item area coordinates based on the read item code data of the identification dictionary device 619 from the identified result, read item The character recognizing device 621 recognizes a character in the area based on the area coordinates.

Based on FIG. 14, FIGS. 1, 4, 5, 6, 7, 1
An automatic recognition processing flow using ruled line coding with 0 and FIG. 13 will be described.

First, in the image input B1 of FIG. 14, the document 79 of FIG. 10 is input from the image input device 100, and the control device 500
Transfers the input document image to automatic recognition 600.

Next, in the run data extraction B2 of FIG. 14, run data is created by checking the start point and end point coordinates of the black dots connected from the document image, and the run length is checked and the run length longer than the preset length is checked. Data is extracted by the run data extraction device 611 shown in FIG.

The ruled line data extraction B3 of FIG. 14 is the ruled line extraction 3 of FIG. 1 in which the run connection is searched based on the long run data and the connected run is set as one ruled line, and the start point end point coordinates and the coordinate value of the ruled line are used. The ruled line data is obtained by obtaining the ruled line inclination value and extracting the ruled line data including the start point / end point coordinates and the inclination value by the ruled line data extraction device 612 in FIG. 13.

The frame block extraction B4 of FIG. 14 extracts the contour data of the black dots according to the run connection relation based on the input run data, selects the contour data of the frame from this contour data, and selects the minimum coordinates of the frame. The frame block data extraction device 613 of FIG. 13 extracts the frame block data composed of the block area data of the maximum coordinates.

Next, the ruled line data extraction B5 for each frame block in FIG. 14 extracts ruled line data for each frame block for ruled line coding based on the ruled line data and the frame block data.
This is performed by the ruled line data extraction device 614 for each frame block.

Next, the ruled line data inclination correction B6 of FIG.
With respect to the ruled line data obtained by the ruled line data extraction 614 for each frame block in FIG. 13, the ruled line data inclination correction device 615 performs inclination correction of the start point / end point coordinates of the ruled line data by the inclination value calculated by the ruled line data extraction device 612. 4 ruled line data 42 is obtained.

The ruled line coding B7 in FIG. 14 is the ruled line data 42.
As the pre-processing for identifying the sheet, the ruled line encoding device 616 of FIG. 13 performs the hierarchical encoding 43 of FIG. 4 by the method of FIG. 5 as follows. First, numbering 51 is performed for each ruled line of the first layer that does not overlap, based on the coordinates of the start point and the end point of the ruled line. Next, for the ruled lines of the second layer and below, search for the ruled line of the upper layer, if the line is longer than that rule, search for the ruled line of the upper layer, if the same, add a + sign to number, and if it is short, add a sign. Numbering 52 is performed. This overlap confirmation and layering will be described by dividing the ruled line 41 into vertical and horizontal ruled lines. The vertical ruled lines are seen in the direction of the arrow 53 (X direction) and overlap confirmation and layering are performed. Horizontal ruled line is arrow 5
When viewed in the direction 4 (Y direction), overlap confirmation and layering are performed. As a result, the first layer has vertical ruled lines 55 and horizontal ruled lines 56.
Therefore, the vertical ruled line 57 and the horizontal ruled line 58 are applied to the second level and below.
As a result, the code for each ruled line is that the vertical ruled line has y1 for ID number 1, y1-1 for ID number 2, y1-1 + 1 for ID number 3, and y1 for ID number 4. Two
The horizontal line is x1 for ID number 1 and x for ID number 2.
2, ID number 3 becomes x2 + 1. Ruled line code data 45
Ask for.

Since the ruled line relation data creation B8 in FIG. 14 performs matching with the identification dictionary device 619, the ruled line relation extraction device 616 in FIG. 13 causes the ruled line data 42 and the ruled line code data 45 to be placed on each vertical ruled line with respect to the vertical and horizontal ruled lines. On the other hand, the connection relationship between the start point coordinates and the end point coordinates of the horizontal ruled line and the connection relationship between the start point coordinates and the end point coordinate of the vertical ruled line on each horizontal ruled line are obtained. As a result, the start point relationship of the vertical ruled line is x1, x2 for y1. , X2
+1 becomes the end point relationship, and the relational expressions of x1 for y1-1 and x2, x2 + 1 for y1-2 are extracted, and the start point relationship of the horizontal ruled line is y1, y1-1, y1-1 + for x1.
1 and x2 becomes y1-2, and the end point relations are y1-1 and y1-1 + 1 for x2 and y1, y for x2 + 1.
Ruled line relation data 46 for extracting the relational expression 1-2 is extracted.

Next, in the sheet identification B9 of FIG. 14, the sheet identification device 618 performs matching of the sheet identification of FIG. 1 on the basis of the extracted ruled line code data, ruled line relation data and the identification dictionary device 619. Follow the procedure below. Ruled line related data 46, frame structure number, Don'tC
The error calculation method 65 performs matching between the dictionary A63 and the dictionary B64 of the identification dictionary 62 including the dictionary structure 61 composed of are data, ruled line relation data, and reading frame code. Specifically, the error calculation method 65 gives an error of 10 in the relational number and the relational expression, and an error of 1 in the equal part of the relational expression. As a result, an error value of error = 0 for the dictionary A63 and an error value of 51 for the dictionary B64 is extracted. Also Don'tCa
In the case of re-matching, the ruled line start point / end point data 41 of FIG. 4 has ruled line relational data 72 when there is one blur 71 and ruled line relational data 74 when there is one noise. And from D = y1-1 * of dictionary A63, y1-1 and y1-1
When the following items are matched as Don't Care, the underlined parts of the ruled line related data 72 and 74 are ignored and matching is performed. As a result, the error becomes 0 in either case, and the minimum error dictionary A63 To match.

As a result, read item area extraction B in FIG.
10 from the read item area extraction device 620 of FIG.
As the reading position extraction 7 of the minimum error dictionary A63 in the identification dictionary 62 of FIG.
From 6, the target ruled line extraction 67 extracts the ruled line data corresponding to the read item, and the read item area coordinates on the document image by the linear equation are extracted.

In the read item character recognition B11 of FIG. 14, the character of the output read item area coordinate is cut out from the document image and recognized by the character recognition device 621 of FIG. 11, and the character information or image of the read item of registration B12 of FIG. Data and character information are registered in the storage device 200.

FIG. 15 is a block diagram of a system for performing automatic recognition and correcting a document having a recognition error in a read item. An image input device 800 for inputting image data of the document 79, an automatic recognition device 807 for recognizing a read item of the input document, a determination device 808 for judging a recognition error as a result of the automatic recognition, and an error in recognition of a read item. Temporary storage device 8 for temporarily storing image data of a document with
02, a recognition information storage device 809 that temporarily stores recognition error information including a recognition character of a reading item, a font size, and a reading item area coordinate of a document having an error in recognizing the reading item; In the case of performing a correction display layout creating device 803 for creating a layout of a correction display, a display device 804 for displaying image data and a character window, an external instruction input device 805 for inputting an instruction from a user, and a search device for searching a registered document. 8
10, a control device 806 for controlling each of these devices.

Next, a method of automatically recognizing the read item of the input document and correcting the case where there is a recognition error will be described with reference to FIGS.

First, the document 79 in FIG. 15 is the image input device 80.
The automatic recognition 81 of FIG. 8 is performed by the automatic recognition device 807 of FIG. 15 to automatically extract and recognize the characters of the read item of the document. The recognition error determination of FIG. 8 is performed by the determination device 808 of FIG. 15 based on this recognition result, and the image data of the recognition error document and the recognition error information temporary storage 83 of FIG. 8 are temporarily stored in the temporary storage device 802 and the recognition information storage device 809 of FIG.
Accumulate in. Next, the user creates the read item correction display layout creation 85 in FIG. 8 by the external instruction input device 805 in FIG.
A correction instruction is input, the control device 806 gives an instruction to the correction display layout creation device 803, and the correction display layout creation device 803 corrects the image data of the document from the temporary storage device 802 and stores recognition error information as recognition information. Input from the device 809, and the correction display layout creation device 8
03 creates a layout of the correction screen. The created correction screen is displayed in the image / read item integrated display correction 86 in FIG.
By the display method of FIG. 9, the recognition result window 92 is integrally displayed on the document image 91 around the reading position. Further, a recognition result window 95 of the reading item 1 is displayed below the document image portion 94 of the reading item 1, and a recognition result window 97 of the reading item 2 is displayed below the document image portion 96 of the reading item 2 Integrated display a93 Changes according to instructions from the user, etc. In case of the recognition error of item 2,
In order to make the integrated display b94 displaying only the recognition result window 97 of the reading item 2 below the document image portion 96 of the reading item 2 or the document image portion 96 of the reading item 1 easy to see, the document image portion 94 of the reading item 1 is displayed. The recognition result window 95 of the reading item 1 is displayed, and the reading item 2
The recognition result window 97 of the reading item 2 is displayed under the document image portion 96 of the above, and the display is changed to the integrated display c95, and the correction is performed. Image data of the modified document in Figure 8
The control device 806 registers the read item information registration 86 in the storage device 801 of FIG. When there is no recognition error 82 in FIG. 8, the control device 806 registers the image data / read item information registration 86 in FIG. 8 in the storage device 801 in FIG.

[0054]

As described above, according to the present invention,
By coding the sheet identification for finding the read item in automatic recognition to the starting point coordinates and length of the ruled line, or the ruled line, it is possible to correspond to documents in a semi-standard format, and the recognition result of the image and the read item when correcting the read item By displaying the windows integrally, it is possible to make modifications that are easy for the user to use.

[Brief description of drawings]

FIG. 1 is a diagram showing a state in which a document image of the present invention is input and sheet identification is performed to register read items or read items and image data.

FIG. 2 is a diagram showing a state in which sheet identification is performed based on the starting point coordinates and length of a ruled line according to the present invention;

FIG. 3 is a diagram showing a method of matching sheet identification and extracting a read item area of FIG. 2 according to the present invention;

FIG. 4 is a diagram showing how ruled lines are coded from ruled line start point end point data and ruled line related data are obtained by coding ruled lines of the present invention to perform sheet identification;

5 is a diagram showing the encoding method of FIG. 4 according to the present invention;

FIG. 6 is a diagram showing how the ruled line relation data obtained in FIG. 4 of the present invention is matched with an identification dictionary;

FIG. 7 is a diagram showing how Don't Care of the present invention performs matching using data;

FIG. 8 is a diagram showing a state in which a reading item of an input document of the present invention is automatically recognized and correction is performed when a recognition error is present;

9 is a diagram showing a state of a display method at the time of correction of FIG. 8 of the present invention,

FIG. 10 is a diagram showing a configuration of a document system for explaining automatic recognition according to an embodiment of the present invention;

FIG. 11 is a diagram showing a configuration of an automatic recognition system using a starting point length of a ruled line according to an embodiment of the present invention;

FIG. 12 is a diagram showing a flow of automatic recognition using a starting point length of a ruled line according to an embodiment of the present invention;

FIG. 13 is a diagram showing a configuration of an automatic recognition system using ruled line encoding according to an embodiment of the present invention;

FIG. 14 is a diagram showing a flow of automatic recognition using ruled line encoding according to an embodiment of the present invention;

FIG. 15 is a diagram showing a configuration of a system for performing automatic recognition and correcting a document having a recognition error according to an embodiment of the present invention.

[Explanation of symbols]

1: Document, 2: Image input, 3: Ruled line extraction, 4: Sheet identification, 5: Matching, 6: Identification dictionary, 7: Read item area extraction, 8: Character recognition, 9: Read item registration, 10: Image registration , 21: start point coordinate, 22: length, 23: ruled line start point length data, 24: start point coordinate, 25: length, 26: identification format data, 27: matching, 28: identification dictionary, 29: style number / reading. Item region data, 31: identification dictionary registration number error extraction, 32: length error extraction, 33: start point coordinate error 1 extraction, 34: start point coordinate error 2 extraction, 35:
Starting point coordinate error 3 extraction, 36: Starting point coordinate error 4 extraction, 3
7: format number / read item area data extraction, 41: ruled line, 42: ruled line data, 43: hierarchical coding, 44: ruled line, 45: ruled line code data, 46: ruled line related data, 5
1: 1st layer numbering, 52: 2nd layer and lower numbering,
53: arrow, 54: arrow, 55: first level vertical ruled line, 5
6: 1st layer horizontal ruled line, 57: 2nd layer and below vertical ruled line, 5
8: horizontal ruled lines below the second layer, 61: dictionary structure, 62: identification dictionary, 63: dictionary A, 64: dictionary B, 65: error calculation method, 66: read item code data, 67: target ruled line extraction, 68: Extraction of read item position by linear method, 71:
In the case of one faint line, 72: Ruled line related data, 73: In the case of one noise, 74: Ruled line related data, 81: Automatic recognition, 82: Recognition error judgment, 83: Recognition error image data / recognition error information temporary storage, 84: Read item correction display layout creation, 85: Image / read item integrated display correction, 86: Image data / read item information registration, 91: Document image, 92: Recognition result window, 93: Integrated display a, 94: Integrated display b, 95: integrated display c, 96: document image of reading item 1, 97: recognition result window of reading item 1, 98: document image of reading item 2, 99: recognition result window of reading item 2, 79: document, 100:
Image input device, 200: storage device, 300: display device,
400: External instruction input device, 500: Control device, 60
0: Automatic recognition device, 700: Search device, 601: Run data extraction device, 602: Ruled line data extraction device, 603:
Ruled line data normalization device, 604: Ruled line start point length data extraction device, 605: Sheet identification device, 606: Identification dictionary device, 607: Read item area extraction device, 608: Character recognition device, A1: Image input, A2: Run Data extraction, A
3: Ruled line data extraction, A4: Ruled line data normalization, A5:
Ruled line starting point length data extraction, A6: sheet identification, A7: read item area extraction, A8: read item character recognition, A9:
Registration, 611: Run data extraction, 612: Ruled line data extraction device, 613 frame block extraction device, 614: Ruled line data extraction device for each frame block, 615 Ruled line data inclination correction device, 616: Ruled line coding device, 617: Ruled line relation data Extraction device, 618: Sheet identification device, 619: Identification dictionary, 620: Read item area extraction device, 621: Character recognition device, B1: Image input, B2: Run data extraction, B
3: Ruled line data extraction, B4: Framed block extraction, B5: Ruled line data extraction for each frame block, B6: Ruled line data inclination correction, B7: Ruled line coding, B8: Ruled line related data creation, B
9: Sheet identification, B10: Read item area extraction, B1
1: Read item character recognition, B12: Registration, 800: Image input device, 801: Storage device, 802: Temporary storage device,
803: correction display layout creation device, 804: display device, 805: external instruction input device, 806: control device, 8
07: automatic recognition device, 808: determination device, 809: recognition information storage device, 810 search device.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Masafumi Koga 1-280, Higashi Koigakubo, Kokubunji, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd. (72) Inventor Kiyomichi Kurino 2880, Kozu, Kagawa Prefecture System Division (72) Inventor Takeyuki Sugimoto 2880 Kokuzu, Odawara City, Kanagawa Hitachi Storage Systems Division

Claims (18)

[Claims] 1. An image input device for inputting a document image and a format of the document image input by the image input device are identified,
In a document system having a recognition device for extracting a predetermined read item region and recognizing item information as a character code from the image of the region, the recognition device is a starting point of a ruled line for each style of a document of an input document image. An identification dictionary that stores data composed of coordinates, the length of ruled lines and the coordinates of the read item area; ruled line information extraction means for obtaining the ruled line starting point coordinates and ruled line lengths of the document image input to the recognition device; An identification means for determining the document format stored in the recognition dictionary by matching the data of the identification dictionary with the obtained ruled line information, and a reading item area corresponding to the document format obtained by the identification means are provided. It has a reading item area extracting means for determining and a character recognizing means for recognizing a character described inside the image of the reading item area. Document system. 2. An image input device for inputting a document image and a format of the document image input by the image input device are identified,
In a document system having a recognition device for extracting a predetermined read item region and recognizing item information as a character code from the image of the region, the recognition device is a starting point of a ruled line for each style of a document of an input document image. And an end point, and an identification dictionary that stores code data indicating the positional relationship between ruled lines, and a ruled line composed of vertical ruled lines and horizontal ruled lines of the document image input to the recognition device to obtain ruled line code data, A document system characterized in that a reading item area is determined from an intersection of the ruled lines by collating with the identification dictionary, and a character is recognized from an image of the area. 3. The document system according to claim 2, wherein the recognition device assigns ruled line code data to vertical ruled lines in a horizontal coordinate order and horizontal ruled lines in a vertical coordinate order. 4. The document system according to claim 2, wherein the recognition device generates ruled line code data, vertical ruled lines in a horizontal coordinate order and a contact relationship with a horizontal ruled line, and horizontal ruled lines in a vertical coordinate order. A document system characterized by being assigned according to the contact relationship with vertical ruled lines. 5. The document system according to claim 2, wherein the recognition device extracts ruled line code data, vertical ruled lines do not overlap with each other in vertical coordinate, and the ruled lines are arranged in vertical coordinate order, and horizontal ruled lines are A document system characterized by extracting ruled lines that do not overlap with each other in the lateral direction coordinates and assigning the ruled lines in the order of the lateral direction coordinates. 6. The document system according to claim 2, wherein the recognition device extracts ruled lines from the ruled line code data that do not overlap the vertical ruled lines with respect to the vertical coordinate and assigns the codes already with reference to the horizontal ruled lines. If the vertical coordinate of the ruled line exists in the same range, a code including the hierarchical relationship is assigned,
For horizontal ruled lines, extract ruled lines that do not overlap with respect to horizontal direction coordinates, and assign a code that includes a hierarchical relationship if the horizontal coordinate of a ruled line that has already been assigned a code by referring to a vertical ruled line is in the same range. Document system characterized by. 7. The document system according to claim 2, wherein the recognition device includes ruled line code data in which the vertical ruled lines are arranged in the horizontal direction, the horizontal ruled lines are arranged in the vertical direction, and the positions and lengths of the ruled lines are included. Document system characterized by assigning. 8. The document system according to claim 1, wherein the ruled line information of the input document image and the data of the identification dictionary are collated by using the discrimination dictionary of the style identified immediately before the collation. Document system characterized by. 9. The document system according to claim 1, wherein as a result of the collation of the input document image and the identification dictionary, the error of the form number of the document number stored in the dictionary is the smallest. A document system that obtains the form number of a document number. 10. The document system according to claim 1, wherein as a result of collating the input document image with the identification dictionary, a frame structure is identified for each independent frame having a different outer contour, and a structure number is extracted. Then, the document system characterized by judging the style by the set of the structure numbers. 11. The document system according to claim 2, wherein the input document image is collated with an identification dictionary by converting ruled line code data into relational data of start and end points of vertical and horizontal ruled lines. And document system. 12. The document system according to claim 1 or 2, wherein the coordinates of the start point and the end point of the black pixel connection for each scanning line are obtained from the input document image data, and the data of the coordinates is used as run data for the document image. A document system having a run data extracting device for extracting ruled lines of the document. 13. The document system according to claim 1, wherein the coordinates of the starting point and the ending point of the black pixel connection for each scanning line are obtained from the input image data, and the run data is extracted from the coordinates to obtain run data. A document system comprising: a run data extracting device that removes short runs that are equal to or less than a predetermined threshold, and then connects the remaining runs to extract them as ruled lines. 14. An image input device for inputting a document image, and a recognition device for extracting the position of an area where a read item of the input document exists and recognizing the character to extract read item information.
A determination device that determines a recognition error such as a character recognition error in the recognition device, a recognition information storage device that stores the position and font size of the recognized read item information, and an input document that is determined to be a recognition error by the determination device. A primary storage device for temporarily storing the information, a display device for correcting read item information of a document determined to be a recognition error, and a display based on the information stored in the recognition information device for the correction display. A modified display layout creation device for creating a layout;
An input document image and its recognition information, which have an image determined to have no recognition error and read item information, and a corrected document image and a storage device that stores the read item information, and which have been determined by the determination device to be a recognition error. A document system characterized in that a correction display layout is created based on the recognition information when the characters of a read item are corrected, and a document image and a window for each recognition result are integrally displayed. 15. The document system according to claim 14, wherein when the document image and the recognition result separate window are integrally displayed on the display device for modification, the display position and the display size of the recognition result separate window are changed. A document system having an input device for inputting a document. 16. The document system according to claim 14, wherein, when the correction display layout creating apparatus has a plurality of recognition items, only a part recognized as a recognition error by the judgment apparatus and a recognition result separate window and a document image are displayed. A document system comprising: 17. The document system according to claim 14, wherein, when the correction display layout creating apparatus has a plurality of recognition items, a recognition result separate window of a portion judged to be a recognition error by the judging apparatus and a recognition error. A document system characterized in that a recognition result separate window of a part determined to be absent is displayed integrally with a document image by changing the background color. 18. The document system according to claim 14, wherein when there are a plurality of recognition items for the corrected display layout creation device, the recognition result separate window of the part judged as the recognition error by the judgment device is highlighted and the document is displayed. A document system characterized by being displayed integrally with an image.