JP3179280B2 - Form processing device including tables - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a form processing apparatus.
In particular, the present invention relates to a form processing apparatus for reading characters and symbols written in a table on a form.

[0002]

2. Description of the Related Art In order to read characters, symbols, and the like written in a table on a form, a table collation process for determining a table format is first performed. Next, the position on the image of a character reading area (hereinafter, referred to as a field area) in the table defined by the table format is calculated. Finally, the characters in the field area are cut out, and the cut out characters are recognized.

A form processing apparatus responding to such a request has been disclosed in Japanese Patent Laid-Open No. 2-217977. FIG. 15 shows a block configuration for performing a table matching process in the disclosed form processing apparatus. In the figure, reference numeral 201 denotes a photoelectric conversion unit, 202 denotes an image buffer, 203 denotes a line segment detection unit, 204 denotes a frame structure extraction unit, 205 denotes a frame structure dictionary storage unit, 206 denotes a frame structure matching unit,
Reference numeral 07 denotes a field image extraction unit, 208 denotes a character detection cutout unit, and 209 denotes a character recognition unit.

In the form processing device, the table collation processing is performed in the following procedure.

First, a form 210 including a table shown in FIG. 16 is read as a form image based on optical scanning in the photoelectric conversion unit 201. The read form image is stored in the image buffer 202. Next, the line segment detection unit 203
In FIG.
7, horizontal ruled lines H1 to H6 are detected. next,
The frame structure is extracted based on the horizontal ruled lines H1 to H6 detected by the frame structure extraction unit 204. Here, the frame structure is represented by the number of horizontal ruled lines, the length, thickness, and positional relationship of each ruled line.

After the frame structure has been extracted, the frame structure information stored in the frame structure dictionary storage unit 205 in advance is read out as a table format. A table comparison between the table format read by the frame structure matching unit 206 and the frame information extracted by the frame structure extraction unit 204 is performed. As a result of the table comparison, if it is determined that the table is different, another table format is read from the frame structure dictionary storage unit 205, and the table comparison is performed again. Also, as a result of the table comparison, when it is determined that the tables are the same table, read field position information indicating a field area position on the form is read from the frame structure dictionary storage unit 205. The read read field position information is output to the field image extraction unit 207.

[0007] Then, the field image extraction unit 207
In step (1), an image of the field area position indicated by the read field position information is extracted from the image. Next, a character image existing in the field area is extracted by the character detection extraction unit 208. Finally, the character recognition unit 209
In step (1), the character image cut out by the character detection cutout unit 208 is recognized.

[0008]

The following points are not taken into consideration in the table collation processing performed by the above-described form processing apparatus.

First, in the table format, ruled lines indicating the attributes of the ruled lines, such as the number of ruled lines detected in the area designated as the line segment detection region 211, and the length, thickness, and positional relationship of each ruled line. Attribute information is used. Therefore, when the ruled line attribute information is similar, it is not possible to distinguish tables that are structurally different. For example, the two table images 220 and 221 shown in FIGS. 18A and 18D are structurally different. However, in the ruled line attribute information, FIG.
Since the horizontal ruled lines H1 and H2 shown in FIG. 18 are very similar and the vertical ruled lines V1 and V2 shown in FIGS. 18C and 18F are the same, the horizontal ruled lines H1 and H2 in the ruled line attribute information are displayed. Does not represent the difference between the tabular images 220 and 221.

Second, if there is distortion in the table, if the table is tilted, or if the table is enlarged or reduced, the position information and length information of the ruled line, which is the attribute information of the ruled line, will be out of order. Table matching cannot be performed correctly.

Third, when handwritten characters and the like are entered in a table, a large number of extra ruled lines are detected from the character portion. Further, the ruled line detected from the character portion comes into contact with the table ruled line, and the apparent table ruled line length increases. As a result, the information on the number and length of the ruled lines is disturbed, so that the table matching process becomes difficult.

Fourth, when the table ruled lines are cut or blurred, the length and the number of ruled lines are different from those of an actual table, and the table collation processing cannot be performed correctly.

Fifth, when extracting a field area, an image of the field area position indicated by the read field position information is directly extracted from the image. The field area position is generally indicated by a coordinate value or the like based on the table area, and does not take into account enlargement / reduction of the table, table inclination, and table distortion. Therefore, in a distorted table, a tilted table, or an enlarged or reduced table, a deviation occurs between a field area position stored in advance and a position on an actual image, and an accurate field area position is obtained. Absent.

The present invention has been made to solve such a problem, and includes a table having similar ruled line attribute information, a table including a disturbance such as handwritten characters, a distorted table, a tilted table, and an enlargement / reduction. It is an object of the present invention to provide a form processing apparatus capable of obtaining a high-accuracy table comparison even for a set table and a table having cuts or fading, and obtaining an accurate field area position.

[0015]

In order to achieve the above object, a form processing apparatus including a table according to the present invention comprises a ruled line extracting means, a table structure analyzing means, a table format database, a table collating means, a field position searching means. , And a result output means.

The ruled line extracting means includes, for example, an image inputting means, a table area extracting means, and a ruled line attribute information calculating means, and extracts ruled lines included in a table area on a form.

The table structure analyzing means may include the ruled line extracting means.
From the ruled lines extracted in
An outside table tangent line detecting means for detecting a certain outside table tangent line,
A method for registering the external circumscribed ruled line as the parent ruled line in the first construction stage
And the parent ruled lines facing each other at the current structural stage.
A means for detecting inscribed ruled lines where both ends touch,
Each of the components based on the parent ruled line and the inscribed ruled line.
A method for registering ruled line structure information for each manufacturing stage as the table structure
A step and the inscribed rule detected in the current construction step
Means for setting the parent rule line in the next structure stage
Things.

The field position searching means searches for a field position on an image.

The result output means outputs a table collation result and a field position search result.

[0020]

In the form processing apparatus including a table according to the present invention, the form including the table is read as an image by the ruled line extracting means, and the ruled line included in the table area on the form is extracted. In the table structure analyzing means, a table format is created from the ruled lines extracted by the ruled line extracting means. The table format database stores the table format created by the table structure analyzing means. In the table collating means, the table structure described in the table format stored in the table format database is collated with the ruled line extracted from the table area on the form by the ruled line extracting means. The field position searching means calculates the field region position on the form image based on the table collation result and the read field position information described in the table format. The result output means outputs the certainty factor and the field area position.

A form including a table is read by the image input means of the ruled line extracting means. The table area on the form image read by the image input means is extracted by the table area extracting means. The ruled line attribute information calculating means extracts ruled lines included in the table area extracted by the table area extracting means, and calculates ruled line attribute information.

In the table structure analysis means, the ruled line
The outline of the rectangular table is constructed from the ruled lines extracted by the extraction means.
Circumscribed ruled line, which is a ruled line, is detected.
Register the circumscribed ruled line as the parent ruled line at the first structural stage. This
At both ends of the parent lines facing each other in the first construction stage
Is detected, and this inscribed ruled line is
Set as the parent rule of the floor. In this way, the current structure
Inscribed with both ends touching the opposing parent rule line in the stage
The rule is detected and set as the parent rule for the next structural stage.
You. Based on parent and inscribed rules at each structural stage
Ruled line structure information for each structure stage is registered as a table structure.
You.

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the overall configuration of a form processing apparatus including a table according to the present invention. As shown in FIG. 1, a form processing apparatus including a table includes a ruled line extracting means 1, a table structure analyzing means 2, a table format database 3, a table collating means 4, a field position searching means 5, and a result outputting means 6.

In the ruled line extracting means 1, a form including a table is read as an image, and a ruled line included in a table area in the form image is extracted. The ruled line extracting means 1 includes an image input means 1a, a table area extracting means 1b, and a ruled line attribute information calculating means 1c.

In the table structure analyzing means 2, a table format is created from the ruled lines extracted by the ruled line extracting means 1. This table structure analyzing means 2 is an external table tangential ruled line detecting means 2
a, ruled line structure information calculation means 2b and table format creation means 2c.

The table format database 3 stores the table format created by the table structure analyzing means 2.

The table collating means 4 collates the ruled lines extracted from the table area by the ruled line extracting means 1 with the table structure described in the table format stored in the table format database 3. The table collating means 4 includes a ruled line associating means 4a and a certainty factor calculating means 4b.

The field position search means 5 obtains the field area position on the image based on the read field position information described in the table format and the ruled line matching result obtained by the table collation means 4. The result output unit 6 outputs the certainty factor calculated by the table matching unit 4 and the field area position calculated by the field position search unit 5.

In a form processing apparatus including a table configured as described above, two processes of a table format creation process and a table reading process are performed. Here, the table format creation process is a process of creating a table format from a form including an empty table. The table reading process is a process of reading a form including a table in which characters are written, and searching for a field area when it is determined to be the same table by comparing it with a table format. Hereinafter, the table format creation processing and the table reading processing of the form processing device including the above-described table will be described separately.

Table Format Creation Processing First, the procedure of the table format creation processing will be described. This table format creation processing is performed by the ruled line extracting means 1, the table structure analyzing means 2, and the table format database 3 shown in FIG. The outline of the procedure of the table format creation processing is as follows. First, the ruled line extracting means 1 extracts ruled lines included in a table area of a form including an empty table in which nothing is entered. Based on the ruled lines extracted by the ruled line extracting means 1, a table format for an empty table is created by the table structure analyzing means 2. Finally, the table format created by the table structure analysis means 2 is stored in the table format database 3.

Next, a detailed procedure of the table format creation processing will be described with reference to FIGS. FIG. 2 shows a form image including a table read as an image. FIG. 3 shows vertical and horizontal ruled lines extracted from a table area on a form image. FIG. 4 shows an example of an intersection feature between ruled lines. FIG. 5 shows the circumscribed ruled line of the table. FIG. 6 shows a flowchart of the processing performed by the ruled line structure information calculating means 2b. FIG. 7 shows an example of the structure analysis performed by the ruled line structure information calculating means 2b. FIG. 8 shows an example of a table format created by the table format creating means 2c.

First, the image input means 1 of the ruled line extracting means 1
At a, a form 40 (see FIG. 2) including an empty table is read. An optical image reading device such as a facsimile (FAX) or an image reader is used for the image input means 1a.
Next, the table area 41 on the form image 40 is extracted by the table area extracting means 1b. The table area 41 is, for example, “Lu, Li,
"A Table Processing Algorithm for Printed
Document Analysis ", D-610, IEICE Spring Institute, 1990". next,
The ruled line attribute information calculating means 1c extracts ruled lines in the vertical and horizontal directions (vertical and horizontal directions) in the table area 41. At the same time, rule line attribute information such as the length, start point and end point, line type, connection relationship between rule lines, intersection position, and intersection feature of each extracted rule line is calculated. These ruled lines and ruled line attribute information are, for example, “Hayashi,
Shimotsuji, et al., "Effective Automatic Input Method of Drawings by Detecting Feature Pattern Information", PRL 83-8, pp. 57-64, 1983. By these methods, a ruled line group having horizontal ruled lines h0 to h6 and vertical ruled lines v0 to v11 shown in FIG. The intersection features described above are horizontal ruled line h and vertical ruled line v.
It represents the contact state of the ruled line, and there are nine forms shown in FIGS. The intersection between the horizontal ruled line h and the vertical ruled line v is indicated by the symbol c. After calculating the ruled line attribute information, the process in the ruled line extracting means 1 ends.

Next, the processing in the table structure analyzing means 2 is executed. In the table structure analysis means 2, the processing is performed by the outside table tangential ruled line detection means 2a, ruled line structure information calculation means 2b and table format creation means 2c shown in FIG.

Outer table tangent ruled line detector 2 of table structure analyzer 2
In the case of a, the external tangential ruled line is detected from the ruled line extracted by the ruled line extracting means 1. Here, the outer table tangent rule line is defined as four rule lines forming an outer frame of each rectangular table when the table is decomposed into a plurality of rectangular tables. The outer circumscribed ruled line is obtained by the following procedure.

First, in FIG. 3, two horizontal ruled lines h0 and h6 located at the uppermost and lowermost positions and two leftmost and rightmost positions are located among the ruled lines having a length equal to or longer than a predetermined length. The vertical ruled lines v0 and v11 are selected as circumscribed ruled line candidates. Next, a ruled line, a horizontal ruled line h1, h3, h4, h5, h6, and a vertical ruled line v0 whose both ends are in contact with two parallel ruled lines of the circumscribed ruled line candidate are detected from the ruled line group. Ru seek horizontal ruled lines and vertical ruled lines located leftmost position and the rightmost position located uppermost and lowermost among the detected border. By this processing, the horizontal ruled line h1
And h6 and the vertical ruled line v0 are obtained. These horizontal ruled lines h
1, h6 and vertical ruled line v0, and the outside selected before
New union with tangent rule line candidates h0, h6, v0, v11
Circumscribed ruled line candidate.

Next, the obtained circumscribed ruled line candidates, that is, the vertical ruled lines v0 and v11 and the horizontal ruled lines h0, h1 and h6 are combined to set a virtual rectangle. The virtual rectangle is a rectangle specified by a total of four virtual circumscribed rule lines including two vertical rule lines and two horizontal rule lines. In the circumscribed ruled line candidates described above, three virtual rectangles (v0, v11, h0, h1), (v0, v
11, h1, h6) and (v0, v11, h0, h6)
Can be set.

Next, it is checked whether or not each virtual rectangle is a correct rectangle table. Here, it is defined as a correct rectangle table when four circumscribed ruled line candidates constituting the virtual rectangle are connected by end points to constitute one rectangle. Then, in the virtual rectangle determined to be the correct rectangular table, the circumscribed rule candidate is registered as the circumscribed rule, and the virtual rectangle is registered as the rectangular table.
In the present embodiment, the rectangular table composed of the circumscribed ruled line candidates v0, v11, h1, and h6 is determined to be a correct rectangular table and registered.

Thereafter, a search is made for a circumscribed ruled line candidate in each virtual rectangular area that is not determined to be a correct rectangular table, and the above processing is executed again. This process is repeated until the virtual rectangular area cannot be searched. When the process is repeated, the table having a complicated outer shape is divided into a plurality of rectangular tables, and circumscribed ruled lines are obtained in each of the divided rectangular tables. As a result, as shown in FIG. 5, the circumscribed ruled lines v0, v
4, a rectangular table A surrounded by h0 and h1 and a circumscribed ruled line v0,
A rectangular table B surrounded by v11, h1 and h6 is obtained.

Next, the table structure is analyzed by the ruled-line structure information calculating means 2b based on the ruled-line tangential lines determined by the ruled-line tangential line calculating means 2a and the ruled lines extracted by the ruled line extracting means 1, and the ruled line structure information is calculated. Is done. The flow of the processing performed by the ruled line structure information calculating means 2 will be specifically described using a flowchart shown in FIG. 6 and a structural analysis example shown in FIG.

First, a stage i = 0 is set in the initialization means 7 shown in FIG. Next, the circumscribed ruled line registration means 8 registers the circumscribed ruled line obtained by the table external circumscribed line detection means 2a as a detected ruled line. 7A to 7D, solid lines indicate ruled lines detected before the current stage, and dotted lines indicate undetected ruled lines. Here, for example, six ruled lines v0, v4, v11, h0, h1, and h6 shown in FIG. 7A are registered.

Next, the divided ruled line detecting means 9 detects a ruled line whose both ends touch the registered ruled line, and detects horizontal ruled lines h3, h4 and h5 and vertical ruled lines v1, v2 and v7 shown in FIG. 7B. You. Then, the determination means 10 determines the end of the process. Here, if no ruled line is detected, the process ends. Conversely, if a ruled line is detected, step i is executed.
Is incremented, and the ruled line information reading means 11 is executed. The ruled line information reading means 11 obtains ruled line structure information of the detected ruled line. The ruled line structure information includes the stage i at which the ruled line is detected and the two in the orthogonal direction where both ends of the ruled line are in contact.
A ruled line number (hereinafter, the two ruled lines are referred to as parent ruled lines).

Then, the ruled line detected at this stage is additionally registered in the ruled line registration means 12, and the process is repeated.
By repeating this processing, ruled line structure information having all ruled lines forming the table is obtained stepwise through the steps shown in FIG. 7C. Finally, as shown in FIG.
All ruled lines are detected before max = 3.

When the processing in the ruled line structure information calculating means 2b is completed, the processing in the table format creating means 2c is executed. The table format creating means 2c creates a table format based on the ruled line attribute information calculated by the ruled line attribute information calculating means 1c and the ruled line structure information calculated by the ruled line structure information calculating means 2b. FIG. 8 shows an example of a table format created from the ruled line group shown in FIG. This table format includes a table structure information section and a read field position information section as shown in FIG. The table structure information section stores table structure information used for table comparison.
In the table structure information, the ruled line information such as the ruled line number, the start and end points, the length, the line type, the parent ruled line number, and the stage of each ruled line constituting the table is summarized in all the ruled lines detected by the ruled line structure information calculating means 2b. Things. The read field position information section stores read field position information used by the field position search means. The read field position information is a ruled line number constituting the field. For example, when searching for the position of the field 20 shown in FIG.
The ruled line numbers h1, h3, v0, v2 constituting 0 are used as read field position information. Where in the table a field is designated is artificially specified based on the table structure information.

When the processing in the table format creation means 2c is completed, the created table format is stored in the table format database 3, and the table format creation processing ends.

[0051] Table reading Processing Next, a description is given of the procedure of the table reading processing. The table reading process includes the ruled line extracting means 1, table format database 3, table collating means 4, field position searching means 5 shown in FIG.
And the result output means 6. The outline of the procedure of the table reading process is as follows. First, a form including a table in which characters are written is read as an image by the ruled line extracting means 1, and ruled lines included in a table area are extracted. Next, the table collation means 4 collates the table structure information described in the table format stored in the table format database 3 with the ruled lines extracted by the ruled line extracting means 1 to calculate the certainty factor. If the table matching means 4 determines that the table is the same as the table format, the field position search means 5 calculates the field area position on the image based on the table matching result and the read field position information described in the table format. You. Finally, the result output unit 6 outputs the certainty factor calculated by the table matching unit 4 and the field area position on the image obtained by the field position search unit 5.

Next, the details of the procedure of the table reading process will be described with reference to FIGS. 1, 3 and 9 to 14. FIG. 9 shows vertical and horizontal ruled lines extracted from a table area in which characters are written. FIG. 10 is a flowchart showing the flow of processing performed by the ruled line association means 4a. FIG. 11 shows a calculation example of the division ruled line ratio. FIG. 12 shows an example in which the division rule rates are the same. FIG. 13 shows an example of interpolation of a broken ruled line. FIG. 14 shows an example of field region extraction.

First, the ruled line extracting means 1 reads a form including a table in which characters are entered as an image, extracts a table region, and then extracts vertical and horizontal ruled lines in the table region.
The processing by the ruled line extracting means 1 is the same as that of the table format creation processing described above. As a result, the vertical and horizontal ruled lines shown in FIG. 9 are extracted in the table area.

When the processing in the ruled line extracting means 1 is completed, the processing in the table collating means 4 is executed. Table collation means 4
Is composed of ruled line association means 4a and certainty degree calculation means 4b shown in FIG. Hereinafter, assuming a case where the ruled line group shown in FIG. 9 extracted by the ruled line extracting means 1 and a table format created from the table shown in FIG.

In the ruled line association means 4a, the ruled lines (hereinafter referred to as table image ruled lines) extracted from the image by the ruled line extraction means 1 and the ruled lines described in the table format (hereinafter referred to as table format ruled lines) are associated with each other. Can be
This association is performed step by step, and in a single association, a table format ruled line group having the same ruled line structure information is associated with a ruled line candidate group selected from table image ruled lines.

The processing performed by the ruled line association means 4a will be described in detail with reference to the flowchart shown in FIG. First, the corresponding stage m = 0 in the initialization means 13.
Is set. Next, the table format reading means 15 creates a ruled line set composed of table format ruled lines having the same ruled line structure information. The ruled line set is the corresponding stage m
The table format ruled line having the same stage i is read from the table format, and the read ruled line is created by being classified according to the parent ruled line. For example, when the corresponding stage m = 1, the table format ruled line v having the stage i = 1 shown in FIG.
The information of 1, v2, v7, h3, h4, h5 is read,
Parent rule lines (v0, v11), (h0, h1) and (h1,
h6) can be created.

Next, ruled line candidate calculating means 16 selects ruled line candidates to be associated with table format ruled lines belonging to each ruled line set from the table image ruled line group. It is clear from the ruled line structure information calculation flow shown in FIG. 6 that the parent ruled lines constituting each ruled line set are already associated with the table image ruled lines at the corresponding stage before m-1. Therefore, for example, the parent ruled line (h1,
The ruled line candidate for associating with the ruled line group having h6) is a ruled line candidate search area 48 including table image ruled lines 46 and 47 corresponding to the parent ruled lines in FIG. 9 corresponding to the parental ruled lines (h1 and h6).
Table image ruled lines that are partially inside are obtained as ruled line candidates. The squeezed ruled line candidates iv0 to iv1
3 and the number CN of ruled line candidates are obtained.

When the calculation process of the ruled line candidates is completed, the process by the divided ruled line calculating means 17 is executed. The divided ruled line calculating means 17 calculates a divided ruled line rate for each ruled line candidate obtained by the ruled line candidate calculating means 16. The division ruled line ratio indicates the probability that an arbitrary ruled line candidate is a ruled line having both ends in contact with the table image ruled line corresponding to the parent ruled line. Here, ruled line candidate j
The divided ruled line probability Cpj of (j = 1 to CN) is set to a value normalized to, for example, 0 to 1. A divided ruled line having a high probability is a value close to 1 and a low divided ruled line has a value close to 0. . In this case, ruled line candidates iv0 to iv0 for the parent ruled lines (h1, h6)
The divided ruled line probability of iv13 takes a value shown in FIG. The divided ruled line rate is calculated based on the ruled line length and the intersection feature. When both ends of the ruled line candidate contact the parent ruled line as shown in Expression (1), Cp = 1.

Conversely, when both ends of a ruled line candidate are short without contacting the parent ruled line, or when both ends of the ruled line contact the parent ruled line but extend beyond the parent ruled line, Cp is calculated as shown in equation (2). It becomes a value close to 0. Since both ends of the table format ruled line always contact the parent ruled line, the closer the value of Cp is to 1, the higher the similarity between the table format ruled line and the ruled line candidate. Here, the length of the ruled line used in calculating Cp is not an absolute length based on the form image size but a relative length based on the interval between parallel parent ruled lines. . The interval between the parent ruled lines is a value determined by association. Therefore, even if the table is distorted, the table is enlarged, or the table is reduced as the ruled line interval is changed, the ruled line which is structurally equal to the table format has Cp = 1.

If both ends of the ruled line candidate touch the parent ruled line and do not protrude: Cp = 1 (1) Other cases: 0 ≦ Cp <1 (2) After the divided ruled line is calculated, the ruled line The processing in the matching means 18 is executed. The ruled line matching means 18 compares the table format ruled line group with the ruled line candidate group, and associates the table format ruled line with the ruled line candidate. A general optimization method such as a DP matching and a Boltzmann machine can be used for the matching, but this embodiment shows an example using the DP matching. In the case of using DP matching, a table format ruled line group constituting a certain ruled line set is associated with a ruled line candidate, and a correspondence that minimizes the energy of a path is determined as a matching result. Here, the energy Em of the minimum route is obtained by Expression (3) when the table format ruled line group F and the ruled line candidate group I are set. In equation (3), FN is the number of ruled lines in the table format, and CN is the number of ruled line candidates. Further, F includes a table format ruled line which has passed through the ruled line candidate search area designated by the parent ruled line and which has been corresponded before the corresponding stage m-1 in addition to the table format ruled line group constituting the ruled line set. For example, in the case of the corresponding rule m = 1 and the parent rule line (h1, h6), FIG.
The table format ruled line groups forming the ruled line set are v2 and v7 as shown in FIG.
Table format ruled lines corresponding to -1 or earlier are v0, v1
It is one. Therefore, F = (v0, v2, v7, v11)
become.

Em = h (FN, CN) (3) h (n, j) = min [h (n, j−1) + Pe1, h (ni, j) + Pe2, h (n) −1, j−1) + d (n, j)] (4) If j is a ruled line already associated with n in the corresponding stage before m−1: d (n, j) = 0 ( 5) ^{otherwise: d 2 (n, j)} = α1 · (Cp n -Cp j) + α2 · f (P n -P j) + α3 · δ (Lt n -Lt j) ... (6) where, h (n, j) indicates the energy value when the table format ruled line n and the table image ruled line j are associated with each other. Pe
1, Pe 2 is a penalty value (constant). Pe2 takes a large value to avoid many-to-one correspondence between table format ruled lines and table image ruled lines. d (n, j) is the degree of difference between ruled line n and ruled line j.
(N, j) is set to a value close to 0, and to a large value if they are not similar. Cp is the division rule rate, and P is the rule position normalized by the table size. Lt is a line type. For example, different values are set depending on the type of isoline, Lt = 0 for a solid line, and Lt = 1 for a dotted line. α1, α2, α3 are constants. Since both ends of the table image rule are always in contact with its parent rule, Cpn (n
= 0 to FN) is set to 1. Therefore, in the equation (6), the first term on the right side indicates the difference as a divided ruled line,
The term indicates the amount of displacement, and the third term indicates the difference in line type.
Δ (x) is Kronecker's delta, and when x = 0, δ
(X) = 0, otherwise δ (x) = 1.

In the equation (6), the second term on the right-hand side is added to the evaluation, so that the matching accuracy of the enlarged, reduced, or distorted table is reduced due to the influence of positional displacement.
However, the second term is necessary for distinguishing between ruled lines generated by disturbance such as handwritten characters and table ruled lines when they have the same divided ruled line rate. For example, as shown in FIG. 12, when a numeral "1" at both ends of which is in contact with the parent rule line is entered in the table, both the table rule line and the number become the divided rule ratio Cp = 1, and without the second term, the table rule line and the numeral Can not be distinguished. Therefore, at the ruled line position P of the second term, values normalized by the width and height of the table are used. Accordingly, the ruled line position P is not affected by the enlargement or reduction of the table, and the enlarged table and the reduced table can be correctly collated. When calculating the degree of difference d (n, j), if j is a ruled line associated with n at the corresponding stage before m−1, d (n, j) = 0 is set. This means that the ruled lines associated before the corresponding stage m-1 are forcibly associated with each other at the corresponding stage m, and has a function of correcting the matching at the current corresponding stage using the past matching result as an index. Therefore, the displacement of the ruled line is absorbed, and the distorted table can be compared with high accuracy. Here, the function f (x) shows a small value when the displacement is small, and can be represented by, for example, Expression (7).

If | x | <threshold value: f (x) = | x | (7) In other cases: f (x) = 1 (8) However, in general, tables and low-resolution images drawn by hand are generally used. The table read by the input device tends to be cut or blurred. In this case, recognition in which one table image ruled line is originally divided into a plurality of lines is performed, and it becomes difficult for the above described ruled line associating means to associate. When a ruled line is cut, an image processing method is generally used, and simple interpolation can be considered. However, there is a possibility that erroneous interpolation is performed, which causes a reduction in matching accuracy.

Therefore, an interpolating process which does not depend on an image processing method is added as a pre-process of the ruled line associating means. This added preprocessing is performed in the following procedure. First, it is assumed that the ruled line candidate calculating means 16 has obtained 11 ruled line candidates shown in FIG. FIG. 13 (A)
, Reference numerals 50 and 51 indicate table image rule lines corresponding to the parent rule lines. ch1 to ch11 indicate ruled line candidates. Here, the virtual ruled line p1 is set using ruled line candidates considered to form the same ruled line among the ruled line candidates ch1 to ch11 of the ruled line candidate group ch. Specifically, the virtual ruled line p11 = (ch
3, ch4) and p12 = (ch7, ch8, ch9,
ch10) is set.

Next, the degree of difference d2 (p1i) is calculated for each virtual ruled line p1i by equation (9). In the equation (9), the first term on the right side is equivalent to the first term on the right side of the equation (6). However, the second term PL (p1i) is newly added, and this PL (p1i) is obtained by dividing the divided ruled line from the virtual ruled line p.
1i represents the loss to creating. This takes a large value when the gap between the ruled lines constituting the virtual ruled line is large or when the intersection feature between the ruled lines impairs continuity. For example, FIG.
As shown in FIG. 3B, the virtual ruled line 53 with respect to the parent ruled lines 50a and 51a is regarded as having lower continuity than 52, and PL takes a large value.

Next, a combination of ruled line candidates that minimizes the difference d2 (p1i) is calculated by an optimization method. For example, ch4 forming p11 and ch10 forming p12 as a result of the optimization are removed, and the combination of the ruled lines forming the virtual ruled line is p11 = (ch3) and p12 = (ch7,
ch8, ch9). The virtual ruled line created last is used, and a new ruled line candidate group I = (ch1, ch
2, p11, ch5, ch6, p12, ch11). However, when the association is performed, the loss PL is added as the fourth term on the right side to equation (6) when calculating the similarity between the virtual ruled line and the table format ruled line. By the above processing, even when the ruled lines are cut or blurred, the optimal interpolation result based on the table format is created, and the ruled lines are accurately associated.

[0067] _{^{d2 2 (p1) = α1 ·}} (Cp n -Cp pl) + PL (p1) ... (9) correspondence between tabular format borders and table image borders in the corresponding stage m the border checking means 18 of the aforementioned When the processing is completed, the corresponding step increment means 19 adds 1 to the corresponding step m, and repeats the above processings 14 to 19 (see FIG. 10) again. In end determining unit 14 corresponding stage m is greater than the maximum step imax described in Table format, borders correlating means 4 a is terminated when the ruled line candidate number CN = 0 in ruled line candidate calculation unit 16.

[0068] When the ruled line correspondence unit 4 a is completed,
Table confidence Cf is calculated in confidence factor calculating means 4 b. The certainty factor Cf is an index indicating the probability that the table on the form is the same as the table format, and is expressed by equation (10).

Cf = Cm + Cu (10) Here, g1 is the number of table image ruled lines associated with the table format ruled line. L1 is a table image ruled line associated with a table format ruled line. Z (L1 _n) is a table format borders numbers corresponding with Table image borders L1 _n. d (x, y) is the same function as equation (6). Thus, Cm indicates the difference between the associated table image ruled line and the table format ruled line. Conversely, g2 represents the number of table image ruled lines that are not associated with table format ruled lines. L2 is a table image ruled line that is not associated with a table format ruled line. Cp 'L2 is the maximum dividing ruled line rate of the table image ruled line L2 calculated in all the corresponding stages m = 0 to imax. Therefore, when both ends of the ruled line that does not correspond to the table format ruled line contact the other two ruled lines (when the table structure is distorted), Cp 'L2 takes a large value. Cu indicates a shift from the table format ruled line to the table format due to the uncorrelated table image ruled line. Each of Cm and Cu takes a small value when the table format and the table on the image are similar. Therefore, when the confidence Cf is close to 0, it is determined that the input table is the same as the table described in the table format. Here, the ruled line information for calculating the degree of difference includes ruled line structure information indicating a table structure in addition to ruled line attribute information indicating the attribute of the ruled line. According to the ruled line structure information, the degree of difference between these ruled lines reflects the structural difference of the table.

When the matching between the ruled lines is performed using such ruled line information, and the degree of certainty is calculated using the similarity between the ruled lines, when a disturbance such as a handwritten character is present or the table is distorted. The table can be compared with high accuracy even when the information is enlarged, reduced, inclined, or even when the ruled line attribute information is similar.

When the certainty factor is calculated, the table matching process 4
Ends. Here, when the certainty factor Cf is larger than a certain value, the table reading process ends because it does not match the table format. Conversely, when the certainty factor Cf is small, the processing in the next stage field position searching means 5 is executed.

The field position search means 5 searches the field area position on the image based on the read field position information described in the table format and the result of the table collation means 4. As shown in FIG. 8, the read field position information describes a plurality of table format ruled line numbers constituting a field to be searched. Here, a case where the position on the image of the field 20 shown in FIG. 8 is searched will be considered. In this case, the table format ruled lines h1, h3, v0, and v2 constituting the field 20 are read from the table format. Next, table image rule lines corresponding to these table format rule lines are obtained. This is apparent from the result of the association between the table format ruled line and the table image ruled line by the table matching means 4. For example, as shown in FIG. 14, table image ruled lines 60, 61, 62 and 63 corresponding to table format ruled lines h1, h3, v0 and v2 are obtained. Next, the position of the field area on the image is determined from the position of the intersection formed by the table image ruled lines 60, 61, 62 and 63. The position of the intersection is known from the ruled line attribute information calculated by the ruled line attribute information calculating means 1c.

By the above processing, the field area 64 on the image corresponding to the field 20 is obtained. Since this is a field position search method based on the result of correspondence between ruled lines, an accurate field region position can be calculated based on the table comparison result even for a distorted table, a tilted table, an enlarged or reduced table. Also, a field having a complicated shape can be defined by increasing the number of table format ruled lines constituting the field.

When the field position is calculated, the field position searching means 5 ends, and finally, the processing by the result output means 6 is executed. In the result output means 6, the confidence calculated by the table matching means 4, the field position search means 5
Is output as a result.

When the processing in the result output means 6 ends, the table reading processing ends.

The present invention is not limited to the above-described embodiment, but can be variously modified without departing from the gist thereof.

[0077]

As described above, according to the present invention,
High-precision table matching can be performed even for tables with similar rule attribute information, tables containing disturbances such as handwritten characters, broken tables, distorted or inclined tables, and enlarged / reduced tables. A form processing device that can obtain a position can be realized.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a form processing apparatus according to the present invention.

FIG. 2 is a diagram showing a form image including a table.

FIG. 3 is a view showing vertical and horizontal ruled lines extracted in a table area.

FIGS. 4A to 4I are views showing intersection features between ruled lines; FIGS.

FIG. 5 is a diagram showing a rectangular table constituting the table.

FIG. 6 is a diagram showing a processing flowchart of a ruled line structure information calculating means.

FIGS. 7A to 7D are diagrams illustrating processing examples of a ruled line structure information calculating unit;

FIG. 8 is a diagram showing an example of a table format.

FIG. 9 is a diagram showing ruled lines extracted from a table in which characters are written.

FIG. 10 is a diagram showing a flowchart of a ruled line association process.

FIG. 11 is a diagram illustrating an example of calculating a ruled line ratio of a ruled line candidate.

FIG. 12 is a diagram showing that divided ruled line rates are the same.

13A and 13B are diagrams showing examples of interpolation of broken or blurred ruled lines.

FIG. 14 is a diagram illustrating an example of extracting a field region.

FIG. 15 is a diagram illustrating a configuration of a conventional form processing apparatus.

FIG. 16 is a diagram showing a form image including a table.

FIG. 17 shows horizontal ruled lines extracted in a table area.

FIGS. 18A to 18F are diagrams showing examples in which table matching becomes difficult.

[Explanation of symbols]

1 ruled line extracting means, 1a image input means, 1b table area extracting means, 1c ruled line attribute information calculating means, 2 table structure analyzing means, 2a table external tangential ruled line detecting means, 2b ruled line structure information calculating means, 2c table format creating means, 3 Table format database, 4 table matching means, 4a ruled line association means, 4b certainty factor calculating means, 5 field position searching means, 6 result output means, 7, 13 initialization means, 8 circumscribed ruled line registration means, 9 divided ruled line detection means , 1
0 determining means, 11 ruled line information reading means, 12 ruled line registration means, 14 end determining means, 15 table format reading means, 16 ruled line candidate calculating means, 17 divided ruled line calculating means, 18 ruled line collating means, 19 corresponding step increment means, 20 Field, form image including 40 tables, 41 table area in form image, 46, 47 table image ruled line associated with parent ruled line, 48 ruled line candidate search area, 50, 50a, 51, 51a table image, 52,
53 virtual ruled lines, table image ruled lines constituting 60 to 63 fields, 64 field regions.

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-62-38984 (JP, A) JP-A-4-167194 (JP, A) JP-A-3-504423 (JP, A) (58) Survey Field (Int.Cl. ⁷ , DB name) G06K 9/00-9/82

Claims (3)

(57) [Claims] 1. A form including a table is read as an image, and a ruled line extracting means for extracting a ruled line included in a table area in the image, a table structure and a character reading area position are determined from the ruled line extracted by the ruled line extracting means. Table structure analysis means for creating the table format shown, a table format database for storing the table format created by the table structure analysis means, and a table structure and a form described in the table format stored in the table format database A table matching unit that matches the ruled line extracted by the ruled line extracting unit from the table area; and a table matching result matched by the table matching unit and a character reading area position information described in the table format. A field position searching means for calculating a character reading area position in the table, a table matching result matched by the table matching means, And a result output means for outputting the character reading area position in the table calculated by the field position searching means. The table structure analyzing means comprises: a ruler line extracted from the ruled line extracting means; An external tangent ruled line detecting means for detecting an external tangent ruled line which is a ruled line, a means for registering the external tangent ruled line as a parent ruled line in an initial structure stage, Means for detecting inscribed ruled lines in contact with each other; means for registering, as the table structure, ruled line structure information for each of the structural stages based on the parent ruled line and the inscribed ruled line in each structural stage; Means for setting the detected inscribed ruled line as the parent ruled line in the next structure stage, and a form processing device including a table. 2. A form processing device including a table according to claim 1, wherein said table collating means compares the ruled line group extracted by said ruled line extracting means with a registered ruled line group described in said table format. Means for associating the table structure in a stepwise manner according to the structure step, wherein in each association step, the inscribed ruled line associated with the ruled line structure information of one of the registered ruled line groups in the structure step is A form processing apparatus including a table, comprising a ruled line association unit for associating the extracted ruled line group with the extracted ruled line group. 3. A form processing apparatus including a table according to claim 2, wherein said ruled line associating means uses, as an index, a correspondence between said extracted ruled line obtained before a current associating step and said registered ruled line. A form processing apparatus including a table, further comprising a ruled line candidate selecting means for selecting the candidate for the extracted ruled line to be associated with the ruled line in the current associating step.