JP2796561B2 - Tabular document recognition method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention reads a document in which characters, tables, figures, and the like are mixed to form document image information, and separates and recognizes individual information from the document image information. It relates to a tabular document recognition system.

[Conventional technology]

2. Description of the Related Art Conventionally, a character recognition device such as an optical character recognition device (OCR) reads a document as image information, separates this image information into rows or columns and a character group, and further subdivides the portion where it is considered to be a character to perform character recognition. However, when recognizing the characters in the table, the characters in the frame have different line spacing and character spacing due to the inclusion of the ruled lines, which increases the number of misrecognitions. The problem is that the meaning of the table itself is lost because it does not recognize the structure of the table itself.

In order to solve this problem, a document image is scanned in the horizontal (or vertical) direction as described in the IEICE Spring National Convention Preprints, p. The histogram of the number is formed, and then, for the peak portion of the histogram, a section whose height is equal to or larger than the threshold is set as the width of the peak, the one having a small width is extracted, and the corresponding document image region is defined as a line segment region. Further, the line segment area is scanned in the direction of the line segment, a line segment is extracted with a portion where the white pixels are continuous over the threshold value as an end point, and then a circumscribed rectangle of the table is obtained from the extracted line segment to generate column information, It has been proposed to recognize the table structure.

[Problems to be solved by the invention]

However, in the above-described conventional tabular document recognition method, a document image is scanned in a horizontal or vertical direction.
In the case of a ruled line that is continuous in the horizontal or vertical direction, the ruled line can be cut out. However, for example, if there is a short vertical ruled line connecting two horizontal ruled lines, this cannot be detected. In order to accurately recognize a finely divided table, it is necessary to divide a document image into regions having a size that can recognize short ruled lines, and perform a horizontal and vertical scan for each region to create a histogram. Yes, the processing time for performing image recognition becomes longer, and it is difficult to determine when there is a ruled line at the periphery of the divided region, and the processing time for connecting ruled line candidates extracted in each divided region also becomes longer. Further, there is an unsolved problem that a character group is not separated by a line such as a solid line or a dotted line, and a table separated by a virtual ruled line by a blank area cannot be recognized. .

Therefore, the present invention has been made by focusing on the unsolved problem of the conventional example, and can perform table recognition accurately in a short time, and can easily display a table even if it is a virtual ruled line by a blank area. It is an object of the present invention to provide a tabular document recognition system capable of performing recognition.

[Means for solving the problem]

In order to achieve the above object, a tabular document recognition method according to claim 1 reads a tabular document in which characters and ruled lines coexist with an image reading device, and reads a ruled line in image information from the image reading device. A table format recognizing method for extracting character information and character recognition, and determining horizontal and vertical ruled lines based on the image information and recognizing the table format. Character recognition means for collectively character recognition of character information of an area defined by each ruled line recognized by the means, the table format recognition means divides the image information into a band-shaped area of a predetermined width, Projection data forming means for forming projection data by projecting each band-shaped area in a direction perpendicular to the longitudinal direction thereof; ruled line candidate extracting means for extracting ruled line candidates from the projection data formed by the projection data forming means; Extraction hand It is characterized in that the in the extracted ruled line candidate and a ruled line determining means for determining a border linked between adjacent band-like region.

In the tabular document recognition method according to claim (2), in the invention according to claim (1), the table format recognition unit scans the image information in one of a vertical direction and a horizontal direction. One of the ruled lines is determined by the projection data forming means, ruled line candidate extracting means and ruled line determining means, and erased image information obtained by deleting the determined ruled line from the image information is scanned by the other, and the projection data forming means and ruled line candidate extracting are performed. Means and the ruled line determining means determine the other ruled line, scan the ruled line erased image information obtained by erasing the determined ruled line from the erased image information, and extract a blank portion. A blank portion other than the horizontal ruled lines is determined as a virtual ruled line, and the table format is recognized based on the horizontal and vertical ruled lines and the virtual ruled line.

[Action]

In the tabular document recognition system according to claim (1),
The binarized image information read by the image reading device is divided into, for example, a plurality of horizontal belt-like regions, and projection data is obtained for each of these belt-like regions in a direction orthogonal to the longitudinal direction. By extracting the vertical ruled line candidates, the vertical ruled line candidates can be accurately extracted, and the vertical ruled lines can be recognized by connecting the extracted ruled line candidates between adjacent belt-shaped regions. In order to extract a horizontal ruled line, the binarized image information is divided into a plurality of strips in the vertical direction, and horizontal ruled line candidates are extracted from projection data orthogonal to the longitudinal direction.
A horizontal ruled line can be extracted by connecting ruled line candidates in each band-shaped region. The character information of the area defined by the extracted ruled lines is collectively recognized by the character recognizing means, so that only characters or only numbers are designated for each row or column item. Thus, the character recognition rate and the recognition speed in the character recognition device can be improved.

Further, in the tabular document recognition method according to claim (2), for example, a vertical ruled line is recognized from the image information, and then the recognized vertical ruled line is deleted from the image information. , And scans the ruled line erased image information in which the horizontal ruled lines have been erased from the erased image information to extract blank portions. Of the extracted blank portions, blank portions other than the vertical and horizontal ruled lines are extracted. Since it is determined as a virtual ruled line and the table format is recognized based on the horizontal and vertical ruled lines and the virtual ruled line, the ruled line recognition time is reduced and accurate ruled line recognition is performed.

〔Example〕

 Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of the present invention.

In the figure, 1 is a image scanner is controlled by a configured processor 4 with a personal computer or the like, as shown in FIG. 2, which is formed by the vertical ruled lines R _V and horizontal ruled lines R _H in the table T It reads a tabular document D on which arbitrary characters are drawn, and outputs binarized image information obtained by binarizing this into black and white pixels at a predetermined level.

The binarized image information of the image scanner 1 is sent to a graphic separation unit 2 and divided into table information and character information divided by the graphic separation unit 2, and the character information is converted into an optical character recognition device (OCR). 3), the character data is recognized by the optical character recognition device 3, and the recognized character data is input to the processing device 4.

The figure separating unit 2 is, for example, a storage device including an interface circuit 2a, an arithmetic processing unit 2b, and a dot-compatible image memory.
2c, when the binarized image information is input from the image scanner 1, the binarized image information is stored in the image memory of the storage device 2c, and then the processing shown in FIG. Execute That is, first run the vertical ruled line extracting process for extracting a vertical ruled line R _V in step, then the process proceeds to step running horizontal ruled line extracting process for extracting a horizontal rule R _H, then depending on the borders proceeds to step not virtual ruled-line extraction process for extracting the virtual border R _I by blank areas corresponding to the ruled line is executed, then the vertical ruled line extracted with proceeds to the processing in step R _V, surrounded by horizontal rule R _H and the virtual border R _I Is sent to the optical character reading device 3.

Here, in the vertical ruled line extraction process, as shown in FIG. 4, the binary image information stored in the image memory in the step is converted into a predetermined width in the horizontal direction (X) as shown in FIG.
A plurality of N strip-like regions BX ₁ ~BX _N sequential vertical predetermined width W min increments read in the horizontal direction so as to divide into, each band region BX ₁ ~BX _N the longitudinal direction for extending direction) (X
Direction (Y direction) perpendicular to the vertical direction (Y direction) to form vertical projection data of a logical value "1" which becomes a black pixel as shown in FIG. 7 (b). Vertical ruled line candidates are extracted from the projection data. The extraction of the vertical ruled line candidate is performed by re-projecting the projection data at a preset threshold value.
The binarized data is searched for a portion where "1" s are continuous. When the continuous width of "1" is smaller than a predetermined width, the ruled line is usually narrow. Extract as candidates and extract others as character candidates.

Next, the process proceeds to a step, in which a ruled line candidate re-extraction process for extracting a vertical ruled line candidate included in the character candidate is executed. That is, as shown in FIG. 7 (a), in the case where a certain band region BX _i includes a horizontal ruled lines R _H, the projection data, as shown in FIG. 7 (b), the horizontal ruled lines R _H Is increased by the logical value “1” representing the vertical ruled line R _H even if re-binarization is performed at a predetermined threshold T ₁ indicated by a dotted line in FIG. 7B.
The logical value “1” continues between them, and the whole is recognized as a character candidate. For this reason, in the ruled line candidate re-extraction process, the maximum value within the width section where the logical value “1” is continuous (when a ruled line is included, the ruled line portion usually has a high peak) is determined by By performing binarization again with the average value, a vertical ruled line candidate included in a character candidate can be extracted as shown in FIG. 7 (c).

Next, the process proceeds to steps, where each of the band-shaped areas BX _{1 to} BX _N
It is determined whether or not extraction of ruled line candidates has been completed for
If there is a band area for which ruled line candidate extraction has not been completed, the process proceeds to step S, where the read address of the next image memory is set so as to be the next band area BX _{i + 1} . when the ruled line candidate extraction has been completed for the band region BX ₁ ~BX _N, to form a vertical ruled line candidate sequence by concatenating the vertical ruled line candidates extracted proceeds to step on each strip-like regions BX ₁ ~BX _N in the vertical direction . This vertical ruled line candidate row is formed by extracting vertical ruled line candidates having an overlap in the vertical direction (Y direction) from each ruled line candidate group between adjacent belt-shaped regions and connecting them.

Next, the process proceeds to a step, in which the vertical ruled line candidate sequence is projected in a direction orthogonal to the extension direction, that is, in the X direction, to obtain projection data. When the projection data is continuous for a certain level or more, it is recognized as a ruled line. The vertical ruled line is determined by obtaining the start and end coordinates from the projection data, and then the process is terminated and the process returns to the process of FIG.

In the horizontal ruled line extraction process, as shown in FIG. 5, first, in a step, the vertical ruled line R _V determined in the vertical ruled line candidate extraction process is deleted from the binarized image information stored in the image memory. Binary image information is created and stored in an image memory.

Next, the process proceeds to a step, wherein the band-shaped area in the vertical ruled line candidate extraction processing is applied to the erased binary image information.
BX ₁ instead ～BX _N to form a vertical stripe region BY ₁ ~BY _M by dividing in the vertical direction at a predetermined width W, a predetermined width of one by reading vertically erase binary image information, the longitudinal Projection data is created by projecting in the X direction orthogonal to the direction (Y direction).

Next, the process proceeds to a step in which horizontal ruled line candidates are extracted by performing the same processing as the vertical ruled line candidate extracting process based on the projection data.

Then, the process proceeds to all the vertical band areas BY
_It is determined whether or not the extraction of horizontal ruled line candidates has been completed for _{1 to} BY _M , and if the extraction has not been completed, the process proceeds to step and sets a read address corresponding to the next vertical band-shaped region BY _{j + 1.} Then, the process returns to the step, and when the extraction of the horizontal ruled line candidates is completed for all the vertical band-shaped areas BY _{1 to} BY _M , the process proceeds to the step.

In this step, a horizontal ruled line candidate row is created by extracting and connecting horizontal ruled line candidates in adjacent vertical band-shaped areas, and then the process proceeds to step to project the horizontal ruled line candidate row in the X direction. Determine the ruled line.

Further, the virtual ruled line extraction process is performed as shown in FIG.
First, as shown in FIG. 8, a rectangular outermost ruled line frame circumscribing the outermost ruled lines of the vertical ruled line R _V and the horizontal ruled line _RH obtained by the vertical ruled line candidate extracting process and the horizontal ruled line candidate extracting process as shown in FIG. Set the inside of F to the virtual ruled line processing area,
Next, the process proceeds to a step, in which the horizontal ruled line _RH is deleted from the vertical ruled line deleted binary image information stored in the image memory to form binary image information as shown in FIG. Then, similarly to the horizontal ruled line candidate extraction processing described above, the binarized image information is converted into a plurality of M band-shaped areas BY _1.
Divided into ～BY _M, for each of these strip-like regions BY ₁ ~BY _M to form an X-direction projection data, the process proceeds to step in with the virtual border candidate region representing a logic value "0" from the projection data Extract as As a result, for example, each band-shaped area BY
_As shown in FIG. 10A, virtual ruled line candidates in a blank portion can be extracted from _i-1 , BY _i and BY _{i + 1} .

Then, the process proceeds to all the vertical band areas BY
_It is determined whether or not the extraction of the virtual ruled line candidates has been completed for _{1 to} BY _{M. If} there is a vertical band-shaped region for which the extraction of the virtual ruled line candidates has not been completed, the process proceeds to step and the read address for the image memory is changed to the next band. After the address is updated to the address corresponding to the area BY _{i + 1} , the process returns to the step. When the extraction of the virtual ruled line candidates has been completed for all the vertical band-shaped areas BY _{1 to} BY _M , the process proceeds to the step.

In this step, the virtual border candidates extracted in step, because all the blank portions are candidates, the difference in the size of the candidate becomes remarkable, the strip-like regions BY _i in example FIG. 10 (a), all since becomes virtual ruled line candidate, and the virtual border candidate of the band-like region bY _i, strip-like regions bY _i-1 and bY _{i + 1} of the virtual border candidate CA ₁ and CA ₂ adjacent thereto
Are calculated, the number of virtual ruled line candidates is compared, and the virtual ruled line candidates are divided according to the larger virtual ruled line candidate to select virtual ruled line division candidates CA ₁ ′ and CA ₂ ′.

Next, the process proceeds to step S, among all the virtual ruled line candidates including the extracted virtual ruled line division candidates, searching for and connecting the virtual ruled line candidates that are adjacent to each other in the adjacent band-shaped region in the horizontal direction, and linking the virtual ruled line candidate columns. Form.

Next, the process goes to a step where the virtual ruled line candidate line overlapping with the vertical ruled line R _V and the horizontal ruled line _RH determined in the above described ruled line candidate extracting process is deleted from the virtual ruled line candidate sequence, and the virtual ruled line candidate sequence is virtualized. After determining the ruled line, the process returns to the processing of FIG. In this determination of the virtual ruled line, since the virtual ruled line candidate column is wider than the ruled line, the middle point of the virtual ruled line candidate at the start and end of the virtual ruled line candidate column is made representative, and the coordinates of this representative point are obtained. Thus, a line segment connecting both representative points is determined as a virtual ruled line.

Note that when determining the virtual ruled line, in rare cases, FIG.
There may be multiple virtual border candidate CA ₁₂ and CA ₁₃ narrow is articulated with respect to the wide of the virtual border candidate CA ₁₁ as shown in,
In this case, if the middle point of the wide virtual ruled line candidate CA _{11 and} the middle point of the narrow virtual ruled line candidate CA ₁₃ are selected as representative points, the virtual ruled line will be erroneously extracted. When the virtual ruled line candidate is formed by concatenating the virtual ruled line candidate with the narrow virtual ruled line candidate, as shown in FIG. And the virtual ruled line is determined, and for the remaining narrow virtual ruled line candidates, the boundary position in contact with the wide ruled line candidate is determined as the end point or start point coordinates of the virtual ruled line.

When knowledge of the table format is completed by determining all ruled lines and virtual ruled lines in this way, the process proceeds to the step of FIG. 4 to collectively collect the character information of the column surrounded by each ruled line into an optical character. Output to the recognition device 3.

Therefore, in the optical character recognition device 3, the tabular document D
Since character recognition is performed for each column, the character recognition rate can be improved, and the table structure can be recognized. It is possible to specify only characters or only numbers, the number of selected characters can be selected, and the recognition rate and the recognition speed can be improved. Further, by converting the characters recognized by the optical character recognition device 3 into control codes and inputting them to the arithmetic processing device 4 composed of a personal computer or the like, the names, addresses, telephone numbers, etc. When a database is created, it is only necessary to read these documents with an image scanner, and the database can be easily created.

In the above embodiment, the case where the image scanner 1 is applied as the image reading apparatus has been described. However, the present invention is not limited to this, and any image reading apparatus such as a television camera and a solid-state imaging device may be applied. Can be.

In the above embodiment, the case where the ruled line of the tabular document is a solid line has been described. However, the present invention is not limited to this. Even when the ruled line is a broken line, a chain line, or the like, the peak of the projection data has a solid line. Since it is only smaller than the case, the ruled line can be easily recognized.

Furthermore, in the above-described embodiment, the case where the reading direction of the image scanner 1 and the ruled line direction of the tabular document match each other has been described.
If the valued image information deviates from the reading direction of the image scanner 1, the vertical ruled line R _V , the horizontal ruled line _RH, and the virtual ruled line
When determining the starting and end coordinates of the R _I, taking the direction of the projection data to be orthogonal to the projection direction when extracting a ruled line candidate of the ruled line candidate string, whereby along with determining the start and end coordinates, borders candidate strings By obtaining the equations, the start point and end point coordinates can be determined. However, for the virtual ruled line candidate, as described above, since there is no actual ruled line, an equation of a straight line connecting the start point and the middle point of the terminal point of the virtual ruled line candidate sequence serving as a representative point is used.

〔The invention's effect〕

As described above, in the tabular document recognition method according to claim (1), the vertical ruled lines and the horizontal ruled lines forming the table of the tabular document are divided into a band-like area orthogonal to both. Since ruled line candidates are extracted based on projection data obtained by projecting image information in a direction orthogonal to the longitudinal direction, the number of divisions of the binarized image information can be reduced, and the recognition speed of the tabular document can be reduced. In addition, since the table structure is recognized and character recognition is performed for each column of the table structure, it is possible to specify only characters or only numbers for each item for a row or a column. By limiting the number of selected characters in the character recognition device, the character recognition rate and the recognition speed of the character recognition device can be improved, and the table format can be recognized. Every day When creating a base, effect that can be created easily by simply reading in character reader tabular document is obtained.

In the tabular document recognition method according to claim (2), in recognizing the ruled line in the effect of claim (1), every time the ruled line is recognized, the recognized ruled line is deleted from the image information. Since ruled line recognition is performed, accurate ruled line recognition can be performed by shortening the ruled line recognition time. In addition, since a blank portion is extracted and a virtual ruled line is determined, the ruled line is not separated by the ruled line, and is Even if the table format is divided by the virtual ruled line by the part, it can be recognized as having a ruled line, so that the character recognition rate and the recognition speed in the character recognition device can be further improved. .

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of the present invention, FIG. 2 is an explanatory diagram showing a tabular document, FIGS. 3 to 6 are flowcharts showing an example of a processing procedure in a graphic separation unit, FIG.
(A), (b) and (c) are explanatory diagrams showing a horizontal band-like region, its vertical projection data and re-binarized data for explaining the operation of the present invention, respectively. FIG. 8 shows a virtual ruled line processing region. FIG. 9 is an explanatory diagram showing erasure binarization information;
FIGS. 10 (a) and (b) and FIGS. 11 (a) and (b)
FIG. 7 is an explanatory diagram for explaining a virtual ruled line extraction process. In the figure, 1 is an image scanner (character reading device), 2 is a graphic separation unit, 3 is an optical character recognition device, 4 is an arithmetic processing device, D is a tabular document, R _V is a vertical ruled line, and R _H is a horizontal ruled line. is there.

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G06K 9/00-9/82

Claims (2)

(57) [Claims] An image reading apparatus reads a tabular document in which characters and ruled lines are mixed, cuts out ruled lines in image information from the image reading apparatus, and recognizes character information as characters. A table format recognition unit that determines a horizontal and vertical ruled line based on the image information to recognize a table format, and character information of an area defined by each ruled line recognized by the table format recognition unit. Character recognition means for collectively recognizing characters, wherein the table format recognition means divides the image information into band-shaped regions having a predetermined width, and projects projection data obtained by projecting each band-shaped region in a direction orthogonal to its longitudinal direction. Projection data forming means for forming, ruled line candidate extracting means for extracting ruled line candidates from the projection data formed by the projected data forming means, and connecting ruled line candidates extracted by the ruled line candidate extracting means between adjacent belt-shaped regions. Ruled line Tabular document recognition system, characterized in that it comprises a ruled line determination means for determining 2. The table format recognizing means scans the image information in one of a vertical direction and a horizontal direction, and determines one ruled line by the projection data forming means, ruled line candidate extracting means and ruled line determining means. The erased image information obtained by erasing the determined ruled line from the image information is scanned on the other side, and the other ruled line is determined by the projection data forming means, ruled line candidate extracting means and ruled line determining means. A blank portion is extracted by scanning the ruled line erased image information erased from the information, and a blank portion other than the vertical and horizontal ruled lines among the extracted blank portions is determined as a virtual ruled line, and a horizontal and vertical ruled line is determined. The table format document recognition method according to claim 1, wherein the table format is recognized based on the virtual ruled lines.