JPH10232939A - Device and method for ruled-line for extracting ruled line from general document image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly extract a ruled line from a general document image whose ruled-line structure can not be predicted in advance. SOLUTION: A threshold is set according to the size of a specified standard pattern and a straight-line pattern is extracted. A ruled-line extracting device finds the circumscribed rectangle of a pixel connection area in an input pattern and calculates its height of the highest frequency (P1). Segments are integrated (P4, P9) while wild-card segments are ignored and the height and width of an extracted straight line and the most frequent values of the height and width of the segments constituting the straight line are calculated (P12 to P15). Those most frequent values and various thresholds are used to perform integrating and removing processes for the straight line (P16 to P21), the black pixel distribution of the circumference of the straight line is checked and removed (P22, P23), and the left straight line is decided as a ruled-line candidate (P24).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a ruled line extracting apparatus and method for extracting a ruled line portion from an arbitrary document image read by a photoelectric conversion device or the like.

[0002]

2. Description of the Related Art In recent years, in order to make office work more efficient,
There is an increasing demand for an electronic filing system that digitizes paper documents and stores them on an optical disk or the like. In a conventional electronic filing system, a paper document is converted into an image by a photoelectric conversion device such as an image scanner, a keyword for search is given to the image, and the image is stored on an optical disk or a hard disk. However, these keywords and the like have to be keyed, and the input operation is complicated.

[0003] As a prior application technique by the present applicant to solve this inconvenience, there is "Apparatus and method for extracting title from document image" (Japanese Patent Application No. 07-341983). According to this method, a document title included in an image is automatically extracted / registered as a keyword. In addition, management information such as title, destination, and transmission source information can be automatically extracted from images of various documents including a table format document. For example, a title outside a table can be extracted with approximately 90% accuracy. I know.

However, the extraction accuracy of titles in the table is only about 55%, which is insufficient for practical use. To extract keywords such as titles in a table with high accuracy, it is necessary to accurately extract ruled lines constituting the table. Ruled line extraction technology has been developed mainly for forms in which characters and the like are regularly arranged.

[0005] As the conventional ruled line extraction technology, there are an "image extraction method" (Japanese Patent Laid-Open No. 6-309498) and an "image extraction device".
(JP-A-7-28937). According to these techniques, it is possible to extract or remove a frame without having to input information such as the position of the frame in the form. A form that can be processed has a single character frame, a block frame (horizontal one line frame or free format frame), or a table having a structure in which the outline of the frame is rectangular and the horizontal frame lines are arranged regularly. It is.

[0006] As a prior art ruled line extraction technology by the present applicant, a "frame extraction device and a rectangle extraction device" (Japanese Patent Application No. Hei.
-203259), “Pattern region extraction method and pattern extraction device” (Japanese Patent Application No. 7-282171), and “Pattern extraction device and pattern region extraction method”
(Japanese Patent Application No. 8-107568).

According to these techniques, even if the outline of the frame is rectangular as shown in FIG. 48, or even if the outline of the frame is not rectangular as shown in FIG. It can be carried out. Further, it is possible to extract and remove a frame in a tabular form having a finer structure in a rectangle surrounded by the frame as shown by a shaded portion in FIG. The outline of the process is shown below.

(1) Thinning: Vertical and horizontal line segments are thinned by mask processing to eliminate the difference in thickness between characters and frames. (2) Line segment extraction: "Image extraction method" (Japanese Patent Laid-Open No. 6-309)
498), a relatively long straight line is extracted using the adjacent projection method. The adjacent projection is a method of adding the projection value of a pixel included in a row or column of interest to the projection value of a surrounding row or column as the final projection value of the row or column of interest. is there. According to this projection method, a pixel distribution around a specific row or column can be grasped globally.

(3) Straight line extraction: The extracted line segments are searched in order, and it is checked whether there is a break of a certain length or more between the line segments. Line segments without such a break are integrated in order, and a long straight line is extracted.

(4) Straight line integration: The extracted straight lines are integrated again. Straight lines separated into two or more parts by blurring are integrated into one straight line. (5) Straight line extension: Only when it is known that the document is a regular form, the straight line shortened due to blurring is extended and restored to its original length.

[0011]

However, the above-described ruled line extraction technique has the following problems. In the technology of the prior application, the form of a form can be processed as long as it has a regular or irregular structure as long as it is a tabular frame composed of rectangular areas. Also, the ruled line that is the target can be processed regardless of whether it is a solid line or a dotted line, regardless of whether it is blurred or not. Further, only when it is known that the table is a regular table, a process of extending a straight line shortened by extreme blurring is performed.

However, in a general input image, characters are described in a bold font as shown in FIG.
Some of the tables have shaded areas. In such a case, a ruled line may be erroneously extracted from a character string crushed by contact between characters, and a correct ruled line and an erroneously extracted ruled line may be erroneously integrated.

A ruled line contacting a cluster of black pixels, such as a shaded portion, or a ruled line contacting a character cannot be extracted. In order to eliminate such inconveniences, it is desirable that only a tabular document such as a form having a known ruled line structure be processed.

However, it is not known in advance what kind of table is included in a general document handled by electronic filing, and there is a high possibility that various images including crushed characters and the like are input. Therefore, there is a problem that the ruled line cannot always be correctly extracted using the technique of the prior application.

It is an object of the present invention to provide a ruled line extracting apparatus and method capable of correctly extracting a ruled line portion from a general document image whose ruled line structure cannot be predicted in advance.

[0016]

FIG. 1 is a diagram showing the principle of a ruled line extracting apparatus according to the present invention. The ruled line extraction device of FIG. 1 is the first, second, third, fourth, fifth, sixth, seventh, eighth, and eighth embodiments of the present invention.
Including the ninth, tenth, and eleventh principles, estimating means 1, storage means 2, line segment extracting means 3, calculating means 4, straight line extracting means 5, graph generating means 6, straight line processing means 7, straight line integrating means 8 and a straight line removing means 9.

In the first principle, the estimating means 1 estimates the size of a standard pattern included in an input image. The straight line extracting means 5 sets a threshold value based on the information on the size of the standard pattern, and extracts information of one or more straight line patterns from the input image using the threshold value.

The standard pattern corresponds to a pattern such as a character having a standard size most often included in an input image.
As the standard pattern, for example, a pixel connection region representing a character is used, and as the size information, for example, the height or width of a circumscribed rectangle of the region is used.

The linear pattern corresponds to a horizontal or vertical pattern extracted from an input image by mask processing using a horizontal or vertical mask and line segment integration processing. The information on the straight line pattern includes, for example, coordinate values of a circumscribed rectangle of a plurality of line segment patterns constituting the pattern. The line segment pattern corresponds to a line-shaped pixel region cut out from the image by the mask processing.

The straight line extracting means 5 determines various threshold values based on the size of the standard pattern, and classifies the straight line patterns in the image based on the threshold values. This allows
A straight line pattern derived from a shaded portion or a contact character can be excluded from ruled line candidates, and a correct ruled line candidate can be extracted.

In the second principle, the straight line extracting means 5
The information of one or more linear patterns is extracted from the input image. The calculating means 4 calculates a representative value of the size of the one or more linear patterns. The straight line processing means 7 sets a threshold value based on the representative value, and processes the information of the one or more linear patterns using the threshold value.

The calculating means 4 obtains a representative size of a straight line pattern from a histogram of the heights or widths of a plurality of straight line patterns, for example. The straight line processing means 7 can set a threshold value around this representative value and perform processing such as excluding a straight line pattern having a larger size from the ruled line candidates, thereby extracting a correct ruled line candidate.

In the third principle, the straight line extracting means 5
The information of one or more linear patterns is extracted from the input image. The calculating means 4 calculates a representative value of the size of one or more line segment patterns constituting the one or more linear patterns. The straight line processing means 7 sets a threshold value based on the representative value, and processes the information of the one or more linear patterns using the threshold value.

As described above, the line segment pattern corresponds to a line-shaped pixel region cut out from the image by the mask processing. The calculating means 4 obtains a representative size of the line segment pattern from a histogram of the heights or widths of the plurality of line segment patterns, for example. The straight line processing means 7 can perform processing such as excluding a straight line pattern composed of only line segment patterns larger than the threshold value based on the representative value from the ruled line candidates, and can extract correct ruled line candidates.

In the fourth principle, the line segment extraction means 3
Information of one or more line segment patterns is extracted from the input image. The storage means 2 stores the information of the one or more line segment patterns separately into information of a large line segment pattern and information of a small line segment pattern. The straight line extracting means 5 is used for the above 1
Examine the connection state of one or more line segment patterns and, when the above-mentioned large line segment pattern exists in the middle, regardless of the size of the large line segment pattern, extract a straight line pattern constituted by other plural line segment patterns I do.

The information on the line segment pattern includes, for example, coordinate values of the circumscribed rectangle. For example, the storage unit 2 adds specific attribute information to information of a line segment pattern larger than an appropriate threshold value and stores the information separately from information of other small line segment patterns. The straight line extracting means 5 ignores a large line segment pattern and extracts a small line segment pattern on both sides when integrating a plurality of overlapping line segment patterns and extracting a circumscribed rectangle as a straight line pattern. A process for appropriately connecting each other is performed.

As a result, a straight line pattern which is not affected by the size of a large pixel area such as a shaded portion or a character can be extracted as a correct rule line candidate from the image of the ruled line.

According to the fifth principle, the straight line extracting means 5
The information of one or more linear patterns is extracted from the input image. When the two linear patterns included in the one or more linear patterns almost overlap, the straight line integrating unit 8 integrates the two linear patterns into one.

The straight line integrating means 8 integrates two almost overlapping linear patterns, thereby reducing redundant straight line information and extracting correct ruled line candidates. According to the sixth principle, the straight line extracting means 5 extracts information of one or more straight line patterns from the input image. Straight line removing means 9
Is information regarding one form of the one or more linear patterns and information regarding a distance between two linear patterns included in the one or more linear patterns.
At least one piece of information is used to determine whether to remove one of the one or more linear patterns.

The straight line removing means 9 determines the likelihood of a ruled line of the straight line pattern based on, for example, the aspect ratio of the straight line pattern and the distance to the next straight line pattern, and removes a straight line pattern that is not considered to be a ruled line. As a result, a straight line pattern derived from a shaded portion, a crushed character string, or the like can be excluded from ruled line candidates, and a correct ruled line candidate can be extracted.

In the seventh principle, the straight line extracting means 5
The information of one or more linear patterns is extracted from the input image. The straight line removing means 9 determines whether to remove one of the horizontal straight line pattern and the vertical straight line pattern based on the connection relationship between the horizontal straight line pattern and the vertical straight line pattern included in the one or more straight line patterns. .

The straight line removing means 9 excludes, for example, a vertical straight line pattern that does not contact any horizontal straight line pattern or a horizontal straight line pattern that does not contact any vertical straight line pattern from ruled line candidates. Thus, a straight line pattern derived from a crushed character string or the like can be excluded from ruled line candidates, and a correct ruled line candidate can be extracted.

According to the eighth principle, the straight line extraction means 5
The information of one or more linear patterns is extracted from the input image. The straight line removing unit 9 removes the shorter one of the two line patterns that are included in the one or more straight line patterns and almost overlap with each other.

The straight line removing means 9 removes the shorter one of the two almost overlapping straight line patterns, so that redundant straight line information can be reduced and correct ruled line candidates can be extracted.

In the ninth principle, the straight line extracting means 5
The information of one or more linear patterns is extracted from the input image. When the size of a straight line pattern obtained by integrating two partially overlapping straight line patterns of the one or more straight line patterns becomes approximately a predetermined value,
The straight line pattern after integration is set as a ruled line candidate.

For example, if the thickness of a straight line pattern obtained by integrating two straight line patterns is about the typical thickness of a straight line pattern, the straight line integrating means 8 executes the integration process. As a result, redundant straight line information can be reduced, and correct ruled line candidates can be extracted.

In the tenth principle, the straight line extracting means 5
Extracts information of one or more linear patterns from an input image. The straight line removing means 9 removes a straight line pattern composed of only line segment patterns larger than the threshold value from the one or more straight line patterns.

The straight line removing means 9 removes, from the ruled line candidates, straight line patterns composed of only line segment patterns that are considerably thicker than the typical line segment patterns, for example.
Thus, a straight line pattern derived from a crushed character string or the like can be excluded from ruled line candidates, and a correct ruled line candidate can be extracted.

In the eleventh principle, the straight line extracting means 5
Extracts linear pattern information from an input image. The graph generating means 6 obtains the number of pixels inside a line segment pattern of a standard size from among one or more line segment patterns constituting the straight line pattern, and obtains a graph of the number of pixels around the straight line pattern. Generate The straight line removing unit 9 determines whether to remove the straight line pattern based on the shape of the graph.

The graph generation means 6 generates a set of line segment patterns of a standard size, for example, excluding a large line segment pattern from a set of line segment patterns forming a straight line pattern. Then, they are shifted to the area around the linear pattern, and a graph representing the relationship between the shift amount and the number of pixels is generated. Further, for example, when the shape of the graph is gentle and the maximum value is not clear, the straight line removing unit 9 removes the straight line pattern from the ruled line candidates.

In the case of a straight line pattern extracted from the inside of a shaded portion, a crushed character string, or the like, pixels are often distributed evenly around the straight line pattern. In such a case, the shape of the graph becomes gentle and the straight line pattern is excluded from the ruled line candidates, so that a correct ruled line candidate can be extracted.

For example, the storage means 2 in FIG. 1 corresponds to the memory 32 in FIG. 3, which will be described later, and the estimating means 1, the line segment extracting means 3, the calculating means 4, the straight line extracting means 5, the graph generating means 6, the straight line processing The means 7, the straight line integrating means 8, and the straight line removing means 9 include a CPU (central processing unit) 31 and a memory 3
Corresponds to 2.

[0043]

Embodiments of the present invention will be described below in detail with reference to the drawings. At present, a form learning system for tabular documents is being developed in order to automatically extract keywords such as titles in a table with high accuracy. In this system, a document including a table is registered in advance, and a correct keyword can be extracted later with high accuracy from the registered document. INDUSTRIAL APPLICABILITY The present invention can be used to correctly extract ruled lines from a document image when learning the form of a tabular document.

According to the present invention, the conventional and prior application ruled line extraction technology for forms is improved, and information on small lines constituting the ruled line is utilized to detect incorrect lines and characters extracted from the original ruled line. Distinguish from the extracted straight line. Thus, even when a character is in contact with a ruled line, a ruled line can be correctly extracted.

Further, even if there is a line segment extracted from a crushed portion in the table, ruled line candidates are obtained by paying attention to only the line segment extracted from the original ruled line. Then, only the correct ruled line is extracted from the shape and positional relationship of the ruled line and the distribution of black pixels of the line segment inside the ruled line.

In the following embodiment, there is one or more frames whose size, position, and inclination are unknown, and various characters including characters that touch the frame and characters that protrude from the frame are added to those frames. The target is a written document. Then, consider a case where a frame is extracted from an image of such a document.

FIG. 2 is a functional block diagram of the ruled line extracting device according to the embodiment. In FIG. 2, a target input pattern 11 is a binary image after correcting an extreme inclination or rotation. The shaded processing blocks are mainly
"Pattern extraction device and method of extracting pattern area"
(Processing different from the prior application such as Japanese Patent Application No. 8-107568).

First, after the reduction processing section 12 reduces the image and the connection pattern extraction section 13 extracts the connection pattern, the ruled line extraction device calculates the mode of the height of the rectangle (processing P1),
The mask processing unit 14 performs thinning.

Next, after the horizontal straight line extraction unit 15 performs horizontal adjacent projection (process P2), horizontal line segment detection (process P3), horizontal line segment integration (process P4), and horizontal line search (process P5), The ruled line extraction device performs horizontal dotted line detection (process P6). Next, the vertical straight line extraction unit 16 performs vertical adjacent projection (process P7), vertical line segment detection (process P8), vertical line segment integration (process P9), and vertical line search (process P10). The extraction device performs vertical dotted line detection (process P11).

Next, the ruled line extraction device calculates the mode of the height of the horizontal line (process P12), calculates the mode of the width of the vertical line (process P13), and calculates the height of the height of the horizontal line. The mode is calculated (process P14), and the mode of the width of the vertical line is calculated (process P15). Next, completely overlapping straight lines are integrated (process P16), and unnecessary straight lines are removed based on the shape of the straight rectangle or the distance to an adjacent straight rectangle (process P17). Next, unnecessary straight lines are removed based on the connection relationship between the vertical and horizontal straight lines (processing P
18) Integrate partially overlapping straight lines (Process P
19).

Next, the ruled line extracting device removes a straight line that almost completely overlaps with another straight line (process P20), and removes a straight line consisting only of a line segment larger than a predetermined threshold (process P21). Then, a mark is added to a line segment larger than another threshold value (process P22), and a straight line is checked while shifting the target line segment, and unnecessary straight lines are removed (process P22).
23) Output the remaining straight line.

The ruled line extracting apparatus of this embodiment is, for example, shown in FIG.
The information processing device (computer) as shown in FIG. The information processing apparatus of FIG.
2, input device 33, output device 34, external storage device 35,
A medium drive device 36, a network connection device 37, and a photoelectric conversion device 38 are provided, and these devices are connected to each other by a bus 39.

The CPU 31 executes a program stored in the memory 32 to perform each processing shown in FIG. As the memory 32, for example, a ROM (read only memory),
A RAM (random access memory) or the like is used. The memory 32 stores the above-described programs and data necessary for processing.

The input device 32 corresponds to, for example, a keyboard, a pointing device or the like, and is used for inputting a request or instruction from a user. The output device 34 corresponds to a display device, a printer, or the like, and is used for outputting a processing result or the like.

The external storage device 35 is, for example, a magnetic disk device, an optical disk device, a magneto-optical disk device, or the like. The above-mentioned program and data are stored in the external storage device 35 and, if necessary, stored in the memory 32.
Can be loaded and used. The external storage device 35 is also used as a database of an electronic filing system for storing images, keywords, and the like.

The medium driving device 36 drives the portable recording medium 40 and can access the stored contents. Examples of the portable recording medium 40 include a memory card, a floppy disk, and a CD-ROM (compact disk read only memory).
y), any computer-readable recording medium such as an optical disk and a magneto-optical disk can be used.
The above-described program and data are stored in the portable recording medium 40, and can be used by loading them into the memory 32 as needed.

The network connection device 37 is a LAN (lo
It is connected to an arbitrary communication network such as a cal area network, performs data conversion and the like accompanying the communication, and communicates with an external information provider database 40 'and the like. This allows
The ruled line extracting apparatus can receive the above-mentioned programs and data from the database 40 'via a network as needed, and load them into the memory 32 for use.

The photoelectric conversion device 38 is, for example, an image scanner, and inputs a general document image to be processed. In the memory 32, data necessary for processing is managed in a structure as shown in FIG. 4, for example. In FIG. 4, the information 41 of one input image includes the number of tables (table type frames) included in the image and information 42 for each table.

Each table information 42 includes coordinate values of a circumscribed rectangle of the table,
Number of cells included in table, information 43 for each cell, number of horizontal straight lines (horizontal straight lines) included in table, information 4 for each horizontal straight line
4. The number of vertical straight lines (vertical straight lines) included in the table, and information 44 for each vertical straight line. Here, a cell represents an area surrounded by ruled lines at the top, bottom, left, and right.

Each cell information 43 includes a coordinate value of a cell, and each straight line information 44 includes coordinate values of a rectangle representing a straight line, attribute information of the straight line, the number of small line segments included in the straight line, and each small line segment. Information 45 and serial numbers of straight lines in the entire image. Each piece of line segment information 45 includes attribute information of the line segment and coordinate values of a rectangle representing the line segment. The attribute information of the straight line and the small line segment may be, for example, distinguishing between a solid line and a broken line, or a wild card (wildca) whose height or width exceeds a certain value.
rd) Used to distinguish a line segment from other line segments. Next, each process of FIG. 2 will be described in order with reference to FIGS.

When the image of the input pattern 11 has a certain resolution or higher and the size of the image is relatively large, the reduction processing section 12 reduces the size of the image to increase the processing efficiency. The input original image is stored as it is.

The connected pattern extracting unit 13 connects the black pixels connected in eight vertical, horizontal, and right connections in order to stably pick up each pattern without depending on the relative relationship between the positions where a plurality of tables are arranged. The region is extracted as a partial pattern by labeling. Then, these partial patterns are determined, and a table pattern included in the image is extracted.

Since the size of the partial pattern obtained by labeling is required later, the connection pattern extraction unit 13
The coordinates of the corners of the circumscribed rectangle approximating the partial pattern are calculated during the labeling process. Then, among the extracted partial patterns, a pattern having a certain size or more is extracted as a table candidate.

When a large connection pattern is extracted, patterns that are inside the large connection pattern and have a certain size or more are also extracted at the same time, and those large and small patterns are treated as patterns having the same label. In this way, when a pattern that originally represents one table is separated into two or more patterns due to blurring, it becomes possible to process these as patterns belonging to the same label. Hereinafter, an allowable blur length will be calculated based on the size of the target pattern.

FIG. 5 shows an example of the labeling process as described above. In the image as shown in FIG. 5A, the connection pattern extraction unit 13 first extracts the pattern 51 having the largest circumscribed rectangle, and adds a label to it. Next, since the horizontal line pattern 56 between the pattern 52 and the pattern 57 has a certain width or more, it is regarded as a part of the large pattern 51, and the following processing is performed as shown in FIG. Now treat these as patterns with the same label. Therefore, the pattern 56 includes the pattern 51
The same label as is added.

Next, in the process P1, the ruled line extracting apparatus extracts the data by the connection pattern extracting unit 13 by using the method of "Title extracting apparatus and method from document image" (Japanese Patent Application No. 07-341983). A histogram representing the frequency distribution of the heights of the obtained rectangles is obtained. And the mode value most freq height is obtained, and this is estimated as the size of a standard character included in the document. The obtained mode value is used for setting various threshold values in the following processing. This mode calculation process is performed as follows.

First, a rectangle height histogram as shown in FIG. 6 is created from a set of circumscribed rectangles as a result of labeling. In FIG. 6, the horizontal axis represents the height of each circumscribed rectangle, and the vertical axis represents the number (frequency value) of rectangles having that height. Here, the height of the circumscribed rectangle is determined using the height of one pixel as a unit height.

Next, the correspondence between the frequency value and the maximum height among the rectangular heights having the frequency value is determined, and is stored in the memory 32 as a rectangular height table. Then, the table is examined in order from the frequency value 0, and when the change in the frequency value is continuous within 1 and the total of the change in the frequency value is equal to or more than the predetermined value, , The highest of these consecutive heights is the mode value of the rectangular height mo
st freq height.

FIG. 7 shows a histogram representing the contents of the rectangular height table corresponding to the histogram of FIG. In FIG. 7, it can be seen that the height at which the frequency value changes rapidly is the mode value. By obtaining the mode value in this manner, the influence of noise smaller than one character can be eliminated, and the height of a character having a standard size included in the document can be obtained.

FIG. 8 shows a simple example of a rectangular height table. In FIG. 8, four frequency values and the maximum height among rectangular heights having each frequency value are stored in pairs. FIG. 9 shows a histogram of the contents of the rectangular height table in order to obtain the mode.

When the histogram of FIG. 9 is viewed in order from a low frequency value, that is, from a high height value,
Heights 10, 9, and 8 with frequency values of 5,
It can be seen that only 5 and 7 have changed. The difference between these successive heights is 1, and the total change in frequency value is 17. Assuming that the threshold value of the change in the frequency value is 9, the height is 10,
Since the sum of the changes of the frequency values is larger in 9 and 8, the height 10 first appearing in them is mos
t freq height.

Next, the mask processing unit 14 extracts the connection pattern having a certain size or more extracted by the connection pattern extraction unit 13 and the connection pattern having a certain size or more existing inside (the former). Is a table candidate). Then, a mask process is performed by using the method of “frame extraction device and rectangle extraction device” of the prior application (Japanese Patent Application No. 7-203259).

This mask processing is performed in order to eliminate an extremely oblique component from the image and to easily extract a long straight line existing only in the table. Specifically, the mask processing unit 14
For the entire image, a fixed horizontal and vertical
Scanning is performed using different types of mask rectangles. Calculate the proportion of the pattern of black pixels in the mask, and if it is above a certain value, fill the entire area in the mask with black pixels and leave it as a pattern. Delete the pattern in. Thus, the vertical and horizontal components are extracted.

Here, when a plurality of rows or columns continue and the above-mentioned ratio reaches a predetermined threshold value or more, the continuous patterns are put together to form a large rectangular range, and the center line is defined as the processing result. I do. Thereby, a thick pattern is thinned. Further, in order to prevent a gap between the line segment patterns of the processing result from being opened, the application range of the mask is set so as to overlap each other. The original image before the mask processing is stored separately from the image after the mask processing.

FIG. 10 shows the result of masking the two patterns 51 and 56 shown in FIG. 5B. In this case, the image before processing shown in FIG. 10A is converted as shown in FIG. FIG.
In (b), only the vertical and horizontal components of the patterns 51 and 56 are extracted, and the entire pattern is thinned. Further, as a result of supplementing some patterns in the mask processing, it can be seen that the patterns 51 and 56 are connected.

Next, in process P2, the horizontal straight line extraction unit 15 calculates a projection value for the mask processing image of the partial pattern as shown in FIG. 10B by using the conventional adjacent projection method. Then, in the process P3, a horizontal line segment or a part of the horizontal straight line having a certain length is approximated by a rectangle as shown in FIG. 11 and detected. In these processes, a mask processed image is used, but in the following processes, an original image is used.

Next, in process P4, the horizontal straight line extraction unit 15 checks the connection state between the line segments, and extracts straight lines formed by some line segments as ruled line candidates. In this process, as shown in FIG. 12, neighboring rectangular line segments among the detected rectangular line segments are integrated to detect a long straight line, and the detected straight line is approximated by its circumscribed rectangle.

When integrating the line segments, as shown in FIG.
If there is a certain or more thick line segment (wild card) 61 on the way, it is ignored, and integration is performed only on the thin line segment excluding the thick line segment. Whether or not a certain line segment is a wild card is distinguished by the attribute information described in the line segment information 45 of FIG. As described above, when there is a large line segment, a straight line including the large line segment is not output, but an integration process is performed so that the large line segment is not reflected on the straight line.

As a result, even if there is a pattern such as a shaded portion or a character or a figure which is in contact with the ruled line, it is possible to extract only the ruled line excluding the pattern. The details of such integration processing will be described later. As shown in FIG. 4, the information on the straight line extracted as a result of the integration includes information on each line segment constituting the straight line.

Next, in process P5, the horizontal straight line extraction unit 15 determines the right and left ends of the horizontal straight line approximated by a rectangle as shown in FIG. Perform a search. If there is a portion (blank) without pixels in the traveling direction during the search, the search is continued in the traveling direction on the assumption that there are pixels up to a certain number of pixels. If a blank exists beyond a certain number of pixels, the search is terminated with that position as an end point. In the following processing, the end point detected by the search is used as the end point of the straight line.

Next, in the process P6, the ruled line extraction device executes the “frame extraction device and rectangle extraction device” (Japanese Patent Application No.
−203259), the horizontal dotted line is extracted.
In this process, partial patterns having a certain size and regularly arranged are searched for, and the positions of the dotted lines are represented by surrounding these patterns with rectangles.

In the following processing, the rectangle indicating the position of the dotted line is treated in the same manner as the rectangle indicating the position of the already detected straight line. However, the attribute information of the straight line corresponding to the dotted line is set with the dashed line attribute, and the attribute information of the straight line corresponding to the solid line is set with the solid line attribute, so that the two are distinguished.

The processing P7, P
8, P9, and P10, and the vertical dotted line detection processing P11 are the same as those of the horizontal straight line and the horizontal dotted line described above. Next, in process P12, the ruled line extraction device calculates a representative value of the horizontal straight line height. In this process, a histogram of the height of the extracted horizontal straight line is created, and the most frequent height is set as the mode value mfheight of the horizontal straight line height,
This is used as a representative value of the horizontal straight line height. Similarly, in process P13, a representative value (mode value) mfwidth of the vertical straight line width is calculated.

These representative values are used in the following processing.
Used to set various thresholds. As a representative value of the horizontal straight line height / the vertical straight line width, an appropriate value other than the above-described mode may be used instead.

Next, in process P14, the ruled line extracting device calculates a representative value of the horizontal line segment height. In this processing, a histogram of the heights of all the horizontal line segments constituting all the horizontal straight lines in the image is created, and the most frequent height is determined as the mode value mfheight of the horizontal line segment height. small
This is used as a representative value of the horizontal line segment height. Also,
Similarly, in process P15, the mode mfwidth of the width of all the vertical line segments constituting all the vertical straight lines
Calculate small.

These representative values are also used for setting various threshold values in the following processing. Also,
As a representative value of the horizontal line segment height / the vertical line segment width, an appropriate value other than the above-described mode may be used instead.

Next, in the process P16, the ruled line extracting device integrates completely overlapping straight lines. In this processing, when one straight line almost completely overlaps another straight line, they are integrated into one. For example, in two overlapping straight lines 62 and 63 as shown in FIG. 15, the height H1 of the non-overlapping portion is a fixed allowable pixel number INTEGDO.
If they are within T, they are regarded as completely overlapping and integrated into one straight rectangle 64. INTEGDOT
Is set to 2, for example.

Next, in process P17, the ruled line extraction device removes unnecessary straight lines based on the shape of the straight rectangles and the distance between the adjacent straight rectangles. Regarding the processing of the horizontal straight line,
First, the height threshold th hei and length threshold t
h len is calculated by the following equations. th hei = most freq height * 2/3 (1) th len = most freq height * 5 (2) Then, in the following four cases, the corresponding horizontal straight line is deleted. (A) Aspect ratio (height / width) of horizontal straight rectangle is threshold value FT
H (= 0.11) or more. (B) When the aspect ratio of the horizontal straight rectangle is equal to the threshold value FTH2 (= 0.
04) is greater than or equal to, and the height of the horizontal straight rectangle is the height threshold th hei or more. (C) Width th of horizontal straight rectangle is threshold th of length If less than len. (D) The width of the nearest horizontal line below the current horizontal line is the length threshold th If less than len.

For example, in FIG. 16, the straight line 67 is deleted because it corresponds to the case (a), the straight line 68 is deleted because it corresponds to the case (b), and the straight line 69 is corresponding to the case (c). To be deleted. Also, a straight line 6
If 5 is a current horizontal straight line, the straight line 66 is deleted because it corresponds to the case of (d).

Further, the distance between two adjacent straight lines is
most freq If it is smaller than the threshold value obtained from the height, one of the straight lines (for example, the shorter one) is removed. The same applies to the processing of the vertical straight line.

Next, in process P18, the ruled line extraction device removes some unnecessary straight lines based on the connection relationship between the vertical and horizontal straight lines. Regarding the processing of the horizontal straight line, first, when the end point of the horizontal straight line rectangle of interest does not touch any vertical straight line rectangle, the horizontal straight line rectangle is deleted. This removes straight lines that do not form a table frame, for example, isolated straight lines extracted from crushed and connected character strings.

For example, in FIG.
Since the two end points 74 and 75 are in contact with the vertical straight lines 71 and 72, the straight line 70 is not deleted, and the horizontal straight line 73 is deleted because it does not touch any of the vertical straight lines. The same applies to the processing of the vertical straight line.

However, in such a removal process, a straight line that should be left may be removed due to a character pattern of an image or the like. For example, in the image as shown in FIG. 18, the end points 77 and 78 of the vertical straight line 76 are not in contact with any horizontal straight line, so the vertical straight line 76 is deleted even though it constitutes a table frame. I will.

Therefore, in order to leave a necessary straight line, the object of the removal processing is limited to two straight lines that are close to each other. In this case, first, the rectangle is located below the current horizontal straight line rectangle i, and the distance from the rectangle i is the mode value m of the rectangle height.
ost freq A horizontal straight-line rectangle j smaller than height is obtained.

For each of these horizontal straight-line rectangles i and j, if the end points of the rectangles do not touch any of the vertical straight-line rectangles, and if the end points of any of the vertical straight-line rectangles do not touch the rectangles, Delete the horizontal straight rectangle. The same applies to the processing of the vertical straight line. If such a removal process is performed, the vertical straight line 76 in FIG. 18 does not become a processing target, and thus remains without being deleted.

In FIG. 19, first, adjacent horizontal straight lines 79 and 80 are targeted.
Since the end points 9 and 80 are in contact with the vertical straight line 82, the horizontal straight lines 79 and 80 are not deleted. Next, a horizontal straight line 81 adjacent to the horizontal straight line 80 is targeted, and this horizontal straight line 81 is deleted because it is not in contact with any vertical straight line. Further, of the adjacent vertical straight lines 83 and 84, the vertical straight line 83 is left because the end point is in contact with the horizontal straight line 79, and the vertical straight line 84 is deleted since it is not in contact with any horizontal straight line.

Next, in the process P19, the ruled line extracting apparatus integrates partially overlapping straight lines. In this process, the horizontal straight lines are first sorted in ascending order of width (length). Next, one or more straight lines partially overlapping the straight line i are obtained by focusing on one straight line i in order of the length.

When a partially overlapping straight line is virtually merged with the straight line i to form one straight rectangle, if the height is equal to or less than a threshold value (mfheight + THDOT), the merging is executed. . The value of THDOT is set to 2, for example. Accordingly, when the result of integrating the straight lines is about the most frequent value mfheight of the heights, they are integrated into one. The same applies to the processing of the vertical straight line.

In FIG. 20, the straight line 85 and the straight line 86 partially overlap, and the straight line 86 and the straight line 87 partially overlap. Among these, when the straight line 85 and the straight line 86 are integrated, the height of the integrated rectangle exceeds the threshold value. Therefore, only the straight line 86 and the straight line 87 are integrated and the circumscribed rectangle 8
Approximate by 8. The same applies to the processing of the vertical straight line.

Here, such straight line integration processing is performed. In some cases, a plurality of line segment rectangles originally extracted from a single straight line pattern may not be completely integrated only by the processes P4 and P9. Because. For example, in the case of two overlapping horizontal straight lines 89 and 90 as shown in FIG. 21, the line segment rectangles (hatched portions) included in the two are separated from each other, and therefore, in the process P4, the straight line 89 and the straight line 90 are not integrated. . Even in such a case, both can be integrated according to the process P19.

Next, in the process P20, the ruled line extracting apparatus removes the shorter one of the two lines that almost completely overlap. In this process, regarding the horizontal straight line, first, one horizontal straight line rectangle is sorted in the order of the length. Next, one horizontal straight-line rectangle i is extracted one by one in the longest order, and attention is paid to another horizontal straight-line rectangle j that partially overlaps the horizontal straight-line rectangle i. At this time, a line having a vertical relationship between the straight line i and the straight line j as shown in FIG. In FIG.
If the straight line 91 is a straight line i, the straight lines 92 and 93 correspond to the straight line j.

Next, a range where the line i and the line j overlap in the horizontal direction is determined, and the ratio len3 / len2 of the length len3 and the width len2 of the line j is determined by the threshold OVER.
When it is larger than RATE, the following processing is performed with the straight line j as a deletion candidate. However, the value of OVERRATE is 1
Shall be smaller.

First, the vertical distance between the straight line i and the straight line j is determined. In this case, the difference between the coordinate values of the straight line i and the straight line j may be used as the distance. In consideration of the case where the straight line is inclined, the distance between the small line segment rectangles in these straight lines is defined as the distance between the straight lines. I will do it. FIG. 2 shows how to obtain such a distance value.
3 will be described.

In FIG. 23, if the horizontal direction is the x-axis and the vertical direction is the y-axis, the straight line i and the straight line j are x = xmink
A section of length len3 from the position of ukan to the position of x = xmaxkukan overlaps in the horizontal direction, and each has a plurality of line segments inside.

First, the ruled line extracting device extracts, from the set of line segments in the straight line i, those line segments determined to be not wildcards in the line segment integration process P4, which first overlap the overlapping portion, This is used as a reference line segment. Here, the line segment 94 is the reference line segment. Then, the x coordinate (xmin1, xmax1) of the reference line segment 94 is obtained.

Next, with respect to a set of line segments within the straight line j, all the line segments whose x coordinate ranges from xmin1 to xmax1 are extracted. Then, the average value of the distance in the y direction between the extracted line segment and the reference line segment 94 in the straight line i is obtained, and the average value is set as the distance value between the straight line i and the straight line j.

Here, since the corresponding line segment in the straight line j is only the line segment 95, the distance d between the straight line j and the reference line segment 94 is the distance value between the straight line i and the straight line j. By such a calculation method, a correct distance between straight lines can be obtained even when the straight lines are inclined.

The ruled line extracting device calculates the straight line i thus obtained.
It is determined whether or not the straight line j is to be deleted based on the distance between the straight line j and the straight line j. First, as shown by a straight line 92 in FIG.
Are above the straight line i, the straight line j is deleted if their distance is within the threshold OVERDOT. OV
The value of ERDOT is set to 1, for example. When the straight line j is below the straight line i as in the straight line 93, the distance between them is equal to the threshold most. freq height
If it is smaller than t, the straight line j is deleted.

As described above, unnecessary horizontal straight lines are removed by deleting the shorter one of the two horizontal straight lines that almost completely overlap with each other. The same applies to the processing of the vertical straight line. Next, in the process P21, the ruled line extraction device removes a straight line composed of only line segments larger than a certain threshold value. In this processing, the height of all line segments in the horizontal straight line is set to the threshold value (m
feight If the line is larger than (small * 2-1), the line is regarded as a line erroneously extracted from the character string and deleted. The same applies to the processing of the vertical straight line.

Next, the ruled line extracting device performs the processing P22, P23
In the case where the line segment rectangle forming the straight line rectangle is larger than the threshold value, the information of the line segment rectangle is marked, and the sum of the black pixels inside the unmarked line segment rectangle is obtained. At this time, a set of line segment rectangles with no mark is moved in a direction perpendicular to the length direction of the straight rectangle, and a graph representing the relationship between the shift amount and the total number of black pixels is created. Then, when the shape of the graph is gentle, the straight rectangle is excluded from the ruled line candidates.

The ruled line extracting apparatus first performs preprocessing for processing P23 in processing P22. In this process,
The height of the line segment in the horizontal straight line is the threshold value mfheight
If it is larger than small, the attribute information of the line segment is set as a wild card so as not to be used in the process P23. The same applies to the processing of the vertical straight line.

Next, in process P23, a straight line check / removal process by line segment shift is performed. In this process,
First, the sum of the numbers of black pixels in a plurality of horizontal line segment rectangles forming one horizontal straight line rectangle is obtained, and the sum is set as the total number of black pixels at the position of the shift amount 0 (center position). Next, the line segment rectangle is shifted up and down in the direction (vertical direction) perpendicular to the length direction of the horizontal straight line rectangle by the height of the horizontal straight line rectangle, and the total number of black pixels at each position is obtained. Create a graph of the total number of black pixels for.

If the distribution of the total number of black pixels at positions separated in both the upper and lower directions is less than a certain ratio with reference to the maximum value of the total number of black pixels near the center position, the straight-line rectangle is regarded as a ruled line. Output. Otherwise, the straight line rectangle is a straight line that is incorrectly extracted from a character string or the like,
Assuming that it is not a ruled line, delete it. The same applies to the processing of the vertical straight line.

For example, in the case of the image shown in FIG.
A graph as shown in FIG. In FIG. 24, graphs 96 and 97 represent the distribution of black pixels around the corresponding horizontal straight line. At this time, the height of the horizontal straight line is h
Assuming that each of the line segments in the horizontal straight line is +
The pixel is shifted one pixel at a time in the section of-/ height, and the total number of black pixels in the line segment at each shift position is plotted.

In the case of the graph 96, the maximum value Peak of the total number of black pixels is located at the position of the shift amount 0, and the distribution of the total number of black pixels is lower than the value of THISHFT times Peak. Here, the value of THSHIFT is
For example, it is set to 0.44. Such a distribution indicates that black pixels are concentrated at the position of the shift amount 0, and the horizontal straight line corresponding to the position is regarded as a ruled line.

On the other hand, in the case of the graph 97, + /
The distribution of the total number of black pixels in the section of −height is gentle, and Peak T
It does not fall below the value of HSHIFT times. Such a distribution is
This indicates that black pixels are scattered above and below the position of the shift amount 0, and the horizontal straight line corresponding to that position is regarded as not a ruled line and is deleted.

When such a straight line check / removal process is performed, straight lines extracted from the shaded portion in the table or the crushed character string are removed, and only the original ruled lines are output as the processing result. The details of this check / removal process will be described later.

Next, specific examples of each of the above processes will be described with reference to FIGS. FIG.
FIG. 26 shows image data after horizontal line segment integration in process P4, and FIG. 26 shows a part of the image in FIG. FIG.
The state of No. 6 corresponds to the image data before the integration of the completely overlapped straight lines in the process P16, and FIG. 27 shows the image data after the integration.

In FIG. 26, it can be seen that the straight rectangle 101 having the label 66 and the straight rectangle 106 having the label 3 are integrated into one in FIG. Similarly,
A straight rectangle 102 with labels 67, 68, 69, 70,
103, 104, 105 are labels 4, 5,
Straight rectangles 107, 108, 109, 110 having 6, 7
And has been integrated.

FIG. 28 shows image data after processing P17 and P18 have been performed on the image of FIG. 27 and straight lines have been removed based on the shape, position and connection relationship of the straight lines. In FIG. 27, a straight rectangle 1 having labels 35, 37, 38
It can be seen that 11, 112, and 113 are deleted in FIG.

FIG. 29 shows the image data before the integration of the partially overlapping straight lines in the process P19.
Indicates image data after the integration. 29, the straight rectangle 121 having the label 25 and the straight rectangle 124 having the label 20 are integrated into one in FIG. Similarly, straight rectangles 122 and 123 having labels 26 and 27 respectively correspond to labels 21 and
22 are integrated with straight rectangles 125, 126.

FIG. 31 shows the image data before removal of the almost completely overlapping straight line in the process P20, and FIG. 32 shows the image data after the removal.
It can be seen that the straight rectangle 131 in FIG. 31 has been deleted in FIG.

FIG. 33 shows image data before removal of a straight line consisting only of a large line segment in the process P21, and FIG. 34 shows image data after the removal. It can be seen that the straight rectangle 141 in FIG. 33 has been deleted in FIG.

FIG. 35 shows the image data before the check / removal of the straight line by the line segment shift in the process P23, and FIG. 36 shows the image data after the check / removal. The straight rectangles 151 and 1 in FIG.
It can be seen that 52 and 153 have been deleted in FIG.

Next, the line segment integration processing will be described in detail with reference to FIGS. FIG. 37, FIG.
8, 39, 40, and 41 are flowcharts of the horizontal line segment integration process P4 in FIG.

In this processing, the horizontal straight line extraction unit 15
Treats a large block of black pixels as a wildcard rectangle,
Attention is paid to a horizontally long line segment rectangle connected by eight connections before and after that. Then, line segment rectangles having an 8-connected relationship with each other are integrated with the wild card rectangle interposed therebetween, and one horizontally long integrated rectangle is obtained as a horizontal straight line.

When the processing is started, the horizontal straight line extraction unit 15
Is the mode value of the height of the horizontal line, mfheight s
Each threshold value of the following equation is calculated using mall, and the number of straight lines is set to 0 (FIG. 37, step S1). th height = mfheight small + TH HEIGHTDOT (3) standard h = mfheight small + 1 (4) where TH The value of HEIGHTDOT is set to 2, for example. Next, the height of one of the plurality of line segment rectangles connected to each other is checked (step S2), and the height is determined as the threshold value th. If it is larger than height, it is marked as a wildcard rectangle (step S3). At this time, the wildcard attribute is set by setting the identification variable use of the attribute information of the line segment rectangle to 9.

For other line segment rectangles, use = 0 is set as a normal rectangle (standard rectangle) (step S).
4). Then, it is determined whether or not all the line segment rectangles to be connected have been marked (step S5). If the line segment rectangle still remains, the processing from step S2 is repeated.

When all line segment rectangles have been marked, 1
Two rectangles are taken out as a current rectangle i, xlf = the left end coordinates of the current rectangle i, xr = the right end coordinates of the current rectangle i, yup = the top coordinates of the current rectangle i, ybl = the bottom coordinates of the current rectangle i, line start = yu
p, line end = ybl (step S
6). Then, it is checked whether the use of the current rectangle i is 0 or 9 (step S7).

If the use of the current rectangle i is 0 or 9, startxlf = xlf, startxr = x
r, startup = yup, startybl = y
bl (step S8), and it is determined whether use = 0 (FIG. 38, step S9). If use = 0, standard st = yup, stand
ard en = ybl, b use = 0, use = 1,
height = ybl-yup + 1 (step S
10).

B use = 0 indicates that the current rectangle i is
Is set as a standard, not a card
Use = 1 means that the current rectangle i has been used
Means that In step S9, use = 0
If not, standard st = 0, standard
d en = 0, b use = 9, height2 = yb
1-yup + 1 is set (step S11). b use
= 9 because the current rectangle i is a wildcard,
It means that it is not set as a standard.

Next, another line segment rectangle is extracted as the current rectangle k, and rxlf = the left end coordinate of the current rectangle k, r
xr = the right end coordinate of the current rectangle k, ryup = the upper end coordinate of the current rectangle k, and rybl = the lower end coordinate of the current rectangle k (step S12).

Whether the current rectangle i is set as the standard, ie, whether b use =
It is checked whether it is 0 (step S13). b u
If se = 0, then use of the current rectangle k is 9
Is checked (step S14). here,
When use = 9, it means that the current rectangle i is a standard and the current rectangle k is a wild card.

When use = 9, xr + 1 ≧ rxlf,
xr <rxr, ybl + 1 ≧ ryup, and yup−
It is determined whether 1 ≦ rybl is satisfied (step S15). When these conditions hold, the current rectangle k
Is on the right side of the current rectangle i, which means that they overlap one pixel (one dot) horizontally and vertically. Therefore, with xr = rxr, the right end of the current rectangle i is extended to the right end of the current rectangle k (step S1).
6).

If use = 9 is not satisfied in step S14,
Next, it is checked whether or not use = 0 (FIG. 39, step S17). Here, when use = 0, it means that the current rectangle i is a standard and the current rectangle k is not a wild card. Therefore, next, xr + 1
≧ rxlf, xr <rxr, ybl + 1 ≧ ryup, and yup−1 ≦ rybl, and the height of the current rectangle k is standard. It is determined whether it is within h +/− 4 (step S18).

When these conditions hold, xr = rx
r, yup = ryup, ybl = rybl, use =
2, hei = rybl-ryup + 1 is set (step S19). That is, the right end of the current rectangle i is extended to the right end of the current rectangle k, and the coordinates of the upper and lower ends are set to the current rectangle k.
Means to replace it with Use = 2
Means that the current rectangle k has been used.

Next, it is determined whether or not hei> height is satisfied (step S20).
gh = hei (step S21). Next, ry
up <line It is determined whether or not start is satisfied (step S22). star
t = ryup is set (step S23). Furthermore, ry
bl> line It is determined whether or not end is satisfied (step S24). end = r
ybl (step S25).

After these processes, b It is determined whether use = 9 (FIG. 40, step S26). FIG.
Steps S13 and S15, step S18 in FIG.
If the determination result is No in S20, S22, and S24, the process immediately proceeds to step S26 and subsequent steps.

Here, b If use = 9, then
It is checked whether the use of the current rectangle k is 9 (step S27). When use = 9, it means that both the current rectangle i and the current rectangle k are wildcards. Therefore, xr + 1 ≧ rxlf, xr <
rxr, ybl + 1 ≧ ryup, and yup−1 ≦ r
ybl is determined (step S2)
8).

When these conditions are satisfied, the current rectangle k is on the right side of the current rectangle i, and both have an overlap of one or more dots in the horizontal and vertical directions. Therefore, with xr = rxr, the right end of the current rectangle i is The rectangle k is extended to the right end (step S29).

If use = 9 is not satisfied in step S27,
Next, it is checked whether use = 0 (step S
30). Here, if use = 0, the current rectangle i
Is a wildcard, meaning that the current rectangle k is not a wildcard. Therefore, xr + 1 ≧ rxl
f, xr <rxr, line end ≧ ryup, and line It is determined whether start <rybl is satisfied (step S31).

When these conditions are satisfied, xr = rx
r, yup = ryup, ybl = rybl, use =
2, line start = ryup, line en
d = rybl, hei = rybl-ryup + 1, st
andard st = ryup, standard e
n = rybl is set (step S32).

This means that the right end of the current rectangle i is extended to the right end of the current rectangle k, and the coordinates of the upper and lower ends are replaced with those of the current rectangle k. Also, use =
2 means that the current rectangle k has been used.
Then, it is determined whether or not “hei> height” is satisfied (step S33).
ei (step S34).

Next, it is determined whether all the connected line segment rectangles have been extracted as the current rectangle k (FIG. 4).
1. Step S35). Steps S26 and S2 in FIG.
8. If the determination result is No in S30, S31, and S33, the process immediately proceeds to step S35 and thereafter. here,
If there is any remaining line segment rectangle, step S13 in FIG.
The subsequent processing is repeated.

When the processing is completed for all the line segment rectangles, b It is determined whether use = 9 (step S36). Where b If use = 9, xl
f, xr, line start, line end
Are stored as the coordinates of the left end, the right end, the upper end, and the lower end of the extracted straight rectangle, and the number of straight lines is incremented by 1 (step S37).

At step S36, b Use = 9 is when the current rectangle i and all the rectangles connected to it are wildcards. In this case, the current rectangle i is not stored as a straight line.

Next, it is determined whether or not all the line segment rectangles have been extracted as the current rectangle i (step S3).
8) If there are any remaining line segment rectangles, the processing from step S6 in FIG. 37 is repeated. If the use of the current rectangle i is not 0 or 9 in step S7, it means that the extracted line segment rectangle has already been used, so the process immediately proceeds to step S38, and the next line segment rectangle is extracted. Then, when all the line segment rectangles have been extracted, the process ends.

In this way, it is possible to jump over the wild card, integrate the horizontal line segment rectangles on both sides thereof, and extract a horizontal line including only standard line segments. The vertical line segment integration process P9 in FIG. 2 is also performed according to a similar flow. Next, the flow of a straight line check / removal process by line segment shift will be described in detail with reference to FIGS. 42, 43, 44, 45, 46, 47
5 is a flowchart of a process of checking / removing a horizontal straight line in a process P23 of FIG. In this processing, the ruled line extraction device determines whether or not the straight line is a correct ruled line based on the distribution of black pixels around each horizontal straight line, and removes other lines while leaving only the correct ruled line.

When the process is started, the ruled line extraction device firstly stores a variable cnt representing the number of processed horizontal straight lines and a variable newcount representing the number of horizontal straight lines determined to be correct ruled lines.
Both are set to 0 (FIG. 42, step S41). However,
cnt and newcount are counted starting from 0.

Next, it is checked whether or not the attribute of the cnt-th horizontal straight line represents a broken line (step S42). If it is a dashed line attribute, all the straight line information of the horizontal straight line is newc
copy as the rule information of the "out" th, cnt and ne
wcount is incremented by 1 (step S
43), the next horizontal straight line is determined in step S42.

If the attribute of the cnt-th horizontal straight line is not the broken line attribute, it is regarded as corresponding to the solid line, and the height of the horizontal straight line rectangle is set to height (step S4).
4) It is determined whether height is 2 or less (step S45). If height is greater than 2, the value is not changed, and if height is 2 or less, it is changed to 3 (step S46).

Then, pos = height, maxs
um = 0, top bound = 0, bottom b
sound = 0, shift = -1 * height, su
It is assumed that m = 0 and j = 0 (step S47). Variable pos
Is used to specify the position in the vertical direction in the image, and the variable maxsum represents the maximum value in the graph of the total number of black pixels.

The variable top “bound” is used as a flag indicating whether or not the section of the graph has reached the upper end of the image, and a variable “bottom” is used. The bound is used as a flag indicating whether the section of the graph has reached the lower end of the image. The value of the variable shift represents the amount of vertical shift of the line segment rectangle in the horizontal straight line, and the variable sum represents the total number of black pixels in a plurality of line segment rectangles.

Next, it is checked whether or not the attribute of the j-th small line segment (line segment rectangle) in the cnt-th horizontal straight line is a wild card (FIG. 43, step S48). If it is the wild card attribute, j is incremented by 1 (step S49), and the next small line segment is determined in step S48.

If the attribute of the j-th small line segment is not a wild card, it is regarded as corresponding to a normal line segment rectangle,
xmin = the left end coordinate of the jth small line, xmax = the right end coordinate of the jth small line, ymin = the upper end coordinate of the jth small line + shift, and ymax = the lower end coordinate of the jth small line + shift. (Step S50). However,
If the position of ymin or ymax determined in this manner is not within the image coordinates, the y coordinate of the nearest image end is set to ym.
in or ymax.

The variables xmin, xmax, ymi
Calculate the number of black pixels in the rectangular area delimited by each value of n and ymax, and Dot and su
m = sum + black dot.

Next, it is determined whether or not ymin corresponds to the upper end of the image (step S51).
p Bound = 1 is set (step S52). Also,
It is determined whether or not ymax corresponds to the lower end of the image (step S53). boun
d = 1 is set (step S54).

Next, it is determined whether or not the value of the variable j corresponds to the last small line segment in the cnt-th horizontal straight line (step S55). If it does not correspond to the last line segment, j
Is incremented by 1 (step S56), and the processing from step S48 is repeated. If it corresponds to the last small line segment, histo [pos + shift] = s
um (FIG. 44, step S57).

Here, histo [p] represents the value of the graph (total number of black pixels) corresponding to the value of the position parameter p in the y direction. For example, pos = height, shif
If t = -height, histo [pos + s
shift] = histo [0], and the parameter value 0
Represents the value of the graph corresponding to the position of.

Next, sum and maxsum are compared (step S58), and if sum is larger than maxsum, maxsum = sum is set (step S59).
Next, shift and height are compared (step S60). If shift is equal to or less than height, s
Assuming that shift = shift + 1, sum and j are returned to the initial values 0 (step S61), and the processing from step S48 is repeated. Thereby, the value of the graph corresponding to the position where the shift amount has changed by one pixel is calculated.

When the value of shift exceeds height, next, histo [pos-1] and histogram
o [pos] is compared (step S62). Here, histo [pos] represents the value of the graph corresponding to the position of the shift amount 0. histo [pos-1] is h
If it is greater than isto [pos], then histo
[Pos-1] is compared with histo [pos + 1] (step S63).

Then, histo [pos-1] becomes hi
If greater than sto [pos + 1], maxpos =
pos-1, maxsum = histo [pos-1]
(Step S64). In addition, histo [pos
-1] is less than histo [pos + 1], ma
xpos = pos + 1, maxsum = histo [p
os + 1] (step S65).

In step S62, histo [p
If os-1] is less than or equal to histo [pos], then histo [pos] and histo [pos + 1]
Are compared (step S66).

Then, histo [pos] is changed to "hist".
If o [pos + 1] or more, maxpos = po
s, maxsum = histo [pos] are set (step S67). Also, histo [pos] is histo
If less than o [pos + 1], then maxpos = po
s + 1, maxsum = histo [pos + 1] are set (step S68).

By such processing, histo [po]
s-1], histo [pos], and histo
The maximum value of [pos + 1] is set to maxsum, and the corresponding parameter value is set to maxpos. Accordingly, this maxsum represents the maximum value of the graph near the position of the shift amount 0, and corresponds to Peak in FIG.

Next, the obtained maxsum (= hist)
Using the value of [o [maxpos]), the threshold th is calculated by the following equation (FIG. 45, step S69). th = maxsum * THSHIFT (5) where the parameter THSHIFT is the maximum value maxs
It represents the ratio of the threshold value th to um, and is appropriately set between 0 and 1.0.

Then, lineokflag1 = 0, l
ineokflag2 = 0, minval = 0xffff
f, shift = −1 * height. Variable li
neokflag1 and lineokflag2 are
The variable minval represents a minimum value of the graph, which is used as a flag indicating a check result on both upper and lower sides of the position corresponding to the local maximum value.

Next, histo [pos + shift]
And th (Step S70). And his
If to [pos + shift] is smaller than th, it is considered that the number of black pixels has decreased above the position corresponding to the local maximum, and lineokflag1 = 1 is set (step S71). Therefore, lineokflag1 =
1 indicates that the check result on the upper side is OK.

Next, histo [pos + shift]
And minval are compared (step S72). And histo [pos + shift] is minval
If smaller, minval = histo [pos +
shift] (step S73).

Next, (pos + shift) and maxp
os (step S74), and (pos + sif
If t) is less than or equal to maxpos, then shift = shi
ft + 1 (step S75), step S70
The subsequent processing is repeated. Thus, the value of the graph is checked again at the position where the shift amount has changed by one pixel.

If (pos + shift) exceeds maxpos, then histo [pos + shift] is compared with th (FIG. 46, step S76). And
If histo [pos + shift] is greater than or equal to th, then histo [pos + shift] and min
val is compared (step S77). And his
If to [pos + shift] is smaller than minval, then minval = histo [pos + shift]
t] (step S78).

Next, shift and height are compared (step S79). If shift is equal to or less than height, shift = shift + 1 is set (step S80), and the processing from step S76 is repeated. Thus, the value of the graph is checked again at the position where the shift amount has changed by one pixel.

In step S76, histo [p
If [os + shift] is smaller than th, it is assumed that the number of black pixels is reduced below the position corresponding to the local maximum, and lineokflag2 = 1 is set (step S8).
1). Therefore, lineokflag2 = 1 indicates that the check result on the lower side is OK.

Next, lineokflag1 and top
The value of bound is checked (FIG. 47, step S82).
And lineokflag1 = 0 and top bo
If und = 1, the target cnt-th horizontal straight line is considered to correspond to the horizontal ruled line near the upper end of the image, and li
neoflag1 = 1 is set (step S83).

Next, lineokflag2 and bottom
om The value of bound is checked (step S84). Then, lineokflag2 = 0 and bottom
If bound = 1, it is assumed that the cnt-th horizontal straight line corresponds to the horizontal ruled line near the lower end of the image, and lineo
kflag2 = 1 is set (step S85).

Next, lineokflag1 and li
Check the final value of neoflag2 (step S
86). lineokflag1 = lineokfla
If g2 = 1, it is determined that the cnt-th horizontal straight line is a correct horizontal ruled line, the line information is copied as newcount-th ruled line information, and newcount is incremented by 1 (step S87).

If lineokflag1 = 0 or lineokflag2 = 0, it is determined that the cnt-th horizontal straight line is not a correct horizontal ruled line, and the line information is not stored as ruled line information.

Next, it is checked whether or not cnt corresponds to the last horizontal straight line in the image (step S88). If a horizontal straight line still remains, cnt is incremented by 1 (step S89), and the next Step S4 for horizontal straight line
The processing after step 2 is repeated. When cnt becomes a value corresponding to the last horizontal straight line, the processing is terminated.

By performing such a process, the ruled line likelihood is checked based on the comparison between the number of black pixels on a straight line and the number of black pixels around the straight line. The extracted straight line is efficiently removed. The same check / removal process is performed on the vertical straight line.

As described above, according to the present embodiment, regardless of whether the image is blurred or not, even if the image is a table image in which solid lines and dotted lines are mixed, a shaded portion, a character and a ruled line are not included. Even if there is a contact portion, a ruled line can be accurately extracted.

The ruled line extraction technique of the present invention is applied not only to electronic filing but also to a technique of cutting out only a graphic from a pattern in which a straight line overlaps a graphic in a broad sense including symbols and characters. For example, the present invention can be applied to a handwritten character recognition device, a printed character recognition device, cutout of patterns such as characters and symbols in drawing recognition, and separation of a ruled line in an image and a contact portion between an object, a figure, and a character.

The shape of the area representing a straight line or a line segment is not necessarily rectangular, and an area of any shape surrounded by a straight line or a curve can be used. in this case,
If the position, height, and width of a straight line or a line segment, the distance between them, and the like are appropriately defined according to the shape to be used, processing similar to that in FIG.

[0183]

According to the present invention, a ruled line portion can be correctly extracted from a general document image whose ruled line structure cannot be predicted in advance.

For example, even when a table has a shaded portion, it is possible to correctly extract ruled lines from the image. Also, when a character and a ruled line are in contact with each other, it is possible to correctly extract the ruled line even if the table structure information is not known in advance. Furthermore, even when the character string is printed with a slight collapse or when characters are printed at a high density, the probability of erroneously extracting ruled lines from the character string is reduced.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of a ruled line extraction device according to the present invention.

FIG. 2 is a functional block diagram of the ruled line extraction device.

FIG. 3 is a configuration diagram of an information processing apparatus.

FIG. 4 is a diagram showing a data structure.

FIG. 5 is a diagram showing a labeling process.

FIG. 6 is a diagram showing a height histogram.

FIG. 7 is a diagram showing a histogram for obtaining a mode value of height.

FIG. 8 is a diagram showing a rectangular height table.

FIG. 9 is a diagram showing a histogram corresponding to the contents of a rectangular height table.

FIG. 10 is a diagram showing a mask process.

FIG. 11 is a diagram showing a line segment detection process.

FIG. 12 is a diagram showing a first line segment integration process.

FIG. 13 is a diagram showing a second line segment integration process.

FIG. 14 is a diagram showing a straight line search process.

FIG. 15 is a diagram showing a process of integrating completely overlapped straight lines.

FIG. 16 is a diagram illustrating a first straight line removal process.

FIG. 17 is a diagram illustrating a second straight line removal process.

FIG. 18 illustrates a straight line that should not be removed.

FIG. 19 is a diagram illustrating a third straight line removal process.

FIG. 20 is a diagram showing a process of integrating partially overlapping straight lines.

FIG. 21 is a diagram showing the inside of a partially overlapping straight line.

FIG. 22 is a diagram illustrating a fourth straight line removal process.

FIG. 23 is a diagram showing how to obtain a distance value between two straight lines.

FIG. 24 is a diagram illustrating a fifth straight line removal process.

FIG. 25 is a diagram showing an image after horizontal line integration processing;

FIG. 26 is a diagram showing an image of a completely overlapping straight line before integration processing.

FIG. 27 is a diagram showing an image after a completely overlapping straight line is integrated.

FIG. 28 is a diagram showing an image after removal processing based on the connection relationship between the shape and position of the straight line and the vertical and horizontal straight lines.

FIG. 29 is a diagram showing an image before a partial overlapping straight line integration process.

FIG. 30 is a diagram showing an image after a process of integrating partially overlapping straight lines;

FIG. 31 is a diagram showing an image before a straight line that is almost completely overlapped is removed.

FIG. 32 is a diagram showing an image after a process of removing almost completely overlapping straight lines.

FIG. 33 is a diagram showing an image before a process of removing a straight line composed of only large line segments;

FIG. 34 is a diagram showing an image after the removal processing of a straight line consisting only of a large line segment.

FIG. 35 is a diagram showing an image before straight line check / removal processing by line segment shift.

FIG. 36 is a diagram showing an image after a straight line check / removal process by line segment shift.

FIG. 37 is a flowchart (part 1) of a line segment integration process.

FIG. 38 is a flowchart (part 2) of a line segment integration process.

FIG. 39 is a flowchart (part 3) of the line segment integration process.

FIG. 40 is a flowchart (part 4) of a line segment integration process.

FIG. 41 is a flowchart (part 5) of the line segment integration process.

FIG. 42 is a flowchart (part 1) of a straight line check / removal process;

FIG. 43 is a flowchart (part 2) of a straight line check / removal process;

FIG. 44 is a flowchart (part 3) of a straight line check / removal process;

FIG. 45 is a flowchart (part 4) of a straight line check / removal process;

FIG. 46 is a flowchart (part 5) of a straight line check / removal process;

FIG. 47 is a flowchart (part 6) of a straight line check / removal process;

FIG. 48 is a diagram showing a simple tabular frame.

FIG. 49 is a diagram showing a complicated tabular frame.

FIG. 50 is a diagram showing a tabular frame in which it is difficult to extract ruled lines.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Estimation means 2 Storage means 3 Line extraction means 4 Calculation means 5 Straight line extraction means 6 Graph generation means 7 Straight line processing means 8 Straight line integration means 9 Straight line removal means 11 Input pattern 12 Reduction processing unit 13 Connection pattern extraction unit 14 Mask processing unit Reference Signs List 15 horizontal straight line extraction unit 16 vertical straight line extraction unit 31 CPU 32 memory 33 input device 34 output device 35 external storage device 36 medium drive device 37 network connection device 38 photoelectric conversion device 39 bus 40 portable recording medium 40 'database 41 image information 42 Table information 43 Cell information 44 Straight line information 45 Line segment information 51, 52, 53, 54, 55, 56, 57, 58 Pattern 61 Wild card 62, 63, 64, 65, 66, 67, 68, 69, 7
0, 71, 72, 73, 76, 79, 80, 81, 8
2, 83, 84, 85, 86, 87, 88, 89, 9
0, 91, 92, 93, 101, 102, 103, 10
4, 105, 106, 107, 108, 109, 11
0, 111, 112, 113, 121, 122, 12
3, 124, 125, 126, 131, 141, 15
1, 152, 153 Straight rectangles 74, 75, 77, 78 Endpoints 94, 95 Segment rectangles 96, 97 Graph

Claims (44)

[Claims] An estimating means for estimating a size of a standard pattern included in an input image; a threshold value being set based on information on the size of the standard pattern; A line extracting unit for extracting information of one or more linear patterns from an image. 2. The ruled line extraction device according to claim 1, wherein the estimating unit estimates a size of a character pattern included in the input image as a size of the standard pattern. 3. The method according to claim 1, wherein the estimating unit estimates the size of the standard pattern from a histogram of the size of the pixel connection region included in the input image.
Ruled line extraction device as described. 4. The information processing apparatus according to claim 1, wherein said straight line extracting means includes information on one of said one or more straight line patterns and information on a distance between two straight line patterns included in said one or more straight line patterns. And a straight line removing unit that determines whether to remove one of the one or more straight line patterns using at least one of the information and the threshold value. Ruled line extraction device as described. 5. The straight line extracting means, based on a connection relationship between a horizontal straight line pattern and a vertical straight line pattern included in the one or more straight line patterns and the threshold value, of the horizontal straight line pattern and the vertical straight line pattern. 2. The ruled line extracting device according to claim 1, further comprising a straight line removing unit that determines whether to remove one of the ruled lines. 6. The straight line extracting means determines, using the threshold value, whether to remove a shorter one of two almost overlapping straight line patterns included in the one or more straight line patterns. 2. The ruled line extracting apparatus according to claim 1, further comprising a straight line removing means for performing the rule. 7. A straight line extracting means for extracting information of one or more straight line patterns from an input image; a calculating means for calculating a representative value of the size of the one or more straight line patterns; A rule processing unit for setting a threshold value and processing the information of the one or more linear patterns using the threshold value. 8. The method according to claim 1, wherein the calculating unit calculates the representative value from one of a histogram of the height of the horizontal linear pattern and a histogram of the width of the vertical linear pattern included in the one or more linear patterns. The ruled line extraction device according to claim 7, wherein 9. The method according to claim 1, wherein the linear processing unit is configured such that when a linear pattern formed by integrating two partially overlapping linear patterns of the one or more linear patterns is approximately the representative value, 8. The ruled line extracting device according to claim 7, further comprising a straight line integrating unit that uses the straight line pattern after integration as a ruled line candidate. 10. A straight line extracting means for extracting information of one or more straight line patterns from an input image, and a calculation for obtaining a representative value of a size of one or more line segment patterns constituting said one or more straight line patterns. Means for setting a threshold value based on the representative value, and using the threshold value to process the information of the one or more linear patterns. . 11. The calculating means obtains the representative value from one of a histogram of a height of a horizontal line segment pattern and a histogram of a width of a vertical line pattern included in the one or more line segment patterns. The ruled line extraction device according to claim 10, wherein: 12. The straight line processing unit includes a straight line removing unit that removes a straight line pattern formed of only line segment patterns larger than the threshold value from the one or more straight line patterns. 11. The ruled line extraction device according to claim 10. 13. A line segment extraction unit for extracting information of one or more line segment patterns from an input image, and combining the information of the one or more line segment patterns with information of a large line segment pattern and a small line segment pattern. A storage unit for storing information separately from the information; and checking a connection state of the one or more line segment patterns. When the large line segment pattern exists in the middle, regardless of the size of the large line segment pattern, And a straight line extracting means for extracting a straight line pattern formed by the plurality of line segment patterns. 14. A straight line extracting means for extracting information of one or more straight line patterns from an input image, and wherein the two straight line patterns included in the one or more straight line patterns substantially overlap each other. And a straight line integrating means for integrating the ruled lines into one. 15. A straight line extracting means for extracting information of one or more straight line patterns from an input image, information on a shape of one of the one or more straight line patterns, and information included in the one or more straight line patterns. Line removing means for determining whether to remove one of the one or more linear patterns by using at least one of the information on the distance between the two linear patterns to be removed. Ruled line extraction device. 16. A straight line extracting means for extracting information of one or more straight line patterns from an input image; A ruled line extracting apparatus comprising: a straight line removing unit that determines whether to remove one of a pattern and a vertical straight line pattern. 17. A straight line extracting means for extracting information of one or more straight line patterns from an input image, and removing a shorter one of two almost straight line patterns included in the one or more straight line patterns. A ruler line extracting device, comprising: 18. A straight line extracting means for extracting information of one or more straight line patterns from an input image, wherein the one or more straight line patterns partially overlap each other.
A ruled line extracting device, comprising: a line integrating unit that sets the line pattern after integration to a ruled line candidate when the size of a linear pattern formed by integrating two linear patterns is approximately a predetermined value. 19. A straight line extracting means for extracting information of one or more straight line patterns from an input image, and a straight line pattern comprising only a line segment pattern larger than a threshold value among the one or more straight line patterns. And a straight line removing unit for removing a ruled line. 20. A straight line extracting means for extracting information on a straight line pattern from an input image, and the number of pixels inside a standard size line segment pattern among one or more line segment patterns constituting the straight line pattern. And a graph generation means for generating a graph of the number of pixels around the straight line pattern, and a straight line removing means for determining whether to remove the straight line pattern based on the shape of the graph. Ruled line extraction device. 21. The apparatus according to claim 21, further comprising a storage unit for marking information on a large line segment pattern among the one or more line segment patterns and storing the information on the one or more line segment patterns. 21. The ruled line extracting apparatus according to claim 20, wherein the means regards a line segment pattern having no mark among the one or more line segment patterns as the standard-sized line segment pattern. 22. The graph generating means generates the graph representing the relationship between the shift amount and the number of pixels by shifting the standard size line segment pattern perpendicular to the length direction of the linear pattern. 21. The ruled line extraction device according to claim 20, wherein the straight line removing unit removes the straight line pattern when the shape of the graph is gentle. 23. A recording medium recording a program for a computer, comprising: a function of estimating the size of a standard pattern included in an input image; and a threshold value based on information on the size of the standard pattern. A computer-readable recording medium recording a program for causing the computer to realize a function of setting and a function of extracting information of one or more linear patterns from the input image using the threshold value. 24. A recording medium on which a program for a computer is recorded, wherein a function of extracting information of one or more linear patterns from an input image, and a representative value of a size of the one or more linear patterns is provided. A program for causing the computer to realize a function to determine, a function to set a threshold value based on the representative value, and a function to process information of the one or more linear patterns using the threshold value A computer-readable recording medium on which is recorded. 25. A recording medium recording a program for a computer, comprising: a function of extracting information of one or more linear patterns from an input image; A function of obtaining a representative value of the size of the line segment pattern; a function of setting a threshold based on the representative value; and a function of processing information of the one or more linear patterns using the threshold. And a computer-readable recording medium recording a program for causing the computer to realize the above. 26. A recording medium recording a program for a computer, comprising: a function of extracting information of one or more line segment patterns from an input image; A function for distinguishing between line segment pattern information and small line segment pattern information; and checking a connection state of the one or more line segment patterns, and when the large line segment pattern exists in the middle, the large line segment pattern A computer-readable recording medium storing a program for causing the computer to realize a function of extracting a straight line pattern formed by a plurality of other line segment patterns regardless of the size of the line segment pattern. 27. A recording medium storing a program for a computer, comprising: a function of extracting information of one or more linear patterns from an input image; and two linear patterns included in the one or more linear patterns. A computer-readable recording medium on which a program for causing the computer to realize the function of integrating the two linear patterns into one when almost overlaps is recorded. 28. A recording medium on which a program for a computer is recorded, comprising: a function of extracting information of one or more linear patterns from an input image; and a form relating to one of the one or more linear patterns. Whether one of the one or more linear patterns is removed using at least one of information and information on a distance between two linear patterns included in the one or more linear patterns. A computer-readable recording medium that stores a program for causing the computer to realize the function of determining whether the program is executed. 29. A recording medium storing a program for a computer, comprising: a function of extracting information of one or more linear patterns from an input image; and a horizontal linear pattern included in the one or more linear patterns. A computer-readable storage medium storing a program for causing the computer to realize a function of determining whether to remove one of the horizontal linear pattern and the vertical linear pattern based on a connection relationship between the vertical linear patterns. 30. A recording medium on which a program for a computer is recorded, wherein a function of extracting information of one or more linear patterns from an input image is included in the one or more linear patterns, and substantially overlaps the function. A computer-readable storage medium storing a program for causing the computer to realize a function of removing a shorter one of two linear patterns. 31. A recording medium on which a program for a computer is recorded, wherein a function of extracting information of one or more linear patterns from an input image partially overlaps the one or more linear patterns. 2
When the size of a straight line pattern formed by integrating two straight line patterns is about a predetermined value, a computer-readable program that records a program for causing the computer to realize the function of using the straight line pattern after integration as ruled line candidates is provided. recoding media. 32. A recording medium on which a program for a computer is recorded, comprising: a function of extracting information of one or more linear patterns from an input image; A computer-readable storage medium storing a program for causing the computer to realize a function of removing a straight line pattern composed only of a large line segment pattern. 33. A recording medium recording a program for a computer, comprising: a function of extracting information of a linear pattern from an input image; and a standard one or more of line segment patterns constituting the linear pattern. A function of obtaining the number of pixels inside a line segment pattern having a large size and generating a graph of the number of pixels around the linear pattern; and determining whether to remove the linear pattern based on the shape of the graph. And a computer-readable recording medium on which a program for causing the computer to perform the functions described above is recorded. 34. Estimating the size of a standard pattern included in an input image, setting a threshold value based on the information on the size of the standard pattern, and using the threshold value to calculate 1 from the input image. A method for extracting ruled lines, comprising extracting information of one or more linear patterns. 35. extracting information of one or more linear patterns from an input image, obtaining a representative value of the size of the one or more linear patterns, setting a threshold based on the representative value, A method for extracting ruled lines, comprising processing information of the one or more linear patterns using a threshold value. 36. Extracting information of one or more linear patterns from an input image, obtaining a representative value of one or more line segment patterns constituting the one or more linear patterns, Setting a threshold value based on the threshold value, and processing the information of the one or more linear patterns using the threshold value. 37. Extracting information of one or more line segment patterns from an input image, distinguishing the information of the one or more line segment patterns into information of a large line segment pattern and information of a small line segment pattern. Checking the connection state of the one or more line segment patterns and, when the large line segment pattern is present in the middle, regardless of the size of the large line segment pattern, a straight line formed by the other plurality of line segment patterns; A method for extracting a ruled line, comprising extracting a pattern. 38. Extracting information of one or more linear patterns from an input image, and, when two linear patterns included in the one or more linear patterns almost overlap, combining the two linear patterns into one A ruled line extraction method characterized by integrating. 39. Extracting information of one or more straight line patterns from an input image, information on one of the one or more straight line patterns, and two straight lines included in the one or more straight line patterns. A method for extracting ruled lines, comprising determining whether to remove one of the one or more linear patterns using at least one of information on a distance between patterns. 40. Extracting information of one or more linear patterns from an input image, and extracting the horizontal linear pattern and the vertical linear pattern based on a connection relationship between the horizontal linear pattern and the vertical linear pattern included in the one or more linear patterns. A method for extracting a ruled line, comprising determining whether to remove one of the patterns. 41. A method for extracting information of one or more linear patterns from an input image, and removing a shorter one of two linear patterns which are included in the one or more linear patterns and almost overlap with each other. Ruled line extraction method. 42. Extracting information of one or more linear patterns from an input image, and extracting at least one of
A ruled line extraction method characterized in that when the size of a straight line pattern formed by integrating two straight line patterns is about a predetermined value, the straight line pattern after integration is used as a ruled line candidate. 43. Extracting information of one or more linear patterns from an input image, and removing a linear pattern composed of only line segment patterns larger than a threshold value from the one or more linear patterns. A ruled line extraction method characterized by the following. 44. Extracting information of a straight line pattern from an input image, and calculating the number of pixels inside a standard size line segment pattern among one or more line segment patterns constituting the straight line pattern, A ruled line extraction method, comprising: generating a graph of the number of pixels around the linear pattern, and determining whether to remove the linear pattern based on the shape of the graph.