JP3534997B2 - Ruled line extraction device and method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a ruled line extracting apparatus and method for extracting a ruled line portion from an arbitrary image read by a photoelectric conversion device or the like. INDUSTRIAL APPLICABILITY The present invention is not only a handwritten character recognition device, but also a printed character recognition device, a character in drawing recognition, a cutout of a symbol, a device for separating ruled lines and objects in an image, a figure, a contact portion of a character,
It is used when cutting out only a figure from a pattern in which a straight line and a figure in a broad sense overlap.

[0002]

2. Description of the Related Art Recently, an optical character reader (optical ch) has been used as an input peripheral device for financial documents, business documents, etc.
There is an increasing demand for character recognition devices represented by aracter reader (OCR).

In the conventional optical character reading device, a character cutting process for cutting out a character pattern for each character from an input image is performed before performing character recognition. In order to realize a high recognition rate of individual characters in an optical character reading device, it is important to accurately perform character cutting processing, which is a preprocessing for recognition.

Therefore, when a character is read by a conventional optical character reading device, a document such as a form in which the writing position of the character is designated in advance (not a dropout color but a black frame or the like, a ruled line and a character having the same color and It was designed to achieve a high recognition rate by writing characters in the specified range for documents written in density).

However, in the conventional optical character reading device, when a character comes into contact with a ruled line or a frame indicating a designated range or a character protrudes from the ruled line or the frame indicating a designated range,
There is a problem that it is difficult to accurately cut out a character and the recognition rate of the character decreases. For example, when removing the frame, it is not possible to deal with slight inclinations and irregularities of the frame of the character area (character frame), and part of the original character may be missing due to changes in the position and line width of the character frame. It disappears or the frame remains.

[0006] As a conventional technique for dealing with these problems, an "image extraction method" by the present applicant (Japanese Patent Laid-Open No.
309498) and "Image Extraction Device" (Japanese Patent Laid-Open No. 7-289).
37). According to these techniques, it is possible to extract or remove a frame without having to input format information such as the position and size of the frame. The form that can be processed by these is a one-character frame, a block frame (horizontal one-line frame or free format frame), or a table with a rectangular outer shape and a horizontal frame line arranged regularly. I have.

As a ruled line extraction technique of the prior application by the present applicant, "frame extraction device and rectangle extraction device" (Japanese Patent Application No.
-203259), "Pattern region cutting method and pattern extracting device" (Japanese Patent Application No. 7-282171), and "Pattern extracting device and pattern region cutting method"
(Japanese Patent Application No. 8-107568).

According to these techniques, as shown in FIG. 40, even if the tabular frame has a structure in which the outer shape is rectangular and the horizontal frame lines are regularly arranged, FIG. 41. Even if the outer shape of the frame is not rectangular as shown in FIG.
For a tabular frame that has a finer structure (nested structure) in a rectangle surrounded by a frame, like the shaded part of
Frames can be extracted and removed. Furthermore, it is possible to process even when the dotted line and the solid line are mixed or the image has some blur. The outline of the processing is shown below.

(1) Thinning: By masking, the vertical and horizontal patterns are thinned to eliminate the difference in thickness between the character and the frame. (2) Line segment extraction: “image extraction method” (Japanese Patent Laid-Open No. 6-309
498), the adjacent projection method is used to extract a relatively long straight line. Adjacent projection is a method in which the projection values of pixels included in the row or column of interest are added to the projection values of the surrounding rows or columns to obtain the final projection value of the row or column of interest. is there. According to this projection method, the pixel distribution around a specific row or column can be comprehensively captured.

(3) Straight line extraction: The extracted line segments are searched in order, and it is inspected whether there is a break of a certain length or more between the line segments. In the meantime, line segments that do not have such discontinuities are sequentially integrated to extract a long straight line.

(4) Line integration: The extracted lines are integrated again. A straight line separated into two or more parts by blurring is integrated into a single straight line. (5) Straight line extension: Only when it is known that the form is regular, a straight line shortened by fading is extended to restore the original length.

(6) Determination of horizontal frame: In accordance with the rules set forth in the prior application "Frame Extracting Device and Rectangular Extracting Device" (Japanese Patent Application No. 7-203259), one set of two lines in order from the top of the table The horizontal straight lines that form the entry fields are extracted as horizontal frame lines, two by two.

(7) Vertical frame determination: The vertical frame line is determined for each line of the above-mentioned entry column. A vertical straight line whose upper and lower ends reach the two horizontal frame lines forming the line of interest is the vertical frame line in that line.

(8) Nesting processing: When the inside of a rectangular range surrounded by the determined vertical and horizontal frames on four sides is further finely divided to form a nested table, the rectangular range is set. Is newly regarded as a table. And the above (1)-
By repeating the process of (7), it is divided into two or more rectangular ranges.

[0015]

However, the ruled line extraction technique described above has the following problems. By actually inputting form data with an image scanner and processing images of various forms, some of the shapes and image states that cannot be handled by the technology of the prior application have a relatively high frequency of appearance. I knew it was. These shapes and states are as follows.

[1] In the technique of the prior application, it is necessary that the size of the partial pattern forming the table is not less than a certain size. However, in an image actually input by a scanner or the like, there are various patterns of blurring and collapsing, and a pattern having a size larger than a predetermined size does not always exist.

In some cases, one pattern is originally divided into pieces due to blurring, and due to crushing, one pattern is divided into pieces.
There may be a pattern having a size different from that of the surroundings at the dotted lines. For this reason, it is necessary to rescue short ruled line patterns that are interrupted due to blurring and dotted line patterns that are partially deformed due to crushing.

[2] In the technique of the prior application, the character area is a rectangular area surrounded by four frame lines on four sides, and two corners are provided at each of the four corners. A prerequisite is that the vertical and horizontal frame lines form a 90-degree corner. However, these conditions do not always match the actual form.

For example, one side of the character area may be constituted by a double line instead of one frame line. Further, the corners of the character area do not necessarily form corners of 90 degrees, and may be formed by R-shaped ruled lines with rounded corners. Further, there may be a case where a pattern having a smaller table structure exists as an independent pattern that does not contact the frame line inside the character area surrounded by the frame line on four sides. Therefore, it is necessary to process such a frequently appearing shape and improve the accuracy.

[3] In the technique of the prior application, the processing is divided into a regular table and an irregular table, and if it is known in advance that the table is regular, the information is used to We are working to improve processing accuracy. When performing processing using information that is regular, as a result of performing processing that expands a straight line shortened by blurring according to a certain rule or creates a new ruled line virtually, Creates different ruled line information.

Therefore, in the process subsequent to the ruled line extracting process, ruled lines that do not actually exist on the image are treated in the same manner as those that actually exist, and thus there is a problem that the accuracy of the process decreases. As an example of such subsequent processing, there is a layout recognition processing for checking whether the input image corresponds to a known form.

[4] As another example of the subsequent processing, there is a character complementing processing described in "Image Extraction Method" (JP-A-6-309498). In this processing, the pixels of the character portion lost when the pixels of the ruled line portion are removed from the image of the character that is in contact with the ruled line or overlaps with the ruled line are complemented to restore the character. As a result, the character pattern divided by the frame is complemented. At this time,
The average value of the sizes of other characters in the same character area as the character to be restored is used for the processing.

However, when the data to be passed to the character complementing process in the ruled line extracting process is created, if the size of each character area is too small, the accuracy of the subsequent character complementing process becomes poor. The reason for this is that the small number of characters contained in a small character area reduces the accuracy of the average value of the character size, and consequently the accuracy of character completion. Therefore, it is necessary to devise a method that does not reduce the accuracy of character completion as much as possible.

[5] In the technique of the prior application, the dotted lines cannot be accurately extracted unless the size of the pattern forming the dotted lines is constant, and the two dotted lines intersect in a T-shape. If there is, the dotted line corresponding to the T-shaped vertical bar tends to be extracted longer than it actually is. Therefore, a technique for more accurately extracting the dotted line is needed.

[6] Further, in the technique of the prior application, when the strip-shaped short line segment rectangle is not included in the rectangular range determined as the straight line rectangle, the position of the ruled line is determined by the position information of the line segment rectangle. Can't judge in detail. For this reason,
There is a problem in that the accuracy of ruled line position extraction is not very good.

As described in the above [1] to [6],
In the ruled line extraction technology of the previous application, the image of a form whose position and format are unknown, the image of poor quality with faintness or crushing, and the image where characters and frames are in contact with each other or characters are protruding from the frames are used. , There is a problem that cannot be dealt with sufficiently.

An object of the present invention is to provide a ruled line extracting apparatus and method capable of efficiently processing various patterns assumed for an arbitrary input image and extracting ruled lines more accurately. is there.

[0028]

FIG. 1 shows the principle of the device of the present invention. The apparatus of FIG. 1 is the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, eleventh, twelfth, and twelfth related to the ruled line extracting apparatus of the present invention. Thirteenth, fourteenth,
And the fifteenth principle and the first of the character cutting device
Including 6 principles.

The apparatus shown in FIG. 1 has the pattern extracting means 1 and the judging means 2-1, 2-2, 2-3, 2-4, 2-.
5, dotted line extraction means 3-1, straight line extraction means 3-2, line extraction means 3-3, correction means 4-1, 4-4, addition means 4-
2, 4-5, 4-6, creating means 4-3, generating means 4-
7, table extraction means 5, frame line extraction means 6, area extraction means 7,
The input means 8 and the change means 9 are provided.

In the first principle, the pattern extraction means 1
Extracts a partial pattern from the input image based on the pixel connection information. The determining unit 2-1 determines a partial pattern, which is located inside the circumscribing rectangle of the target partial pattern and has a size equal to or larger than a certain size, as a processing target, and the determining unit 2-2
Among the partial patterns that are not determined as the processing target by the determination unit 2-1, a partial pattern having an elongated shape is determined as the processing target.

The partial pattern represents a set of connected pixels included in the input image. The judging means 2-1 considers the partial patterns having a certain size or more as table candidates,
Paying attention to this, the partial pattern having a certain size or more located inside thereof is determined as the processing target of the ruled line extraction processing.
The determining unit 2-2 adds a partial pattern, which has not been determined to be a processing target, to a processing target when the vertical or horizontal width thereof is relatively small and has an elongated shape.

As a result, the state in which a certain ruled line is separated into small patterns due to blurring, or the state in which a part of the dotted line is crushed to form an elongated shape pattern is rescued,
It becomes possible to add to the processing target, and more accurate ruled line extraction processing is performed.

In the second principle, the pattern extraction means 1
Extracts a partial pattern from the input image based on the pixel connection information. The dotted line extraction means 3-1 extracts the dotted line pattern based on the partial pattern, and extracts other dotted line patterns so as not to include the partial patterns forming the dotted line pattern.

The dotted line pattern represents a portion in which a plurality of partial patterns are regularly arranged. When the dotted line extraction means 3-1 extracts a dotted line pattern, a partial pattern that has once been determined to form a certain dotted line pattern is constrained to be unable to form another dotted line pattern, thereby making a plurality of dotted line patterns mutually. It can be extracted so that it does not overlap. Therefore, even if two dotted lines intersect in a T-shape, the possibility of extracting one dotted line longer than it actually is is reduced.

In the third principle, the pattern extracting means 1
Extracts a partial pattern from the input image based on the pixel connection information. The dotted line extraction means 3-1 calculates the distance between the two partial patterns using the position coordinates of the circumscribing rectangle of each partial pattern, and extracts the dotted line pattern based on the distance between the partial patterns.

For example, the dotted line extracting means 3-1 uses the distance between the end points of the circumscribed rectangles of the two partial patterns as the distance between the partial patterns, and determines the regularity of the arrangement of the partial patterns based on the distance. It is possible to extract the dotted line pattern from the arrangement of the partial patterns having different sizes.

In the fourth principle, the pattern extraction means 1
Extracts a partial pattern from the input image based on the pixel connection information. The dotted line extraction means 3-1 extracts the dotted line pattern based on the partial pattern, and the correction means 4-1.
Changes the position of the end point when the position of the end point of the dotted line pattern needs to be corrected.

As described above, the correction means 4-1 corrects and changes the positions of the end points of the dotted line pattern, thereby improving the accuracy of dotted line extraction. In the fifth principle, the straight line extracting means 3
-2 extracts a straight line pattern from the input image based on the pixel connection information. The adding means 4-2 adds the information for associating the straight line pattern and the line segment pattern forming the straight line pattern to both or one of the straight line pattern information and the line segment pattern information.

A straight line pattern corresponds to a horizontally or vertically long pattern extracted from an input image by mask processing using a horizontally or vertically long mask, and is divided into one or more shorter line segment patterns for management. To be done. Adding means 4
-2 associates both by adding line segment pattern identification information to straight line pattern information, or adding straight line pattern identification information to line segment pattern information.

As a result, it becomes easy to refer to the information on the line segment pattern from the information on the straight line pattern, and the position of the straight line pattern can be recognized in detail as necessary. In the sixth principle, the straight line extraction means 3-2 extracts a straight line pattern from the input image based on the pixel connection information. The adding means 4-2 adds the information indicating the positional relationship of the plurality of line segment patterns forming the straight line pattern to the information of the line segment pattern.

For example, the adding means 4-2 adds the identification information of one line segment pattern of the two adjacent line segment patterns to the information of the other line segment pattern to describe the positional relationship between them. To be done. Thereby, it becomes easy to refer to the information of the adjacent line segment pattern from the information of a certain line segment pattern, and the process of searching the information of the line segment pattern becomes efficient.

In the seventh principle, the straight line extracting means 3-2
Extracts a straight line pattern from the input image based on the pixel connection information. The adding means 4-2 adds information indicating the processing performed on the straight line pattern when the straight line pattern is extracted, to the information on the straight line pattern.

As the processing performed on the straight line patterns, for example, an integration processing for integrating the straight line patterns,
There is a stretching process for stretching a straight line pattern in a specific direction. The adding unit 4-2 adds the type of processing and information indicating whether or not the processing is performed to the linear pattern information. As a result, in layout recognition processing or the like, it is possible to determine whether the target straight line originally existed or was deformed by specific processing, and it is possible to perform more detailed processing.

In the eighth principle, the straight line extracting means 3-2
Extracts a straight line pattern from the input image based on the pixel connection information. The creating unit 4-3 virtually creates a line segment pattern in a portion of the area representing the straight line pattern where the line segment pattern forming the straight line pattern does not exist.

Since the creating means 4-3 virtually creates an appropriate line segment pattern, it becomes possible to refer to the information of the line segment pattern even in an area where the line segment pattern originally does not exist. As a result, the position of the straight line pattern is determined more accurately, and the accuracy of other processing that requires the position information is improved.

In the ninth principle, the line extracting means 3-2
Extracts a straight line pattern from the input image based on the pixel connection information. The correction unit 4-4 changes the size of the line segment pattern so as to compensate for the portion of the region representing the straight line pattern where the line segment pattern forming the straight line pattern does not exist.

For example, by correcting the line segment pattern by the correction means 4-4 to eliminate the area where the line segment pattern does not exist, the position of the straight line pattern can be more accurately determined as in the eighth principle. The accuracy of other processing that requires the position information is improved.

In the tenth principle, the straight line extracting means 3-
2 extracts a straight line pattern from the input image based on the pixel connection information. The creating unit 4-3 virtually creates a line segment pattern in the area representing the straight line pattern, and the correcting unit 4-4 changes the size of the line segment pattern forming the straight line pattern. The determining unit 2-3 creates a new line segment pattern or a surrounding line depending on the size of a portion in the line pattern that does not include the line segment pattern. Determine whether to change the size of the minute pattern.

Depending on the size of the portion where the line segment pattern does not exist, either the creation of the line segment pattern by the creating means 4-3 or the change of the size of the line segment pattern by the correcting means 4-4 can be selected. By so doing, the processing efficiency can be improved.

In the eleventh principle, the frame line extracting means 6
Is a frame line pattern extracted from the input image based on the pixel connection information, and the region extraction means 7 extracts a rectangular region surrounded by the vertical and horizontal frame line patterns. The determining unit 2-5 determines whether or not the corner portion of the rectangular area has a specific shape, and the adding unit 4-6 uses the information indicating the possibility of the specific shape for the complementary processing. Append to information.

The frame line pattern represents a pattern that is a candidate for a ruled line in a table among straight line patterns. Usually, a frame region pattern of two vertical lines and a horizontal line forms a rectangular area corresponding to one character area. It For example, the determination means 2-5
Examines the corners of the rectangular area and finds that the vertical and horizontal ruled lines are R
It is determined whether or not it corresponds to the ruled line, and the adding means 4-6.
Adds information indicating the possibility of a ruled line with R to the information to be passed to the character complementing process.

As a result, in the character complementing process, the amount of data for determining whether the pattern in the rectangular area is a character or a part of a ruled line is increased, so that the accuracy of the character complementing process is improved.

In the twelfth principle, the frame line extracting means 6
Is a frame line pattern extracted from the input image based on the pixel connection information, and the region extraction means 7 extracts a rectangular region surrounded by the vertical and horizontal frame line patterns. The determining means 2-5 determines whether or not a double line exists in the rectangular area, and when it is determined that the double line exists, the generating means 4-7,
Information for the complementary process is generated based on the determination result.

When the determining means 2-5 determines that a double line exists near the periphery of the rectangular area, the generating means 4-7, for example, processes the area not including the double line in the character complementing process. Generate the target information. This allows
In the character complementing process, there is no possibility that a part of the double line in the rectangular area is regarded as a character, so that the accuracy of the character complementing process is improved.

In the thirteenth principle, the frame line extracting means 6
Is a frame line pattern extracted from the input image based on the pixel connection information, and the region extraction means 7 extracts a rectangular region surrounded by the vertical and horizontal frame line patterns. When a certain number of rectangular areas that are less than a certain size are lined up, the generation means 4-7 collects a plurality of rectangular areas and generates a processing range of the complementary processing.

For example, even if the squares for one character are arranged side by side in one row like the amount column of a form,
The generating unit 4-7 regards them as one character area, and collectively generates information that is a processing target of the character complement processing. As a result, in the character complementing process, it is possible to refer to the size of the character in another square, and the accuracy of the character complementing process is improved.

In the fourteenth principle, the pattern extraction means 1 extracts a partial pattern from the input image based on the pixel connection information, and the table extraction means 5 extracts a partial pattern which is a candidate for a table as a table pattern. The frame line extraction means 6
A frame line pattern is extracted from the table pattern. The input unit 8 extracts a frame pattern of a partial pattern, which is located inside the circumscribing rectangle of the table pattern and has a certain size or more, and which is not regarded as a part of the frame line pattern, as a new table pattern. Input to the means 6.

The table extracting means 5 extracts, for example, a partial pattern having a certain size or more as a table pattern, and the frame line extracting means 6 has, for example, one or more parts located inside the circumscribed rectangle of the table pattern. A frame line pattern is extracted from the partial pattern. The input unit 8 regards the remaining partial patterns that are not extracted as the frame line pattern and have a certain size or more, as candidates for another table. And
The frame extraction unit 6 recursively processes this as a new table pattern.

As a result, even if an independent small table is included in the table, the ruled lines of the table can be extracted, and the small table is not erroneously processed as a character. .

In the fifteenth principle, the table extracting means 5 is
A table pattern, which is a table candidate, is extracted from the input image based on the pixel connection information, the frame line extracting unit 6 extracts a frame line pattern from the table pattern, and the region extracting unit 7 extracts vertical and horizontal frame line patterns. Extract the rectangular area surrounded by. The input means 8 inputs the rectangular area in the nested state into the frame line extraction means 6 as a new table pattern, and the change means 9
Changes the number of times the recursive process is repeated by the frame line extraction means 6 and the input means 8 based on information provided from the outside or information created inside.

The rectangular area extracted by the area extracting means 7 is regarded as another table candidate, the input means 8 inputs it again to the frame line extracting means 6, and the frame line extracting means 6 recursively processes. Then, the ruled line of the nested structure is extracted. The changing unit 9 changes the setting of the number of repetitions of this nesting process based on a user's instruction or information automatically generated internally.

By setting the number of repetitions based on the user's instruction, it is possible to perform a process in which the depth of the nested structure of the table to be processed is specified in advance, and to set it based on the information generated internally. Thus, it is possible to perform processing without specifying the depth.

In the 16th principle, the pattern extracting means 1 extracts a partial pattern from the input image based on the connection information of the pixels, and the line extracting means 3-3 extracts a solid line or a dotted line pattern from the partial pattern. Extract. The determining means 2-5 determines whether or not the line pattern constitutes a frame, and the adding means 4-6 adds information indicating the state of the line pattern to the information of the line pattern determined to constitute the frame. Add. The area extracting means 7 extracts a character area based on the line pattern, and the generating means 4-7 generates information for character complement.

The line pattern includes a straight line pattern and a dotted line pattern, and the determining means 2-5 determines that a frame is formed when the vertical and horizontal line patterns form a rectangular area. Here, the information indicating the state of the line pattern includes both or one of the identification information of the line segment pattern or the partial pattern forming the line pattern and the information indicating the processing performed on the line pattern. . By the addition means 4-6 adding such information to the information of the line pattern,
The accuracy of the processing of the area extracting means 7 and the layout recognition processing is improved.

Further, the generation means 4-7 uses, as information for character complement, information indicating the possibility of R ruled lines, character area information considering the possibility of double lines, and small characters arranged in a line. Character area information or the like in which areas are combined is generated. By passing these pieces of information to the character completion processing, the processing accuracy is improved as described above.

As described above, according to the present invention, it is possible to extract ruled lines (frame lines), character regions, etc. with high accuracy from an image with poor image quality that is blurred or crushed. For example, each means 1, 2-1, 2-2, 2-3, 2- of FIG.
4, 2-5, 3-1, 3-2, 3-3, 4-1, 4
2, 4-3, 4-4, 4-5, 4-6, 4-7, 5,
Reference numerals 6, 7, 8, and 9 correspond to a CPU (central processing unit) 31 and a memory 32 in FIG. 3, which will be described later.

[0067]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. According to the present invention, it is possible to extract a frame regardless of whether or not there is a blur in an image, and even in a table in which a solid line and a dotted line are mixed, as long as the character area is formed of a rectangular range. . In the following embodiments, there are one or more frames whose size, position, and inclination are unknown, and various characters including characters touching the frame and characters protruding from the frame are written on these frames. For documents that have Then, consider the case where a frame is extracted from the image of such a document.

FIG. 2 is a functional block diagram of a character recognition device to which the ruled line extraction device of the embodiment is applied. In FIG. 2, the ruled line extraction device includes a reduction processing unit 12, a connection pattern extraction unit 13, a mask processing unit 14, a horizontal straight line extraction unit 15, a vertical straight line extraction unit 16, and a nested structure processing unit 17. Further, the shaded processing blocks mainly represent different processing (changed portion) from the prior application such as “Pattern extracting device and pattern region cutting method” (Japanese Patent Application No. 8-107568).

First, the reduction processing section 12 uses the input pattern 1
Reduce 1 The input pattern 11 is a binary image after correction of extreme inclination and rotation. Connection pattern extraction unit 1
3 extracts a pixel connection pattern from the reduced image, and the mask processing unit 14 thins it.

At this time, when the size of the image pattern is a small pattern that does not reach a certain size, the connected pattern extraction unit 13 processes on condition that the width of either the vertical or horizontal is small and elongated. Extract as a target. The reason for imposing such a condition is to reduce the possibility that the character pattern will be the target of ruled line extraction processing. When extracting a small pattern, various threshold values are adjusted.

When processing a small pattern that meets this condition, the mask processing section 14 adjusts the size of the mask according to the size and uses an appropriate one of several masks. . This makes it possible to extract a ruled line or the like that is interrupted and shortened due to blurring.

Next, the horizontal straight line extraction unit 15 performs horizontal adjacent projection (process P1), horizontal line segment detection (process P2), horizontal line segment integration (process P3), horizontal straight line detection (process P4), and horizontal straight line search ( After performing the process P5), the ruled line extraction device performs horizontal dotted line detection (process P6) and horizontal straight line integration (process P7).

Next, the vertical line extraction unit 16 causes vertical adjacent projection (process P8), vertical line segment detection (process P9), vertical line segment integration (process P10), vertical line detection (process P11), and vertical line. After performing the search (process P12), the ruled line extraction device
Vertical dotted line detection (process P13) and vertical straight line integration (process P1)
4) is performed.

In the dotted line detection processes P6 and P13, the ruled line extraction device calculates the distance between patterns for patterns smaller than a certain size. At that time, the distance between the circumscribed rectangles of the two patterns is used as the distance between the two patterns, instead of the distance between the midpoints of the two patterns.

As a result, even if the size of the pattern is slightly different due to the crushing, if the interval is constant, it can be extracted as a dotted line. Further, by not using the pattern once used for the dotted line extraction again, it is possible to avoid extracting a dotted line that does not originally exist.

When the dotted line is partially crushed, it may not be extracted as a solid line or a dotted line depending on the size of the crushed part. Therefore, the ruled line extraction device performs a pixel search within a fixed length range from both ends of the extracted dotted line after extracting the dotted line. Then, it is judged to what extent the dotted line exists, and the part which could not be extracted as the dotted line is supplemented.

In the straight line integration processing P7 and P14, the ruled line extraction device virtually creates a line segment rectangle in a region within the range regarded as a straight line and where the strip-shaped line segment rectangle does not exist. create. The created position, thickness, etc. are calculated based on the position, thickness, etc. of surrounding line segments. Thereby, in the subsequent processing, a virtually created line segment rectangle can be used, and the position of the straight line can be accurately evaluated.

Next, the ruled line extraction device performs rule processing on the table (process P15), determines a horizontal frame line (process P16),
Vertical and horizontal frame lines are determined (process P17), and vertical and horizontal frame lines are extracted. Then, the nested structure processing unit 17 determines a horizontal frame line within the rectangular area surrounded by these frame lines (process P18),
Vertical and horizontal frame lines are determined (process P19).

Here, the table rule processing P15 is performed only when it is known in advance that the processing target is a table having a regular structure. The ruled line extraction device performs this process P
At 15, the process of extending the straight line and the process of creating a new straight line are performed.

At this time, flag information indicating what kind of processing has been performed to extract a straight line, and information about the actual pixel density in the straight line rectangle,
I will add it to the straight line information. These pieces of information enable the character recognition device to perform weighting on each line in the subsequent layout recognition process P23 and the like.

Next, the ruled line extracting device performs a rectangular expression using the extracted vertical and horizontal frame lines (process P20) and calculates a character area (process P21). Then, data relating to the ruled line extraction result is created and the subsequent processes P22 and P23 are performed.
Pass to.

At this time, regarding the shape of the ruled line, new information is added to the shape having a high appearance frequency out of the shapes different from those conventionally assumed. Frequently appearing shapes include R-shaped ruled lines with rounded corners and double lines in which two straight lines form one set.

Regarding the R ruled line, pay attention to the corners of the character area surrounded by the ruled line, and if the pattern exists within a certain fixed range, it is highly likely that the R ruled line is a ruled line. The information is passed to the character complementing process P22. In addition, regarding the double line, a character area in consideration of its existence is calculated, and the calculated information is passed to the character complementing process P22.

When the information to be passed to the character complementing process P22 is created, the input original image is used instead of the reduced image used in the previous processes. This makes it possible to determine the shape of the frame in more detail.

If the character areas have a certain size or less and a certain number or more of character areas around the character areas have the same size, the character areas are set to 1
The data to be passed to the character complementing process P22 is created collectively. This makes it possible to refer to the size of a character included in another related character area even if the character area has a small number of characters.

Then, the character recognition device uses the data passed from the ruled line extraction device to perform character complement (process P22),
Layout recognition (process P23), character recognition (process P2)
4) and table recognition (process P25) are performed.

In the character complementing process P22, the character recognition device determines, based on the passed information, whether the pattern of the corner portion of the character area corresponds to a character or a ruled line with R. In the present embodiment, more accurate information about the position and shape of the ruled line is passed, so the accuracy of the character complementing process P22 is improved.

Further, as described above, in the ruled line extracting process, a flag indicating the state of the ruled line, such as whether or not each ruled line is expanded, is generated as information indicating the reliability of the ruled line. , And is passed to the layout recognition process P23. Here, the reliability of the ruled line means information indicating that the extracted ruled line may actually exist at that position.

The character recognition device performs layout recognition processing P2.
When matching processing of the ruled line structure is performed in 3, the influence on the processing of the ruled line is changed based on the reliability of each ruled line. In this way, highly accurate matching processing is performed by mainly using highly reliable ruled lines.

By performing the respective processes shown in FIG. 2, even if the image of a form whose position and format are unknown or the image with blur and poor image quality, the frame in the table format frame is accurately extracted. You can Further, even when the character and the frame are in contact with each other or the character is outside the frame, the character region can be accurately extracted.

The ruled line extracting apparatus of this embodiment is, for example, as shown in FIG.
It is realized by an information processing device (computer) as shown in. The information processing apparatus of FIG. 3 includes a CPU 31 and a memory 3.
2, input device 33, output device 34, external storage device 35,
A medium driving device 36, a network connection device 37, and a photoelectric conversion device 38 are provided, and these devices are connected to each other by a bus 39.

The CPU 31 executes the program stored in the memory 32 to perform each processing shown in FIG. As the memory 32, for example, ROM (read only memory),
RAM (random access memory) or the like is used. The memory 32 stores the above-mentioned programs and data necessary for processing.

The input device 32 corresponds to, for example, a keyboard, a pointing device, etc., and is used for inputting a request or an instruction from the user. The output device 34 corresponds to a display device, a printer, etc., and is used to output processing results and the like.

The external storage device 35 is, for example, a magnetic disk device, an optical disk device, a magneto-optical disk device, or the like. The above-mentioned program and data are stored in the external storage device 35, and if necessary, they are stored in the memory 32.
It can be loaded and used. The external storage device 35 is also used as a database for storing input images and the like.

The medium driving device 36 can drive the portable recording medium 40 and access the stored contents. As the portable recording medium 40, a memory card, a floppy disk, a CD-ROM (compact disk read only memor)
y), an optical disk, a magneto-optical disk, or any other computer-readable recording medium can be used.
The above-described program and data can be stored in the portable recording medium 40, and can be loaded into the memory 32 and used as necessary.

The network connection device 37 is a LAN (lo
It is connected to an arbitrary communication network such as a cal area network), performs data conversion accompanying communication, and communicates with a database 40 'or the like of an external information provider. As a result, the ruled line extraction device can receive the above-mentioned program and data from the database 40 'via the network and load them into the memory 32 for use as needed.

The photoelectric conversion device 38 is, for example, an image scanner and inputs an image such as a form to be processed. Next, referring to FIG. 4 to FIG.
Each processing of will be described in order.

When the image of the input pattern 11 has a certain resolution or higher and the size of the image is relatively large, the reduction processing unit 12 performs the image reduction process for the efficiency of the process. The input original image is stored as it is.

The concatenated pattern extraction unit 13 uses the "pattern area cutting method and pattern extraction device" (Japanese Patent Application No. 7-2).
Labeling processing is performed by the method shown in 82171). In this process, the black pixel connection area connected by 8 connections in the upper, lower, left, and right directions is extracted as a partial pattern in order to stably pick up each pattern without depending on the relative relationship of the positions where a plurality of tables are arranged. And give it a label.

Then, these partial patterns are discriminated and the patterns of the table included in the image are extracted. Further, since the size of the partial pattern obtained by labeling is necessary later, the connected pattern extraction unit 13 calculates the coordinates of the corners of the circumscribed rectangle that approximates the partial pattern during the labeling process.

Next, among the extracted partial patterns, a pattern having a certain size or more is extracted as a table candidate. When extracting a large connected pattern, patterns that are inside the large connected pattern and have a certain size or more are also extracted at the same time, and those large and small patterns are treated as patterns having the same label.

As for small patterns that have not reached a certain size, they are extracted as table candidates only when either the vertical or horizontal width is smaller than a certain value and have an elongated shape. Give the same label as the large connection pattern. When processing these small patterns, the threshold value and the like are automatically changed in proportion to the size thereof.

As described above, when the partial pattern to be processed is extracted based on the pixel connection information, the size of the pattern to be processed does not reach the size of the pattern located inside the circumscribed rectangle of the pattern of interest. Even in the case of a pattern, a pattern having an elongated shape with an extremely small vertical or horizontal width is regarded as a processing target. As a result, it is possible to accurately perform the extraction process even on an image in which the pattern is faintly separated into smaller patterns or in which a part of the dotted line is crushed.

FIG. 4 shows an example of an image after the above-mentioned labeling processing. In FIG. 4, patterns 41, 4
Nine connection patterns of 2, 43, 44, 45, 46, 47, 48, 49 are extracted. The connected pattern extraction unit 13 firstly selects the pattern 41 having the largest circumscribed rectangle.
Is extracted and a label is added to it. Next, pattern 4
Since the horizontal line pattern 46 between the pattern 2 and the pattern 48 has a certain width or more, it is considered to be a part of the large pattern 41.

The horizontal line pattern 47 between the pattern 43 and the pattern 49 has a size smaller than a certain value, but has a small line width and a long horizontal line. Therefore, the horizontal line pattern 47 is a part of the large pattern 41. To consider. Thus, as shown in FIG. 5, all the patterns 41, 46 and 47 are treated as patterns belonging to the same label.

FIG. 6 is a flowchart of the processing of the connection pattern extraction unit 13. When the process is started, the connection pattern extraction unit 13 first performs a labeling process on the input image (step S1), and the vertical or horizontal length of the circumscribed rectangle of the partial pattern having a certain label is the threshold TH1.
It is determined whether or not the above (step S2).

If it is greater than or equal to TH1, the label number is stored (step S3), and it is determined whether the processing has been completed for all labels (step S4).
Then, if any unprocessed label remains, the processing from step S2 is repeated.

If both the vertical and horizontal lengths of the circumscribing rectangle do not reach TH1 in step S2, then it is determined that the vertical or horizontal length is the threshold value TH2 and the circumscribing rectangle is vertical or horizontal. It is checked whether or not it has an elongated shape (step S5). If this condition is met,
If the partial pattern is left to be processed by performing the processing of step S3 and subsequent steps and if it is not satisfied, step S4
Perform the following processing. When all the labels have been processed, the process ends.

By thus determining the size of the pattern using the two threshold values, it becomes possible to leave a small partial pattern that is highly likely to be a ruled line as a processing target. Note that, of the partial patterns left in step S3, those that are inside the patterns considered as table candidates are rewritten to the same label as that pattern.

Next, the mask processing unit 14 extracts the connection pattern extracted by the connection pattern extraction unit 13 and has a certain size or more, and the connection pattern existing therein (the pattern having the same label as the former). And are candidates for the table. Then, the “frame extraction device and rectangle extraction device” of the prior application
Masking is performed according to the method of (Japanese Patent Application No. 7-203259).

This mask processing is carried out in order to facilitate extraction of long straight lines existing only in the table by eliminating the extreme oblique component from the image. Specifically, the mask processing unit 14
The entire image is scanned using horizontally and vertically long mask rectangles. Calculate the ratio of the black pixel pattern in the mask, and if it is more than a certain value, fill the entire area in the mask with black pixels and leave it as a pattern.
If it is less than a certain value, the pattern in the mask is deleted. In this way, the vertical and horizontal components are extracted.

Here, when a plurality of rows or columns continue and the above ratio reaches a predetermined threshold value or more, a large rectangular range is formed by combining those continuous patterns, and the center line thereof is set as the processing result. To do. As a result, the thick pattern is thinned. Further, in order to prevent the gap between the line segment patterns of the processing result from being opened, the mask application ranges are set so as to overlap each other. The original image before the mask processing is stored separately from the image after the mask processing.

The mask processing section 14 carries out mask processing for each of the extracted connected patterns, and when the size of the pattern is larger than a certain size, the mask of the same size is used as in the previous application. When the size of the pattern is less than a certain size, the length of the mask is changed appropriately according to the size.

FIG. 7 shows the three patterns 4 shown in FIG.
The result of performing the mask processing on 1, 46, and 47 is shown. In FIG. 7, patterns 41, 46, 4
Only the vertical and horizontal components of 7 are extracted, and the entire pattern is thinned. In this case, the mask length is adjusted for each pattern, and masks having different lengths are used.

FIG. 8 is a flowchart of the processing of the mask processing section 14. When the processing is started, the mask processing unit 1
4 determines whether the connected pattern of a certain label is the label of the pattern extracted by the connected pattern extraction unit 13 (step S11).

If it corresponds to the label extracted by the concatenated pattern extraction unit 13, the mask processing parameter is calculated from the size of the circumscribed rectangle of the partial pattern of the label based on a predetermined calculation method (step S12). ).
At this time, for example, the length of the mask is calculated based on the vertical or horizontal length of the circumscribing rectangle.

Next, a mask is generated using the calculated parameters, and mask processing is performed on the partial pattern (label image) of the label of interest (step S13). Then, it is determined whether or not the processing has been completed for all the labels (step S14), and if any unprocessed label remains, the processing from step S11 is repeated.

In step S11, if the processing target does not correspond to the label extracted by the concatenation pattern extraction unit 13, the processing from step S14 onward is performed, and when all the labels have been processed, the processing ends. As described above, by changing the size of the mask according to the size of the partial pattern to be processed, it becomes possible to extract the mask image even from the small pattern.

Next, in the process P1, the horizontal straight line extracting section 15 calculates a projection value for the mask processing image of the partial pattern as shown in FIG. 7 by using the conventional adjacent projection method. Then, in the process P2, a horizontal line segment or a part of a horizontal line having a certain length is approximated to a rectangle as shown in FIG. 9 and detected. Although the masked image is used in these processes, the original image is used in the following processes.

Next, the horizontal straight line extracting unit 15 performs the processes P3 and P3.
4, "frame extraction device and rectangle extraction device" of the prior application
Using the method of (Japanese Patent Application No. 7-203259), the connection state between line segments is investigated, and the horizontal straight lines formed by some line segments are extracted as ruled line candidates. In these processes, FIG.
As shown in, a long straight line is detected by integrating neighboring line segment rectangles among the detected line segment rectangles, and the detected horizontal line is approximated by the circumscribed rectangle.

In the process P4, after the detection of the horizontal straight line, information indicating the relationship between the straight line and the line segments constituting the straight line is stored. As this information, regarding one or more line segments forming a certain straight line, information indicating the order in which the line segments are arranged and which line segment forms which straight line is used. The processes P8, P9, P10, and P11 of the vertical straight line extraction unit 16 are the same as the case of the horizontal straight line.

In the processes P4 and P11, for example, the following information is added to the straight line and line segment information. The straight line extraction units 15 and 16 firstly, for all straight lines and line segments,
Number each one when extracting. Then, in the information indicating the line segment, the number of the straight line to which it belongs and the number of the line segment on the right side (below in the case of a vertical line) that constitutes the straight line are added. Further, to the information indicating the straight line, the number of the line segment on the leftmost side (the topmost in the case of a vertical line) among the line segments forming the straight line is added.

The following straight line search processing P5, P12,
In the straight line integration processing P7, P14, etc., information indicating the relationship between these straight lines and line segments is updated each time the length, position, and the number of line segments forming the line change.

FIG. 11 is a flowchart of the line segment integration processing and the straight line detection processing by the straight line extraction units 15 and 16. When the processing is started, the straight line extraction units 15 and 16 first input the information of the line segment rectangles extracted in the processing P2 and P9 (step S21), and the line segments that are in contact with each other or overlap each other. Are integrated (step S2
2) The coordinate values of the circumscribing rectangle that surrounds the plurality of integrated line segments are used as coordinate information representing a straight line (step S23).

Next, to the information of each of the plurality of integrated line segments, the number of the adjacent line segment on the right side is added in the case of a horizontal line, and the number of the adjacent line segment below is added in the case of a vertical line ( Step S2
4). Then, the number of the leftmost line segment is added in the case of a horizontal line and the number of the topmost line segment is added in the case of a vertical line to the information of the straight lines formed by them (step S25), and the processing is performed. To finish.

As described above, when a straight line is extracted, information relating to a straight line and a line segment forming the straight line is added to one or both of the straight line and the line segment, so that the position of the certain straight line is detailed. Can be described. Further, by adding information describing the positional relationship between the line segments forming the straight line to the information about the line segment, the position of the straight line can be described in more detail. Then, by passing these detailed information to the character complementing process P22, the process P22 becomes efficient.

Next, in the process P5, the horizontal straight line extracting section 15 detects the pattern (pixel) in the straight line as shown in FIG. 12 in order to accurately detect the left and right ends of the horizontal line which is approximated by the rectangle. The search is performed left and right.

When a portion having no pixel (blank) exists in the traveling direction during the search, it is assumed that there are pixels up to a certain number of pixels, and the search is continued in the traveling direction. If there is a blank space over a certain number of pixels, the search is terminated with that position as the end point. In the following processing, the end points detected by the search are used as the end points of the straight line. Similarly, in the process P12, the vertical straight line extraction unit 16 searches for a pattern vertically in order to accurately detect the upper and lower ends of the straight line.

Here, as a result of the straight line search, as shown in FIG. 13, the search extends to a region where the line segment 52 forming the straight line 51 does not exist, and the end point (search end point) 5 outside the straight line 51.
Suppose that 3 and 54 are obtained. In this case, as shown in FIG. 14, line segment rectangles 55 and 56 of appropriate sizes are newly created. The positions and thicknesses of the line segment rectangles 55 and 56 are determined from the inclination of the table calculated in advance, the thickness of the neighboring line segments, and the like. And the circumscribed rectangle 5 including the line segment rectangles 55 and 56
Let 1'be a new straight line rectangle.

As described above, the line segment position information is required by virtually creating a line segment for a portion in which the line segment forming the straight line does not exist in the area indicating the straight line. The accuracy of the processing of is improved. Further, by changing the size of the line segment around the portion where the line segment does not exist, it is possible to eliminate the portion where the line segment does not exist.

For example, line segments 57 and 5 as shown in FIG.
In a straight line consisting of 8 and 59, both ends are end points 60,
It is assumed that the decision is 61. At this time, the straight line extraction unit 15
16, the line segments 57 and 59 at both ends are extended to end points 60 and 61, respectively, and line segments 57 'and 59' are formed.
Is generated and these coordinate values are stored as line segment information.
This improves the accuracy of other processing that requires the position information of the line segment.

As described above, when the position of the end point needs to be corrected after the straight line is extracted, the accuracy of the straight line extraction is improved by correcting and changing the end point. Furthermore, a straight line search process is adopted that selects whether to newly generate a line segment or change the size of surrounding line segments depending on the size of a portion that does not include a line segment that constitutes a straight line. You can also

FIG. 17 is a flowchart of such a straight line search process. When the processing is started, the straight line extraction unit 1
First, the pixels 5 and 16 search for pixels in the horizontal or vertical direction from the end points of the straight line (step S31), and determine whether or not the search processing has been performed for a certain length or longer (step S32).

If a pixel is not found by searching for a certain length or more, a line segment is virtually created in the blank portion as in FIG. 14 (step S33). Then, information indicating the positional relationship between the created line segment and the adjacent line segment is created (step S34), and the process ends. In order to express the positional relationship between two adjacent line segments, for example, the number of the right line segment is added to the information of the left line segment. Alternatively, the number of the lower line segment is added to the information of the upper line segment.

Further, in step S32, when the search is not performed for a certain length or more, the line segments at both ends forming the straight line are extended to the search end point as in FIG. 16 (step S35), and the process is ended. To do.

As described above, by selecting whether to newly generate a line segment or change the size of the surrounding line segment depending on the size of the portion where the line segment does not exist, the processing efficiency is improved. Therefore, the processing speed is improved. Next, in the process P6, the ruled line extraction device extracts a horizontal dotted line according to the method of the “frame extraction device and rectangle extraction device” (Japanese Patent Application No. 7-203259) of the prior application. Further, in the process P13, a vertical dotted line is similarly extracted. However, the process P6 is not always performed before the process P13, and may be performed after the process P13 as necessary. In these processes,
Find the partial patterns that have a certain size and are regularly arranged, and enclose these patterns in a rectangle to express the position of the dotted line.

By the way, in the method of the prior application, FIG.
As shown in, the distances d1, d2, d3, d4, d5, and d6 between the midpoints of the circumscribing rectangles surrounding each pattern are calculated, and the distances are compared to determine whether the patterns are regularly arranged. Was there.

The dotted line in FIG. 18 originally corresponds to the case of a two-dot broken line, and the patterns are regularly arranged. However, in the distance calculation method of the prior application, the distances d1, d2, d4, d5 and the distance d are set.
It cannot be said that 3 and d6 are equal. Therefore, there is a high possibility that the dotted line will not be extracted. The same applies when the normal dotted line is partially crushed to form a pattern as shown in FIG.

Therefore, the intervals D1, D2, D3, D4, D5, D6 between the circumscribing rectangles of each pattern will be used as the distance instead. Even if the sizes of the patterns are different, the intervals between the patterns are almost equal, so that it is easy to detect the regularity of the patterns, and the dotted lines can be extracted.

Further, as shown in FIG. 19, the size of the points forming the dotted line is considerably large, and if the distances dd1, dd2, dd3 between the midpoints used in the method of the prior application are too large, It may be judged and may not be regarded as a dotted line.
Even in such a case, by using the intervals DD1, DD2, and DD3 between the circumscribing rectangles of the respective patterns, it is determined that the points are regularly arranged, and it is possible to extract the points as dotted lines.

As described above, when the dotted lines are extracted, the distance between the partial patterns is calculated by using the position coordinates of the circumscribing rectangle of each partial pattern, so that the dotted lines (broken lines) composed of patterns of different sizes can be obtained. It becomes possible to extract. Further, by not using the pattern once determined to form the dotted line for the processing again, it is possible to reduce the possibility that the end of the dotted line is extended excessively and the dotted line is extracted longer.

For example, as shown in FIG. 20, in the case of a dotted line pattern intersecting in a T shape, first, two vertical dotted lines are
Range from rectangle c1 to rectangle c4 and rectangle d1 to rectangle d
It is extracted in the range up to 4. Here, if the pattern once determined to constitute the dotted line can be used again, the two horizontal dotted lines are extracted in the range from the rectangle a1 to the rectangle d2 and in the range from the rectangle b1 to the rectangle d3, respectively. However, the interval between the rectangle c2 and the rectangle d2 is not so wide.

However, since there are no points in the eyes of the user between the rectangles c2 and d2 and between the rectangles c3 and d3, these horizontal dotted lines do not extend to the rectangles d2 and d3. It is natural to think. Therefore, the rectangles c2, c3, d2, and d3 that are once determined to form the vertical dotted line are not used when extracting the horizontal dotted line, so that the horizontal dotted line is accurately extracted as shown in FIG. It becomes possible to do.

In FIG. 21, the extraction range of the vertical dotted line is shown in FIG.
Similar to the case of 0, the horizontal dotted line extraction range is rectangle a.
It is understood that the horizontal dotted lines are more accurately extracted from 1 to rectangle a2 and rectangle b1 to rectangle b2.

FIG. 22 is a flowchart of such dotted line detection processing. When the processing is started, the ruled line extraction device first inputs the information of the circumscribed rectangle of the pattern of each label (step S41) and extracts the vertical dotted line (step S).
42). Next, the label used by the vertical dotted line is omitted from the processing target (step S43), the horizontal dotted line is extracted (step S44), and the process is terminated. As a result, the pattern regarded as part of the vertical dotted line is not simultaneously regarded as part of the horizontal dotted line.

As described above, when extracting a dotted line, it is assumed that the dotted line cannot be constructed again for a pattern that is once determined to constitute the dotted line. It can prevent being judged.

By the way, in this extracting method, unless the dotted line in the T-shaped horizontal bar direction is extracted first, the same result as the method of the prior application will be obtained. Therefore, the pattern of FIG.
In the case of a pattern rotated by 0 °, it is necessary to switch the order of the vertical dotted line extraction processing and the horizontal dotted line extraction processing. Therefore, this is an effective processing method when the tendency of the image to be processed is known and it is possible to determine which of the vertical and horizontal directions should be extracted first.

After extracting the dotted lines, the ruled line extracting apparatus searches for a certain range of pixels from the end points of the dotted lines to the left and right in the case of horizontal lines and the upper and lower sides in the case of vertical lines, in the same manner as the straight line search processing P5 and P12. To do. Then, when the search proceeds to the outside of the end point, a new virtual line segment rectangle is created at the place where the search was performed.

In this way, even if the dotted line is not partially extracted due to crushing by searching for pixels within a fixed length from both ends of the dotted line after extracting the dotted line,
It becomes possible to accurately extract the end points. Further, when the position of the end point needs to be corrected, the accuracy of the dotted line extraction is improved by correcting and changing the position.

In the following processing, the rectangle indicating the position of the dotted line is treated in the same manner as the straight line rectangle indicating the position of the already detected solid line. Therefore, information that associates a certain dotted line with the line segments that make up the dotted line is added to one or both of the dotted line and the line segment, and information that describes the positional relationship between the line segments is added to the information of the line segment. . As a result, the position of the dotted line can be described in more detail, and the subsequent character complementing process P22
Etc. are made more efficient.

Next, the ruled line extracting device reintegrates the extracted horizontal straight lines in process P7. The horizontal straight line search process P5 absorbs a blur having a relatively short length, but the straight line integration process P7 is performed to absorb a larger blur that cannot be completely absorbed in the process P5.

For example, from the image after the mask processing shown in FIG. 23 (a), the horizontal straight line extraction unit 15 performs the processing shown in FIG. 23 (b).
Horizontal lines 71, 72, 73, 74, and 75 shown in are extracted. In this case, the horizontal straight line 72 and the horizontal straight line 73 are separated due to partial blurring of the horizontal ruled line. Therefore,
Horizontal lines whose distance in the y direction is less than or equal to a fixed value and distance in the x direction are also less than or equal to the fixed value are integrated into one long horizontal line. As a result, the horizontal line 72 and the horizontal line 73 are integrated into one horizontal line 76, as shown in FIG.

Here, in order to deal with a tilted processed image, the ruled line extraction apparatus first determines the tilt of a straight line having a predetermined length or more extracted before the process P7 by using the position information of the line segments forming the straight line. Etc., and calculate the average value of those slopes. Next, the distance xd between two arbitrary straight lines in the x direction is calculated, and a set of straight lines having a certain value or less is extracted. Then, from the average value of xd and the slope of each set, the corresponding y
The distance yd1 in the direction is calculated, and if the distance yd in the y direction between the straight lines of the set is close to yd1, they are integrated.

For example, as shown in FIG. 24, when the horizontal straight line has an upward slope, the average value of the horizontal straight lines in the image is calculated. Next, the value of yd1 is calculated by multiplying the average value of inclination by the distance xd of the pair of straight lines 77 and 78 in the x direction so that yd1 / xd becomes the same value as the average value of inclination. Then, the distance yd of the straight lines 77 and 78 in the y direction
If is equal to yd1 within a certain allowable range, the straight line 77 and the straight line 78 are integrated. For the calculation of the inclination of the straight line, the coordinate value of the line segment indicated by the diagonal line is used.

Further, some of the plurality of horizontal ruled lines included in the table are faint, and horizontal straight lines 81, 82, as shown in FIG.
When 83, 84, 85, 86, 87 and 88 are extracted, the straight line 83 and the straight line 88 have the same y coordinate value. For this reason, when the straight line integration method of “Pattern region cutout method and pattern extraction device” (Japanese Patent Application No. 7-282171) of the prior application is used, as shown by a broken line B, a straight line 8
3 is integrated with the straight line 88.

However, according to the method of the present invention, an appropriate distance in the y direction is calculated according to the average slope of the surrounding straight line,
As shown by the broken line A, the straight line 83 is considered to be connected to the straight line 87. As a result, the straight line 83 is integrated with the straight line 87, and similarly, the straight line 84 is integrated with the straight line 88.

By the way, originally, no strip-shaped line segment rectangle exists between the straight lines to be integrated.
Therefore, in addition to simply changing the circumscribed rectangle of a straight line, a virtual line segment rectangle is created based on the position information of surrounding line segments in an area where the line segments do not exist in the integrated straight line. deep.

FIG. 26 shows such a line segment creating process. In FIG. 26, when the straight line 91 and the straight line 92 are integrated and the straight line 93 is generated, new line segment rectangles 94 and 95 are created in the region where the line segment rectangle between the straight line 91 and the straight line 92 does not exist. To be done. These line segment rectangles 94, 95
The size and position of the are determined appropriately based on the information of the line segment rectangles around them.

When the target frame is an irregular frame, the lengths and positions of the ruled lines included in the frame are various, and therefore the straight lines separated by a certain distance or more are not integrated. . Also in the vertical straight line integration process P14,
Similar to the case of a horizontal line, vertical lines whose distances in the x direction are less than or equal to a certain value and distances in the y direction are less than or equal to a certain value are integrated, and a virtual line segment is created between them. .

FIG. 27 is a flow chart of straight line integration processing for creating a line segment. When the process is started, the ruled line extraction device first inputs the extracted straight line information (step S51) and integrates straight lines with a certain distance or less (step S52). Next, a range in which no line segment between the integrated straight lines exists is detected (step S53), and a line segment rectangle is virtually created in the detected range (step S5).
4).

Then, the positional relationship between the created line segment and the line segment originally present is described (step S55), and the process is terminated. In step S55, in the case of a horizontal straight line, the number of the line segment adjacent to the right is added to the information of a certain line segment, and in the case of the vertical straight line, the number of the line segment adjacent to the lower line is added to the information of a certain line segment. By doing so, the positional relationship between each line segment is described.

By creating such a line segment, the information can be used when it is necessary to know the position of the straight line accurately in the process after the character complementing process P22 and the like. Therefore, it is possible to prevent the accuracy of the subsequent processing from being lowered due to the absence of the line segment.

The above-mentioned processing is performed regardless of whether the frame structure is regular or irregular. Only when it is known in advance that it has a regular structure, the ruled line extraction device performs table rule processing P15 in order to determine the frame line using the regularity (rule). This process follows the method of “Pattern extraction device and pattern region cutting method” (Japanese Patent Application No. 8-107568) of the previous application, and after the straight line integration process P14, a rectangular region extraction process (process P16).
~ P21) is performed before. The table rule processing P15 is
It includes the following three processes.

The first process is a process for detecting straight line candidates forming a rectangle in a nest as shown in FIG.
In this process, if a large number of horizontal straight lines have substantially the same x-coordinate values of the left and right end points, they are unconditionally regarded as straight line candidates forming a nest regardless of whether or not they are blurred. .

The second process is a process for creating straight line candidates. In this processing, a new straight line is created so that the intervals of all the straight lines having the same end point and the same length detected in the first processing are equal.

The third processing is straight line expansion processing. In this process, when it is known that the object is a regular form, the straight line shortened due to blurring is extended. At this time, based on the straight line after integration, the straight line is extended along the inclination of the target form, and the extended portion is expressed using a short line segment rectangle in the same manner as in the case of straight line extraction. The thickness of the created rectangle is calculated based on the thickness of the straight line rectangle before the extension.

As for the expanded straight line, information indicating that the expansion process has been performed is added to the information of the straight line. As this information, for example, a straight line expansion ratio (ratio of the expanded part to the length of the straight line after expansion) is used. If the expansion ratio is 0, it means that the expansion process is not performed, and if it is larger than 0, it means that the expansion process is performed.

FIG. 28 is a flow chart of such a linear expansion process. When the process is started, the ruled line extraction device first performs a process of extending a short straight line according to the rules of the table (step S61), and calculates the ratio of the total length and the length of the stretched part for each straight line ( Step S6
2). Then, the calculated ratio is added as the information of each straight line (step S63), and the process ends.

Further, in addition to the information indicating that the decompression processing has been performed, the information indicating the type of processing performed when the straight line is extracted, such as the information indicating that the integration processing has been performed, is added to the straight line information. I'll do it. These pieces of information are used as criteria for the subsequent processing. For example, in the layout recognition process P23, the weight of the process of each straight line can be made variable based on the added information, and the process can be performed in more detail, which improves the accuracy.

Here, the method of processing a regular table has been described, but even if the table is not a grid-like table, if some rule is known in advance, the content of the process can be changed each time according to the rule. Processing accuracy and processing speed can be increased. For example, if a rule is given that there is no rectangle that is in a nested state, the nesting process described later can be omitted.

Next, the ruled line extraction device performs processing P16 to P16.
20, a rectangular representation of the frame is performed using the method of “Pattern region cutting method and pattern extraction device” (Japanese Patent Application No. 7-281171) of the previous application. Here, by determining the straight lines forming the frame among the vertical and horizontal straight lines detected so far, and extracting the rectangular range surrounded by the four sides of the top, bottom, left, and right in order from the upper left of the image, Represent the frame structure.

For example, in the case of the table shown in FIG. 29 (a), first, as shown in FIG. 29 (b), it is divided into a plurality of outline frames, and then, as shown in FIG. 29 (c), Each line is divided into smaller rectangles. At this stage, in the case where the rectangle is further subdivided (nested) like the leftmost shaded portion in the second row of the table, the rectangle is regarded as a new table, and FIG. ), It is further subdivided.

First, in process P16, the ruled line extracting apparatus determines a set of two horizontal straight lines forming one line in order from the top of the frame according to a certain rule, and sets them as horizontal frame lines. In the process P17, a gap formed by two horizontal lines forming a pair, that is, a vertical line is extracted for each line, and in the extracted vertical line, both the upper and lower ends are converted into two horizontal frame lines. What has reached is the vertical frame line. Then, a rectangular range surrounded by the determined horizontal and vertical frame lines on all sides is extracted.

The nesting structure processing section 17 carries out nesting processing P18 and P19 when the extracted rectangular range is further subdivided. In the processes P18 and P19, the entire extracted rectangular range is regarded as a new table,
Processing is done recursively. Then, the processes P16 and P17
Similarly, the horizontal frame line and the vertical frame line within the rectangular range are determined. At this time, only the inside of the rectangular range corresponding to the nesting is set to have a shortable blur length.

The ruled line extraction device repeats the processing by the nested structure processing section 17 until a new nested structure does not appear, and finally in process P20, the rectangular representation of the table is completed. In this way, the coordinates of each rectangular area extracted from the reduced image are determined.

Then, in the processing 21, the ruled line extraction device determines the coordinate values of the rectangular area as input pattern 1 which is the original image.
It is converted into the coordinate value in 1 and the area is set as the character area. Then, the inside of the character area in the original image is searched along the four frame lines, it is determined whether a character or the like is in contact with the frame line, and the determination result is passed to the character complementing process P22.

The character recognizing device performs the processes P22 to P25 using the information generated by the ruled line extracting device. Process P2
In No. 2, when the character and the frame line are in contact, the contact character is complemented using the original image, and then the process P is performed.
At 24, character recognition is performed on the character complemented image. When the characters are not in contact with each other, the character area is directly used for character recognition. Further, in the process P23, the layout recognition process using the original image is performed, and the process P23 is performed.
In 25, table recognition is performed based on the results of the processes P23 and P24.

By the way, the four straight lines forming the rectangle representing each item in the table do not necessarily form a normal rectangle. When I input various forms that were actually used with a scanner, I found that the following frequently appeared. (A) A form with rounded corners, called a form with R. (B) When the character area is composed of double borders.

The accuracy of the subsequent character complementing process P22 depends on whether or not the character area has such characteristics. However, the rounded portions and double lines at the corners of the frame are often crushed in the reduced image, and cannot be clearly identified. Therefore, in process 21, based on the data of the original image, it is determined whether or not the character area corresponds to these states, and the information based on the determination result is used as the character complement process P.
I will give it to 22. This determination process is performed as follows.

FIG. 30 shows an example of a part of the form formed by the ruled line with R in (A). In FIG. 30, regarding the portion extracted as the upper left character area, the area 101 surrounding the outside of the four-sided frame line is shown by a solid line, and the area 102 inside the four-sided frame line is shown by a dotted line. . The positions of the boundaries of these areas are
It is determined based on the coordinate information of the line segment rectangle within each frame line.

Regarding the rounded portion 103 at the upper left,
A ruled line pattern partially exists inside the dotted line representing the inner coordinate. Therefore, the ruled line extraction device, when there is a pattern in contact with the position indicating the coordinates inside the frame line at any of the four corners of the character region, the ruled line extraction device surrounds the frame region. Is considered to be a ruled line with R. Then, the information indicating the R ruled line and the information indicating the position thereof (any one of the four corners) are added to the information of the character area.

As described above, when the pattern exists inside the corner portion of the character area in contact with the frame line, it is possible that the corners of the frame surrounding the area are rounded and curved. It is assumed that there is a character, and that information is added to the data created for the character complementing process P22. Thereby, in the process P22, the number of judgment criteria for judging whether the pattern of the character area is a character or a character other than the character is increased, so that the processing accuracy is improved.

FIG. 31 shows a part of the form including the double line of (B). When calculating the position coordinates of the frame line surrounding the character area, if two straight lines are close to each other, such as straight lines 104 and 105 indicated by arrows, it is considered as a double line and The position coordinates of the straight line are calculated separately. Then, the area inside the straight line 105 close to the inside is defined as the character area.

Further, in FIG. 32, the straight line 106 indicated by an arrow corresponds to the frame line of the upper side, and the straight line 107 indicated by another arrow corresponds to a line segment representing a part of the character "7".
These straight lines 106 and 107 are parallel to each other and appear to be double lines, but since the straight line 107 does not reach the left vertical frame line, it is determined that they are not double lines.

As described above, when the two straight lines partially form a double line, it may not be a double frame line in practice, and therefore, the line exists within the character area from end to end. Double lines only if there are two straight lines. The same applies to the vertical straight line. Thus, if there is a straight line that is as long as the frame line inside the character area and is close to the frame line,
Since it is considered that the frame line and the inner straight line form a double line, it becomes possible to describe the ruled line information more accurately.

Further, in the process P21, when a plurality of rectangular regions (character regions) whose sizes are smaller than a certain fixed value are present side by side (a certain number or more) side by side, they are grouped together and the character completion process P22 is performed. The area to pass.

In the character complementing process P22, since the average value of the sizes of the characters existing in the region to be processed is used, the processing accuracy decreases when the region is small and the number of characters is small.
For example, as shown in FIG. 33, when the character “7” in the character area 117 is in contact with the right side, if this area 117 is set as the character area of the process P22 alone, another character exists in that area. Therefore, the average value of the character size cannot be calculated.

Therefore, a plurality of character areas 111, 112, 113, 114, 11 of the same size arranged side by side are arranged.
5, 116, and 117 are put together in one character area 118 as shown by the dotted line, and this is passed as the processing area of the character complementing process P22. As the information of the character area,
The number and position of horizontal ruled lines and vertical ruled lines, rectangular coordinates indicating the area where the character exists, and the like are passed. The same applies when a plurality of small character areas are arranged vertically.

As described above, when a certain number or more of character areas are less than a certain size, they are collectively regarded as one area, and the character completion process P is performed.
By creating the data to be passed to 22, the processing accuracy of the process P22 is improved.

Further, as shown in FIG. 34, when another table 122 exists independently inside the table 121, the character area of the outer table 121 is first extracted as indicated by the dotted line. However, since the inner table 122 is not in contact with the frame line of the table 121, the nested structure processing unit 17 determines that it is not in the nested state, and regards it as a character pattern.

Therefore, in the process 21, if a partial pattern having a size larger than a certain value exists in a certain character area, the ruled line extracting process of FIG. 2 is recursively performed on the partial pattern. To If one or more character areas are extracted from the partial pattern, the pattern is regarded as a table. As a result, the character area of the table 122 is extracted as shown by the dotted line in FIG.

As described above, among the patterns located inside the circumscribed rectangle of the pattern of interest, the patterns having the size of a certain value or more and not forming the frame line are again
By treating this pattern as a table candidate and performing a series of ruled line extraction processing, it is possible to perform processing even when a small table exists independently inside the table.

At this time, whether or not to process the inner table or the number of repetitions of the recursive process can be changed based on the information given from the outside or the information created inside the process. . As described above, by making it possible to change the specification of the recursive process, the structure of the table to be processed can be specified in advance, or not specified.

The information given from the outside means a specific instruction inputted by the user. As the information created inside the process, for example, a threshold value calculated by multiplying the size of the pattern of the entire outer table or the target character area by a certain ratio is used. Then, the inner pattern smaller than this threshold is excluded from the recursive processing target.

Further, in the process 21, it may be determined whether or not to consider it as a subsequent process target depending on the size of the extracted character area. For example, with respect to a character area of a certain size or less, by excluding it from a processing target such as character recognition, even if a line segment on a character is accidentally extracted as a straight line, it is excluded from the ruled line candidates. It becomes possible.

Further, the process P1 of the nested structure processing section 17
Also regarding 8 and P19, the flexibility of processing can be improved by changing as follows. Now, when the inside of the extracted character area is in a nested state, it is called single nesting, and when there is a further nested state in the character area that constitutes the single nesting, Will be called double nesting.

At this time, the nesting structure processing section 17 can change the number of nests to be processed by the information given from the outside or the information created inside the processing. . As a result, the structure of the table to be processed can be specified in advance or not specified.

The information given from the outside is as described above. As the information created inside the process,
For example, a threshold value calculated by multiplying the size of the pattern on the outside table by a fixed ratio is used. Then, a character area smaller than this threshold is excluded from the nesting processing target.

FIG. 36 shows an example of a touching character touching the frame line of the character area. For example, in the case of a character area including such contact characters, the following information is generated in process P21 and passed to the character complement process P22.

(A) Contact position: Of the four sides of the character area,
Information that indicates the sides (top, bottom, left, and right) of the characters that are in contact. (B) Processing range: coordinate information of a rectangular range obtained by enlarging the character area in the contact direction.

(C) Number of vertical ruled lines: The number of vertical ruled lines existing within the processing range. (D) Vertical ruled line information: Ruled line number, the number of vertical line segments that form a ruled line, and information about those vertical line segments. The information of the vertical line segment includes coordinate information of the line segment, types of solid line, dotted line, and the like, flag information indicating whether the line segment is a virtually created line segment, and the like.

(E) Number of horizontal ruled lines: The number of horizontal ruled lines existing in the processing range. (F) Horizontal ruled line information: Ruled line number, the number of horizontal line segments forming the ruled line, and information about those horizontal line segments. The information content of the horizontal line segment is the same as that of the vertical line segment information.

(G) Line number: The line number to which the character area belongs. (H) Column number: The number of the column to which the character area belongs. (I) Nesting flag: Flag information indicating whether or not the character area forms a nested state.

(J) Ruled line with R: Flag information indicating whether a ruled line with R exists in the character area. Areas 131, 132, 133, and 1 indicated by dotted lines in FIG.
Reference numeral 34 represents the processing range of each corresponding character area.
The area 131 corresponds to the case of left contact, the area 132 corresponds to the case of bottom contact, the area 133 corresponds to the case of vertical contact, and the area 134 corresponds to the case of upper contact.

Of these, when the area 131 is enlarged, FIG.
7, and the information of the corresponding character area is as follows. (A) Contact position: left (b) Processing range: coordinate information of the area 131. (C) Number of vertical ruled lines: 1 (d) Vertical ruled line information: The number of vertical line segments is 2. The vertical line segment information is
Information on the line segment rectangles 135 and 136 in FIG. (E) Number of horizontal ruled lines: 0 (f) Horizontal ruled line information: None. (G) Row number: 2 (h) Column number: 1 (i) Nesting flag: 0 (not nesting) (j) Ruled line flag with R: 0 (not ruled line with R) Character area does not include contact characters In this case, the information (a), (c), (d), (e), (f) is omitted from the above information. The processing range of (b) matches the coordinate information of the character area.

FIG. 38 is a flow chart of a process for creating such character area information. When the process starts
The ruled line extraction device first extracts line segment rectangles that form the ruled lines on the four sides of the rectangular range, and determines the coordinate values of the inner rectangular area surrounded by the ruled lines based on the coordinate values thereof (step S7).
1). Then, it is checked whether or not the image pattern exists immediately inside (about 1 pixel or 2 pixels inside) of the obtained coordinate value (step S72).

If the image pattern exists just inside the rectangular area, the input pattern 11 which is the original image before reduction is next generated.
Is processed (step S73), pixels around the corresponding rectangular area are projected in both vertical and horizontal directions, and the position of a straight line is detected based on the projection value of the pixel (step S74). Then, it is determined whether or not there are two straight lines existing closer than a certain distance (step S75), and if there is such a set of straight lines, then whether or not the lengths of these straight lines are substantially equal to each other. Is determined (step S76).

If it is determined that they have the same length within a certain allowable range, the two straight lines are both ruled lines (step S
77), if not determined to be the same length, only the longer straight line is set as the ruled line (step S78). And the horizontal and vertical 2
It is checked whether or not the image pattern is present in the rectangular area of the corner where the ruled lines of the book intersect (step S79). If there are no two straight lines that are close to each other in step S75, the determination in step S79 is performed as it is.

If an image pattern exists at the corner, information indicating that there is a possibility of a ruled line with R is added to the character complementing process P.
It is added to the information to be passed to 22 (step S80). Here, the information is added by setting the R-ruled line flag described above (setting the value to 1). Then, as information to be passed to the character complementing process P22, information such as the position of the ruled line, the direction in which the character pattern and the ruled line are in contact with each other is stored (step S81). The information stored here corresponds to the above information (a) to (j).

In step S79, if there is no image pattern at the corner, the process of step S81 is performed as it is. Then, it is determined whether the processing has been completed for all the character areas (step S82). Step S
In 72, if there is no image pattern just inside the rectangular area, the determination in step S82 is performed as it is.
Then, if an unprocessed character area remains, step S7.
The processing from 1 onward is repeated, and if there is no character area remaining, the processing ends.

As described above, according to the present invention, the processing accuracy is improved by addressing the problems detected in the process of actually processing various forms. However, a method for processing a frame graphic having a shape as shown in FIG. 39, which cannot be expressed in the rectangular range, is a future subject.

The ruled line extraction technique of the present invention is applied not only to a handwritten character recognition device, but also to a technique of cutting out only a figure including a symbol or a character in a broad sense from a pattern in which a straight line overlaps. For example, it can be applied to a printing character recognition device, cutting out patterns of characters, symbols, etc. in recognition of drawings, and separating ruled lines in an image and contact portions of objects, figures, characters and the like. Further, it can be applied when extracting file management information in an electronic filing system from an image.

Further, the shape of the area representing a straight line or a line segment is not necessarily rectangular, and an area of any shape surrounded by a straight line or a curved line can be used. in this case,
If the positions, heights, and widths of straight lines and line segments, the distance between them, and the like are appropriately defined according to the shape to be used, the same processing as in FIG. 2 can be performed.

[0214]

According to the present invention, it is possible to more accurately extract ruled lines from an image with poor image quality that has a blur or a blur. Therefore, the frame line and the character area can be extracted with high accuracy. Further, it is possible to automatically generate the information indicating the state of the extracted ruled line, the detailed information for the character complementing process, etc., and the accuracy of the subsequent process is improved.

[Brief description of drawings]

FIG. 1 is a principle diagram of a ruled line extraction device of the present invention.

FIG. 2 is a functional block diagram of a character recognition device.

FIG. 3 is a configuration diagram of an information processing device.

FIG. 4 is a diagram showing a labeling-processed image.

FIG. 5 is a diagram showing patterns belonging to the same label.

FIG. 6 is a flowchart of a connection pattern extraction process.

FIG. 7 is a diagram showing a mask processed image.

FIG. 8 is a flowchart of mask processing.

FIG. 9 is a diagram showing line segment detection processing.

FIG. 10 is a diagram showing line segment integration processing.

FIG. 11 is a flowchart of straight line detection processing.

FIG. 12 is a diagram showing a line search process.

FIG. 13 is a diagram showing a first straight line after the search.

FIG. 14 is a diagram showing a line segment creating process.

FIG. 15 is a diagram showing a second straight line after the search.

FIG. 16 is a diagram showing line segment expansion processing.

FIG. 17 is a flowchart of a straight line search process.

FIG. 18 is a diagram showing a two-dot broken line.

FIG. 19 is a diagram showing a dotted line composed of large dots.

FIG. 20 is a diagram showing a first dotted line detection process.

FIG. 21 is a diagram showing a second dotted line detection process.

FIG. 22 is a flowchart of a dotted line detection process.

FIG. 23 is a diagram showing a first straight line integration process.

FIG. 24 is a diagram showing second straight line integration processing.

FIG. 25 is a diagram showing a third straight line integration process.

FIG. 26 is a diagram showing a fourth straight line integration process.

FIG. 27 is a flowchart of straight line integration processing.

FIG. 28 is a flowchart of a linear expansion process.

FIG. 29 is a diagram showing a rectangle representation process.

FIG. 30 is a diagram showing a part of a form with R.

FIG. 31 is a diagram showing a double line.

FIG. 32 is a diagram showing a case where the line is not a double line.

FIG. 33 is a diagram showing a processing area for character complement.

FIG. 34 is a diagram showing a character area of an outer table.

FIG. 35 is a diagram showing a character area of an inner table.

FIG. 36 is a diagram showing contact characters.

FIG. 37 is a diagram showing a character area including a contact character.

FIG. 38 is a flowchart of a process of creating information to be passed to the character complement process.

[Fig. 39] Fig. 39 is a diagram illustrating a frame that cannot be represented by a rectangle.

FIG. 40 is a diagram showing a simple tabular frame.

FIG. 41 is a diagram showing a complicated tabular frame.

[Explanation of symbols]

1 pattern extraction means 2-1, 2-2, 2-3, 2-4, 2-5 determination means 3-1 dotted line extraction means 3-2 straight line extraction means 3-3 line extraction means 4-1 and 4-4 Correction means 4-2, 4-5, 4-6 Addition means 4-3 Creation means 4-7 Generation means 5 Table extraction means 6 Frame line extraction means 7 Area extraction means 8 Input means 9 Change means 11 Input pattern 12 Reduction processing Part 13 Connection pattern extractor 14 Mask processor 15 Horizontal straight line extractor 16 Vertical straight line extractor 17 Nested structure processor 31 CPU 32 Memory 33 Input device 34 Output device 35 External storage device 36 Medium drive device 37 Network connection device 38 Photoelectric conversion Device 39 Bus 40 Portable recording medium 40 'Database 41, 42, 43, 44, 45, 46, 47, 48, 4
9 patterns 51, 51 ', 71, 72, 73, 74, 75, 76,
77, 78, 91, 92, 93 Linear rectangles 81, 82, 83, 84, 85, 86, 87, 88, 1
04, 105, 106, 107 Straight lines 52, 55, 56, 57, 58, 59, 57 ', 5
9 ′, 94, 95, 135, 136 Line segment rectangles 53, 54, 60, 61 Search end points 101, 102 Region 103 Corners 111, 112, 113, 114, 115, 116, 1
17,118 Character area 121,122 Table 131,132,133,134 Processing range

Continuation of front page (56) Reference JP-A-2-266479 (JP, A) JP-A-2-210586 (JP, A) JP-A-1-173178 (JP, A) JP-A-5-233873 (JP , A) JP 5-290112 (JP, A) JP 2-232780 (JP, A) JP 6-309498 (JP, A) JP 8-262974 (JP, A) JP 3-250278 (JP, A) JP-A-1-269185 (JP, A) JP-A-4-142679 (JP, A) JP-A-9-309498 (JP, A) JP-A-1-119875 (JP, A) A) Japanese Unexamined Patent Publication No. 1-113883 (JP, A) Japanese Unexamined Patent Publication No. 3-122773 (JP, A) Japanese Unexamined Patent Publication No. 8-115421 (JP, A) Japanese Unexamined Patent Publication No. 8-221506 (JP, A) (58) Investigation Areas (Int.Cl. ⁷ , DB name) G06T 11/60 G06F 17/21 G06T 7/60 G06K 9/00

Claims (22)

(57) [Claims] 1. A straight line extracting means for extracting a straight line pattern from an input image based on connection information of pixels, and information indicating processing performed on the straight line pattern when the straight line pattern is extracted, A ruled line extraction device comprising: an addition unit that adds the pattern information. Wherein said adding means, the information indicating that the line pattern is generated by integrating a plurality of linear patterns, borders according to claim 1, characterized in that added to the information of the linear pattern Extractor. 3. The ruled line extracting apparatus according to claim 1 , wherein the adding means adds information indicating that the straight line pattern has been subjected to expansion processing to the information of the straight line pattern. Configuration and the straight line extracting means for extracting a linear pattern from wherein the input image based on the connection information of the pixel, and generation means for generating a virtually line pattern in the region representing the linear pattern, the straight line pattern And a correction unit for changing the size of the line segment pattern, and a line segment pattern is newly created according to the size of a portion of the region representing the straight line pattern where the line segment pattern forming the straight line pattern does not exist. A ruled line extraction device comprising: a determination unit that determines whether to create or change the size of a surrounding line segment pattern. 5. A frame line extracting means for extracting a frame line pattern from an input image based on pixel connection information, an area extracting means for extracting a rectangular area surrounded by vertical and horizontal frame line patterns, and the rectangular area It is characterized by further comprising: determination means for determining whether or not the corner portion has a specific shape; and addition means for adding the information indicating the possibility of the specific shape to the information for the complementary processing. Ruled line extraction device. Wherein said determining means determines if the pattern in contact with any of the border pattern in the portion of the corner of the rectangular region is present, the corners of the rectangular region is likely to be round angle state The ruled line extracting device according to claim 5, wherein: 7. Based on connection information of pixels from an input image
A frame line extraction means for extracting a frame line pattern, Area to extract rectangular area surrounded by vertical and horizontal frame pattern
Extraction means, Inside the rectangular area, of the same length as the frame
Double if a straight line pattern exists close to the border
Determination means for determining that a line exists, If it is determined that the double line exists, the straight line pattern
Area that does not include
And a generating means for generating information for the complementary processing.
Ruled line extraction device characterized by and. 8. A pattern extracting means for extracting a partial pattern from an input image based on pixel connection information, a table extracting means for extracting a partial pattern as a table candidate as a table pattern, and a frame line pattern from the table pattern. A frame line extracting means for extracting a partial pattern which is located inside the circumscribing rectangle of the table pattern and has a certain size or more, and which is not regarded as a part of the frame line pattern is a new table pattern. The ruled line extracting apparatus further comprises: input means for inputting to the frame line extracting means. 9. A number of iterations of the recursive process by the frame line extracting means and the input means, according to, further comprising a changing means for changing, based on information created by the internal information or given from the outside Item 8. The ruled line extraction device according to item 8 . Wherein said input means, wherein said partial pattern based on the size of, and determines whether the partial pattern regarded as the new table pattern
8. Symbol mounting the ruled-line extraction device. 11. A table extracting means for extracting a table pattern which is a table candidate based on pixel connection information from an input image, a frame line extracting means for extracting a frame line pattern from the table pattern, and vertical and horizontal frame lines. Area extraction means for extracting a rectangular area surrounded by a pattern; input means for inputting a nested rectangular area into the frame line extraction means as a new table pattern; and the frame line extraction means and input means A ruled line extracting device, comprising: a changing unit that changes the number of times the recursive process is repeated based on information provided from outside or information created inside. 12. The input means, based on the size of the rectangular region, according to claim 11 SL, characterized by determining whether the said rectangular region regarded as the new table pattern
The placing of a ruled line extraction device. 13. A recording medium for recording a program for a computer, which has a function of extracting a linear pattern from an input image based on connection information of pixels, and a function for extracting the linear pattern when extracting the linear pattern. A computer-readable recording medium in which is recorded a program for causing the computer to realize a function of adding information indicating the applied processing to the information of the linear pattern. 14. A recording medium for recording a program for a computer, the function of extracting a straight line pattern from an input image based on connection information of pixels, and a virtual line segment pattern in an area representing the straight line pattern. And a function of changing the size of a line segment pattern forming the straight line pattern, and a portion of the area representing the straight line pattern where the line segment pattern forming the straight line pattern does not exist. A computer-readable recording medium recording a program for causing the computer to realize a function of determining whether to newly create a line segment pattern or change the size of a surrounding line segment pattern according to the size. 15. A recording medium for recording a program for a computer, which has a function of extracting a frame line pattern from an input image based on pixel connection information, and a rectangular area surrounded by vertical and horizontal frame line patterns. A function of extracting, a function of determining whether or not a corner portion of the rectangular area has a specific shape, and a function of adding information indicating the possibility of having the specific shape to information for complementary processing, A computer-readable recording medium recording a program for realizing the above-mentioned computer. 16. A recording medium recording a program for a computer, having a function of extracting a partial pattern from an input image based on connection information of pixels and a partial pattern which is a candidate for a table as a table pattern. A function, a function of extracting a frame line pattern from the table pattern, and a part of the partial pattern of a certain size or more located inside the circumscribed rectangle of the table pattern, which is not regarded as a part of the frame line pattern A computer-readable recording medium in which a program for causing the computer to realize a function of recursively processing a pattern as a new table pattern is recorded. 17. A recording medium for recording a program for a computer, which has a function of extracting a table pattern that is a table candidate from an input image based on connection information of pixels, and a frame line pattern from the table pattern. A function to extract, a function to extract a rectangular area surrounded by vertical and horizontal frame line patterns, a function to recursively process a nested rectangular area as a new table pattern, and the frame line extraction function And a computer-readable recording medium recording a program for causing the computer to realize a function of changing the number of times of recursive processing by the input function based on information given from the outside or information created inside. 18. A straight line pattern is extracted from an input image based on pixel connection information, and information representing processing performed on the straight line pattern when the straight line pattern is extracted is used as the straight line pattern information. A ruled line extraction method characterized by adding. 19. A straight line pattern is extracted from an input image based on connection information of pixels, and a line segment pattern forming the straight line pattern is not present in a region representing the straight line pattern. , It is determined whether to virtually create a line segment pattern in the area representing the straight line pattern or to change the size of the surrounding line segment pattern, and based on the determination result, a new line segment pattern is created or an existing line segment pattern is created. A ruled line extraction method, characterized in that the size of the line segment pattern is changed. 20. A frame line pattern is extracted from an input image based on pixel connection information, a rectangular area surrounded by vertical and horizontal frame line patterns is extracted, and a corner portion of the rectangular area has a specific shape. It is determined whether or not there is a possibility that the corner portion has the specific shape, and information indicating the possibility is added to the information for the complementary processing. 21. A partial pattern is extracted from an input image based on pixel connection information, a partial pattern that is a candidate for a table is extracted as a table pattern, and a frame line pattern is extracted from the table pattern. Ruled line extraction characterized by recursively processing, as a new table pattern, partial patterns that are not considered to be part of the frame line pattern among partial patterns of a certain size or larger located inside the circumscribed rectangle Method. 22. A table pattern which is a table candidate is extracted from an input image based on pixel connection information, a frame line pattern is extracted from the table pattern, and a rectangular area surrounded by vertical and horizontal frame line patterns is extracted. However, the nested rectangular area is recursively processed as a new table pattern, and the number of repetitions of the recursive processing is changed based on the information given from the outside or the information created internally. A characteristic ruled line extraction method.