JP2003317107A - Method and device for ruled-line detection - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a ruled-line extracting method for correctly extracting ruled-lines from a read-in image and reproducing them in document data. <P>SOLUTION: When a binary image is inputted, a reduced image is generated by an OR thinning-out process, connection components of black pixels are extracted through a labeling process, and a mask process for extracting straight- line components is carried out. Then segment extraction is carried out and cells are extracted; and ruled lines constituting nesting are determined and a double line is decided. Lastly, oblique lines are extracted. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for accurately extracting ruled lines and the like from an image.

[0002]

2. Description of the Related Art In recent years, with the spread of input devices such as scanners, the opportunities for handling image data have increased, and along with this, OCR (Optical Character) for reading a document.
Reader: Character recognition device or software) and document restoration are increasing in demand.

When a double line is present in a document image, two ruled lines may be close to each other or partially connected, or may be separated from each other. However, the operation is unstable because it may be extracted as one ruled line or may be extracted as two ruled lines.

In the case of a structure in which multiple nests are superposed, since the structure is not known in advance, it is not possible to know how many nested states are present. Therefore, if the number of times of recursive processing for extracting the cells of the nested portion is performed indefinitely, if the ruled lines are erroneously extracted from the characters in the cells, the nested structure may be excessively extracted. Further, if the number of times of recursive processing is fixed, processing becomes impossible due to the structure.

When diagonal lines are drawn between the vertices of cells in the table included in the document image, it is considered that diagonal lines exist on the line connecting the vertices of the cells, and the vertices are connected. When the black pixels on the straight line are equal to or more than a certain ratio of the length of the straight line connecting the vertices, it is determined that there is a diagonal line. However, diagonal lines do not always connect the vertices exactly,
In many cases, the lines are misaligned from the vertices, and if the lines are partially distorted due to paper distortion, etc., the line connecting the cell vertices and the diagonal line do not overlap, and extraction fails. I will end up.

[0006]

As described above, in the conventional technique, when a document including ruled lines such as a table is read and recognized, the ruled lines are correctly recognized and correctly reproduced in the document data. It was difficult.

An object of the present invention is to provide a ruled line extracting method which can correctly extract ruled lines from a read image and reproduce them in document data.

[0008]

A ruled line extracting method of the present invention comprises a ruled line position estimating step of estimating a position of a ruled line from a reduced image of an input image, and a ruled line extracted from the input image to determine an area surrounded by four sides. The step of extracting and the step of extracting the extraction of the area by recursive processing when the area has a nested structure, and the step of extracting the recursive processing more than the size of characters inside or around the area. , When the extracted area becomes smaller, the step of ending the recursive processing, and using the high-resolution input image, project the pixel density of the ruled line at the estimated ruled line position in the ruled line direction, A ruled line determination step of determining how many lines the ruled line is composed of, based on the number of peaks of the pixel density, and a vertex or an apex facing each other in a region surrounded by four sides by the ruled lines extracted from the input image. A step of determining whether or not there is a diagonal component in the periphery of the, and a step of determining whether or not a diagonal line exists in the facing vertex or a region sandwiched by the diagonal component in the periphery thereof. Is characterized by.

The ruled line extracting apparatus of the present invention comprises ruled line position estimating means for estimating the position of a ruled line from a reduced image of an input image, means for extracting a region surrounded by four sides by the ruled line extracted from the input image, and If the area has a nested structure,
Means for extracting the area by recursive processing, and the recursive processing when the area to be extracted is smaller than the size of characters inside or around the area. And a high-resolution input image to project the pixel density of the ruled line at the estimated ruled line position in the ruled line direction, and by the number of peaks of the pixel density,
A ruled line judging means for judging how many lines the ruled line consists of, and whether or not there are diagonal vertices at the opposite vertices of the area surrounded by the ruled lines extracted from the input image on the four sides or around the vertices. And a means for determining whether or not there is a diagonal line in the area between the facing vertices or the diagonal component in the periphery of the facing vertex.

According to the present invention, even if a document image or the like captured by a scanner or the like includes ruled lines, the ruled lines are extracted and judged to be used in combination with character recognition, so that the captured image data is processed by a word processor. The document data can include ruled lines that can be edited with software.
Therefore, it is possible to edit the read image on the information processing apparatus, and the work efficiency when reading a form or the like is improved.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Regarding the double line, first, the input image is reduced so that the image tends to be crushed, and the ruled line candidates are extracted using this reduced image. By using the reduced image, it is easy to extract the ruled line even if there is blurring. This ruled line extraction processing may be performed by any method as long as a ruled line candidate area is extracted. Next, the position of the area of the extracted ruled line candidate in the input image is calculated, and it is determined using the input image whether or not the area of the ruled line candidate is a double line. A determination is made for each small area obtained by finely dividing one ruled line candidate. Black pixel projection processing is performed in the small area in the ruled line direction, and when the peak value is two, it is determined that the small area is a double line. For the determination of the peak value, certain constant values a and b (a ≧ b) are used, and as a result of projection, the number of black pixels is b or less between regions (positions where lines exist) where the number of black pixels is a or more. If there is a region (background region), it is assumed that two lines are present, and this region is a double line. When a certain percentage or more of a certain ruled line candidate is a double line, it is determined that the ruled line candidate is a double line.

If it is determined that the line is a double line, and two lines are spaced apart by a certain distance or more, it is not a double line but two separate ruled lines. Suppose there is.
For a certain interval or more, the size of the characters written in the surroundings is calculated and determined based on the size.

In the case of a structure in which multiple nests are superposed, since the structure is not known in advance, it is necessary to make a decision to stop the recursive processing for extracting the cells of the nest part. The judgment is to stop when the vertical and horizontal size of the nest becomes smaller than a certain size. By determining the size of surrounding characters and determining the size based on the size, it is possible to prevent erroneous extraction of a small nested structure in which characters are not written.

When extracting a diagonal line drawn between the vertices of a cell (a box surrounded by ruled lines) in the table in the image, the entire diagonal line is extracted as one line. The extraction is not performed as a set of small components in the diagonal direction. First, it is determined whether or not there is a diagonal component at and near the vertex of the cell. When there is a diagonal component at or near the apexes facing each other, it is determined whether or not there is a diagonal line between the extracted diagonal components.

The embodiments of the present invention will be described in more detail below. 1. Input image It is assumed that the image to be input is a binary image having no extreme inclination. When an image is input with an inclination, the method of the present embodiment is applied after the image is corrected to a substantially horizontal direction by a known method for correcting the image direction. Further, the color image and the gray scale image are converted into a binary image by a known method, and then the following processing is performed. 2. Linked pattern extraction In order to stably pick up each pattern without depending on the relative relationship of the positions where a plurality of frames are arranged,
In the extraction of the connection pattern, a pattern of black pixels that are connected by 8 connections (connected in the directions of up, down, left, right, upper right, lower right, upper left, lower left) is extracted by labeling (see Japanese Patent Application No. 7-203259). .

FIG. 1 is a schematic diagram for explaining the labeling process. As shown in (a) of the figure, when an image is captured, it is operated horizontally for each line to determine the presence or absence of black pixels, and the existing black pixels are sequentially labeled with numbers or the like. First, when scanning the line 1 in the same figure (a), two independent black pixels are found and labeled with "1" and "2", respectively. Then, if we scan line 2, we still find two independent black pixels,
Each of these black pixels is found in the previous line and is connected to the black pixels with the labels "1" and "2" by eight connections, so that they are not new labels but are connected. Labeled with black pixels. That is, the black pixels in line 2 are also labeled with "1" and "2", respectively. Next, scanning line 3, one black pixel is found, thus giving it a new label "3". However, as shown in FIG. 7B, the label "3"
The black pixels of are connected to the black pixels of the labels “1” and “2”, which are connected by eight connections, so the label “1”,
The label of the black pixel of "2" is rewritten to "3". By doing so, the black pixel shown in FIG.
It is extracted that it is a block of black pixels connected by eight connections.

The above processing is performed on the entire image. 3. Mask processing Mask processing thins the components in the vertical and horizontal directions in order to eliminate extreme diagonal components from the image and to facilitate extraction of long straight lines existing only in the frame. That is, only the vertical and horizontal line segments are used to represent the linear component included in the image. Therefore, a long horizontal line is expressed by connecting a plurality of horizontal line segments. In order to accurately extract the linear component candidates, a low resolution image equivalent to 100 dpi is used for the processing (see Japanese Patent Application No. 7-203259). 4. Line segment extraction mask processing Adjacent projection value (adjacent projection: projection value of a certain row or column is added to the projection values of the surrounding rows or columns, and that value is used as the projection value. It is possible to take a broader view (Japanese Patent Application No. 5-1032).
No. 57)) is calculated and a straight line or a part of the straight line is approximated to a rectangle and detected in the horizontal and vertical directions. The same processing is performed in both vertical and horizontal directions. In this process, the image resulting from the mask process is used.

FIG. 2 is a diagram for explaining the result of the line segment extraction processing. In the figure, a slightly inclined straight line
It is represented by three rectangles. Rectangle is relatively short so that long straight lines are not represented by one rectangle,
Use a horizontally long rectangle. As a result, various horizontal straight lines and vertical straight lines are expressed by connecting the rectangles. 5. In the straight line extraction line segment extraction, a long straight line is detected by integrating neighboring line segments among the extracted rectangle-approximated line segments. The detected straight line is approximated to a rectangle (Japanese Patent Application No. 7-20325).
(See No. 9).

FIG. 3 is a diagram for explaining straight line extraction. When a straight line is extracted as a line segment, it is approximated by a rectangle. When two of these rectangles are in contact with each other or are closer than a predetermined value, it is determined that these two rectangles form part of one straight line. The set of rectangles thus determined will eventually cover one straight line as a whole. One straight line detected in this way is approximated using a rectangle including the whole. 6. Extraction of Double Lines As a method of determining double lines, images with different resolutions are used for each purpose. A candidate position of a short line segment is extracted using the reduced image, and then the original image is used to determine whether or not the line segment is a double line. The judgment is made by projection. After that, the integration result is extracted as a double line.

FIG. 4 is a diagram for explaining the OR thinning processing. As shown in FIG. 3A, the image is halved.
When the pixel size is reduced to, in OR thinning out, the pixel values of a group of four pixels are ORed and replaced with one pixel. That is, if at least one of the four pixels contains a black pixel, the reduced pixel is a black pixel.

FIG. 3B is a diagram for explaining an example of OR thinning processing. It is assumed that the image on the left side of FIG. This image is divided into four pixels, and the pixel values are ORed in each. Then, of the four pixels,
A section including at least one black pixel is replaced with one black pixel after reduction. Further, only when all four pixels are white, one pixel after reduction is made white.
As a result, a reduced image with a rough image is obtained as shown on the right side of FIG.

This OR thinning process uses the result already extracted in the line segment extraction process. Next, the original image corresponding to the extracted candidate position is investigated, and whether or not the position is a double line is determined in detail. Judgment is performed for each of the small areas (called line segments) that form the ruled line, and the method is determined by projecting the distribution of black pixels in the line segment in the direction perpendicular to the line segment in the direction parallel to the line segment. To do. When a projection value is taken in a straight line direction within a line segment and there are two or more locations where the projection value is above a certain value, and there is a location where the projection value is below a certain value between them, The line segment is determined to be a double line.

FIG. 5 is a diagram for explaining the black pixel projection processing. The distribution of black pixels in the direction perpendicular to the line segment is added within the rectangle in the rectangle obtained by piecewise approximating the straight line, and the histogram of the black pixel distribution is obtained. Then, a part of the straight line is a portion where the distribution of black pixels is larger than a predetermined value determined through experiments, etc., and is blank when the distribution of black pixels is smaller than the predetermined value which is also determined through experiments. Is said to be. From this, if there are two peaks in the distribution of black pixels, it can be determined that there is a double line. It will be appreciated that the respective predetermined values (threshold values) used for determining the peak and the valley of the histogram are appropriately determined by those skilled in the art using the present embodiment, and can be determined by experiment. However, the threshold value may be determined by a method other than experiment.

Further, the line type may change within one straight line. Since the point where the line type changes is the cell boundary, the end point position is extracted using the position where the ruled lines intersect. When it is determined that a part of a certain ruled line is between the intersections of the ruled lines, a double line is again determined between the part determined as the double line and the intersection of the ruled lines. When it is determined that the length of the ruled line is equal to or greater than the length between the intersections of the ruled lines is a double line, it is determined that the line between the intersections of the ruled line is constituted by the double line.

Further, even if the double line is partially crushed and is in a blackened state, there is a portion judged to be the double line with respect to the length of the ruled line of interest. If it is a certain value or more, it is considered as a double line, and even if the line type changes within one ruled line by dividing the ruled line at the part where the thickness changes, You can judge.

The constant value for determining the double line described above is also appropriately determined by those skilled in the art. Further, when the line intervals are large, the double line needs to be extracted as two completely separated lines. In that case, if the line spacing is less than a certain value, the two ruled lines are regarded as one double line. At this time, the threshold value of the interval is calculated from the surrounding character size. If the vertical or horizontal size of the surrounding characters is greater than or equal to a certain ratio, then two lines are used.

In the above embodiment, the case of the double line is shown, but the same applies to the triple line and above. 7. Cell Extraction A cell is extracted using the ruled line candidate determined in the above process. Any method may be used for extracting cells, which are areas surrounded by straight lines on four sides (see Japanese Patent Application No. 7-203259). 8. Nested Structure Ruled Line Cell Extraction FIGS. 6 and 7 are diagrams illustrating a method of extracting a ruled line portion of a nested structure.

The nested portion is dealt with by treating the inside of the cell as a table and performing recursive processing. In the past, there was a limit on the number of recursion. The reason for setting the limit is that if the number of times of recursive processing is simply increased, a small nested structure due to ruled lines erroneously extracted from the character portion is excessively extracted. Therefore, since three or more nests are supported (three thick lines in FIG. 6 are nested), the character size around the cell of interest is compared with the length of the ruled line targeted for nesting, Make it possible to automatically determine whether to stop recursive processing.

An example is shown in FIG. Lines indicated by arrows in the figures (a) and (b) are single nests, and both figures show the same format. However, FIG. 9A shows a ruled line erroneously extracted from a part of a character, and FIG. 9B shows a ruled line. In this case, if the line indicated by the arrow is a ruled line forming a nest, in the case of FIG. 9A, the nest cell is too small as compared with the character size, so it seems unsuitable. In FIG. 6B, since the characters are small, the size of the nested cells separated by the line indicated by the arrow is sufficiently larger than the character size and is appropriate. Therefore, the cell size that can be created by performing the nesting process is compared with the value of the result of estimating the size of the surrounding characters, and if the cell width is larger than a certain percentage of the character size, the process is performed as a nest, If the cell is too small compared to the character size, it is assumed that the recursive process will not perform nested extraction.

FIG. 8 is a diagram for explaining another extraction method of the nested structure. When a pattern such as a part of a character is in contact with a ruled line, if a part of the character pattern in contact with the ruled line is in the same direction as the ruled line as shown in FIG. As shown in FIG. 7B (FIGS. 8B and 8C, a part of FIG. 8A is cut out), and the character size is calculated assuming that the patterns are originally connected.
As a result, the character size becomes like the rectangle in FIG. 7C, and the average character size is calculated using the result.

That is, when the character size thus obtained is smaller than the width of the cell formed by the ruled line, it is determined that the ruled line is erroneously extracted. In the case of FIG. 8, since the horizontal line of “sa” is continuous, it is determined that the line is mistakenly extracted. As described above, in the determination of the nested structure, the method described with reference to FIG. 7 determines whether or not the ruled line is erroneously detected from the average size of the characters outside the cell due to the ruled line in question. , This doesn't work if the cell in question contains less than the surrounding text. Therefore, as described with reference to FIG. 8, the character included in the cell in question is extracted, and its size is used to determine whether or not the ruled line is erroneously detected. Be accurate. 9. Diagonal Line Extraction FIGS. 9 to 11 are diagrams for explaining the oblique line extraction processing.

The diagonal line is a line dividing the inside of the cell obliquely as shown in FIG. Extraction is performed by a diagonal line extraction filter having the same slope as the slope connecting the cell vertex coordinates. The diagonal line extraction processing unit is set to a size that is slightly larger than the approximate character. As a diagonal line extraction filter,
It is conceivable that the area is an elongated rectangular area inclined by a predetermined angle.

However, since the start and end points of the diagonal line often do not coincide with the cell vertices, the filter position is changed near the cell vertex to find the diagonal line end. If there is a diagonal line at the position facing each other near the apex, it is considered that the cell may include a diagonal line.If there is a diagonal line in the center of the cell, check if there is a diagonal line in the center. As long as the cell is judged to include diagonal lines. As for the diagonal line extraction from the central portion, as shown in FIG. 9, the diagonal line is extracted in the diagonal line extraction unit. As a result, if the diagonal line extraction unit continuously forms a long diagonal line, a long diagonal line exists. I will.

In the diagonal line extraction, the length of a diagonal line filter having a certain fixed size is used as a unit to divide into small regions forming diagonal lines for extraction, and a straight line having a certain known angle is extracted. For extraction, a pixel search between certain two points is performed according to the set angle, and a projection value in an oblique direction is obtained. If the calculated value is greater than or equal to a certain value, it is determined that there is a diagonal line connecting the two points. Regarding the angle, it is estimated from the coordinates of the vertices of the cell what the angle will be when the vertices of the opposing cells are connected, and the pixel search is performed along the angle. Even if each point for pixel search is a white pixel,
If there is a black pixel within a certain range, the pixel of interest is regarded as black.
As a result, when the black pixel density is equal to or higher than a certain level, it is assumed that a diagonal line exists.

FIG. 10 is a diagram for explaining the diagonal line extraction processing when a cell has rounded corners. In many cases, cells have rounded corners, and the end points of diagonal lines are often located at the ends of the rounded corners instead of at the center. Therefore, a cell having a rounded corner is extracted on the assumption that not only the center of the rounded corner, which is the apex of the cell, but also the end point of the diagonal line at the end. For information about rounded corners, use the location and the approximate size of the rounded corner, and if the center of the rounded corner is the starting point,
Extraction is performed assuming a total of three types of diagonal lines when starting points are both ends of a circular arc of a rounded corner.

As shown in FIG. 10, the drawing shows a case where the right end point of the circular arc of the round corner of the cell has the end point of the diagonal line. It is estimated that the angle of the diagonal line is the angle formed by the line connecting the corners of the arc to be extracted and the facing angle. Thereby, the diagonal line at the corner can be detected, and the middle part of the diagonal line is detected by the method described above.

In the above method, since the black pixel density having the set direction between the two points is used, if a cell having a high black pixel density such as a black-and-white inversion cell is detected, it is erroneous. It will be extracted. To prevent this, check the pixel density around the part extracted as a diagonal line,
If the surrounding area is not filled, it is determined to be a diagonal line. Existence of an oblique line having the same angle as the inclination obtained from the cell vertices, away from the area determined to be an oblique line (eg, the range of 1/3 to 2/3 of the side length of the cell, see FIG. 11) To judge. As a result, when the pixel density is low and there is a portion where there is no diagonal line, it is determined that the detected black pixel is finally a diagonal black line pixel, and when the surrounding pixel density is high, It is determined that the cell of interest has a high black pixel density and is not a cell including diagonal lines.

In the present embodiment, since it is determined whether or not a diagonal line exists between two certain points, it can be determined even when the diagonal line intersects in the cell, and the diagonal line and the character overlap. Even if the diagonal line is present, the diagonal line can be extracted because the black pixel densities of the diagonal line portion and the character portion differ. It also has the following features. 1) Create a reduced image for a black and white binary input image,
A means for estimating the ruled line position using a reduced image, and a means for determining the number of ruled lines based on the number of places where the pixel density in the ruled line direction in the estimated ruled line position has a peak using a higher resolution image By holding, it is possible to extract a blurred or partially crushed double line. 2) In 1) above, the surrounding character size is estimated, and a certain percentage of the vertical or horizontal length is set as the maximum value of the spacing between ruled lines considered as double lines. If it is equal to or larger than the value, it is assumed that there are two ruled lines.
You can judge whether it is a small cell or a double line. 3) From the extracted ruled line candidates, a region (cell) surrounded by four sides by ruled lines is extracted, and in the process of determining the ruled lines, it is possible to recursively process a portion having a nested structure. It has a structure and compares the vertical or horizontal length of the written character with the side length of the cell, and the side length of the cell is greater than the threshold calculated from the vertical or horizontal length of the written character. If it is small, the recursive process is stopped, and if it is large, it is judged to continue, and even if nesting is multiplexed by extracting cells with a nested structure,
Even with a simple structure, ruled lines and cells can be accurately extracted. 4) For a black-and-white binary input image, the diagonal lines drawn between the vertices of the cells in the table that face each other are extracted, so it is determined whether there is a diagonal component in the vertices that face each other and the surrounding area. If there is a diagonal component at each of the vertices facing each other and their surroundings, a diagonal line does not exist by having a means for determining whether or not a diagonal line exists in the area sandwiched between them. The cells can be judged early in the processing, and the processing time can be shortened. 5) Regarding the means for extracting the diagonal component in 4) above, a certain small rectangle is used as the unit area, and the aspect ratio is calculated from the inclination estimated from the vertex information of the target cell. The pixel density on the straight line connecting the vertices of the unit area is calculated. If the pixel density is a certain value or more, it is assumed that the unit area has an oblique component, By extracting the direction component as a set of unit areas, it is possible to process even if there is some distortion or blurring.

12 to 19 are flowcharts of the processing according to the embodiment of the present invention. FIG. 12 is a flowchart showing the flow of overall processing. First, in step S1, an image is input. The image is basically a binary image, but a color image or a grayscale image may be binarized. Next, in step S2, a reduced image is created by OR thinning or the like. In step S3, a labeling process is performed to detect a block of black pixels. Next, the masking process described above is performed (step S4), and the line segment is extracted (step S5). Then, a double line candidate is extracted in step S6, and a straight line is extracted in step S7. In step S8, cells are extracted, and in step S9, the ruled lines forming the above-mentioned nesting are determined. Then, in step S10, the double line is determined, and step S11
In, the diagonal line is extracted.

FIG. 13 is a flow chart showing the double line candidate extraction processing. First, after the line segment extraction is completed, black pixel projection processing is performed in the ruled line direction for each line segment in step S20. In step S21, it is determined whether or not there are two peak values of the projection result. When determining the triple line or higher, it is determined whether or not there are three or more peak values. If the determination in step S21 is no, the target line segment is not a double line. Step S21
If the determination is YES, in step S22,
It is determined whether or not the peak value projection value is a certain value or more.
If the determination is NO in step S22, it is determined that the target line segment is not a double line. Step S2
If the determination in step 2 is YES, it is determined in step S23 whether or not the projection value between the peak values is equal to or less than a certain value. If the determination in step S23 is no, the target line segment is not a double line. Step S23
If the determination is YES, it is assumed that the target line segment is a double line.

Here, when extracting candidates of triple lines or more, not only the number of peak values but also the values of all peak values are not less than a certain value or all the valley values between the peak values are not more than a certain value. Etc.

FIG. 14 shows the first processing of the double line determination processing. First, in step S25, cells are extracted. In step S26, each side of the cell is processed. In step S27, it is determined whether or not there is a certain double-line area in the side. Step S2
If the determination in 7 is NO, it is determined that the side to be processed is not a double line, and if YES is determined, the side to be processed is a double line.

FIG. 15 shows the second processing of the double line determination processing. In step S30, two adjacent ruled lines are selected. In step S31, the average size of the characters in the cell formed by each ruled line is calculated. Then, in step S32, it is determined whether the interval between the two ruled lines to be processed is larger than the average character size. When the determination in step S32 is YES, it is determined that the two ruled lines are independent ruled lines. If the determination in step S32 is no, in step S33, it is determined that the two ruled lines form a double line, and the cells created between the two ruled lines are deleted.

FIG. 16 shows an oblique line extraction process. First, in step S40, a processing target cell is set,
In step S41, the inclination of the diagonal line is estimated from the cell vertex coordinates. In step S42, the unit for extracting diagonal lines is set, and in step S43, diagonal line detection of unit length is performed at cell vertices. Next, step S
At 44, it is determined whether or not there are diagonal line components near the facing vertices. If the determination in step S44 is no, it is determined that there is no diagonal line. When the determination in step S44 is YES, step S4
In step 5, the diagonal component on the straight line connecting the vertices is extracted. In step S46, it is determined whether or not the diagonal component on the straight line connecting the vertices has a ratio higher than a certain value. If the determination in step S46 is no, it is determined that there is no diagonal line. When the determination in step S46 is YES, it is determined that a diagonal line exists.

FIG. 17 shows an oblique component extraction process. In step S50, a fixed length is determined as the diagonal line extraction unit. In step S51, the inclination of the straight line connecting the vertices of the cells is taken as the inclination of the oblique component. In step S52, two end points are set as regions for examining the presence of the diagonal component. In step S53, black pixels are projected in the tilt direction on the area between the two points. In step S54, it is determined whether or not it is found that a certain number or more of black pixels are present as a result of projection. If the determination in step S54 is yes, it is determined that there is a diagonal component. If the determination in step S54 is no, in step S55, the positions of the two points are moved left and right within a certain range, and black pixel projection is performed. Step S56
At, as a result of projection, it is determined whether or not there is a certain number of black pixels or more. If the determination in step S56 is yes, it is determined that there is a diagonal line component. When the determination in step S56 is NO, step S57
In, the position of two points is moved up and down within a certain range,
Perform black pixel projection. In step S58, as a result of the projection, it is determined whether or not there are more than a certain number of black pixels. If the determination in step S58 is yes, it is determined that there is a diagonal line component. If the determination in step S58 is no, it is determined that there is no diagonal line component.

FIG. 18 shows a ruled line determining process for forming a nest. In step S60, horizontal ruled line candidates are selected. In step S61, the average size of characters within a certain range centered on the selected horizontal ruled line candidate is calculated. The process of step S61 may be the process described later with reference to FIG. In step S62, it is determined whether the interval between the paired horizontal ruled lines is larger than the average character size. If the determination in step S62 is no, in step S65, the next horizontal ruled line pair is selected. If the determination in step S62 is YES, it is determined in step S63 that it is a horizontal ruled line, and then in step S64, the vertical ruled line is processed. Step S64
The process of step S60 is to perform the processes of steps S60 to 63 and step S65 for the vertical ruled line.

FIG. 19 is a flow chart for explaining another method of the process of step S61 of FIG. In step S66, it is determined whether or not there is a pattern in contact with the selected horizontal ruled line candidate. If the determination in step S66 is no, in step S70, the average size of characters within a certain range centered on the selected horizontal ruled line candidate is calculated, and the process proceeds to step S62 in FIG. When the determination in step S66 is YES, step S67
In, in the pattern that touches the horizontal ruled line candidate, the portions that sandwich the ruled line and have the same direction form the same character.
Then, in step S68, the character size is calculated on the assumption that parts forming the same character are connected. Further, in step S69, the calculated character size is set as the average character size, and the process proceeds to step S62 in FIG.

FIG. 20 is a diagram for explaining the hardware environment required when the method of the embodiment of the present invention is implemented by a program. CPU 11 connected to bus 10
Reads the basic software such as BIOS from the ROM 12 which is also connected to the bus 10 when the information processing device 8 is activated, and makes the information processing device 8 available to the user. The program that realizes the embodiment of the present invention is
Although it may be stored in the ROM 12, it is generally stored in the storage device 17 such as a hard disk, or a portable recording medium 19 such as a flexible disk, a CD-ROM, a DVD or the like. The program stored in the portable recording medium 19 is stored in the RAM 1 via the reading device 18.
3 and is executed by the CPU 11. The program stored in the storage device 17 is still the bus 1
It is copied to the RAM 13 via 0 and executed by the CPU 11.

The input / output device 20 includes general input / output devices such as a display, a keyboard, a template, and a mouse, but in order to use the information processing device 8 as an image reading device, a scanner or the like is provided as an input device. There is a need.

The communication interface 14 connects the information processing apparatus 8 and the database of the information provider 16 via the network 15 so that data can be exchanged. The program can be downloaded from the database of the information provider 16 to the information processing device 8 for use, or the program can be used while the information processing device 8 is connected to the database of the information provider 16 and other information processing devices. The program may be executed under a network environment.

(Supplementary Note 1) Using the ruled line position estimation step of estimating the position of the ruled line from the reduced image of the input image and the high resolution input image, the pixel density of the ruled line at the estimated ruled line position is projected in the ruled line direction. A program for causing an information processing apparatus to implement a ruled line extraction method, which comprises a ruled line determination step of determining how many lines a ruled line is composed of, based on the number of peaks of the pixel density.

(Supplementary Note 2) The estimated ruled line position is calculated by comparing the size of the character around the estimated ruled line position with the width between the lines when the ruled line is composed of a plurality of lines. 2. The program according to appendix 1, further comprising a step of determining whether the line in 1 is a ruled line composed of a plurality of lines or a line in which separate lines are lined up.

(Supplementary Note 3) When it is determined that the ruled line at the estimated ruled line position is a ruled line composed of a plurality of lines over a predetermined length or more, the entire ruled line is separated from the plurality of lines. The program according to Appendix 1, further comprising a step of determining that the ruled line is

(Supplementary Note 4) A step of extracting a region surrounded by four ruled lines extracted from the input image, and when the region has a nested structure, the extraction of the region is performed by recursive processing. A ruled line extracting method comprising: a step; and a step of terminating the recursive processing when the area to be extracted is smaller than the size of a character inside or around the area. A program that causes an information processing device to realize.

(Supplementary Note 5) A step of determining whether or not an apex facing the area surrounded by the ruled lines extracted from the input image on all sides or a diagonal component around the apex, and the apex facing the apex or its A program for causing an information processing apparatus to realize a ruled line extraction method, which comprises a step of determining whether or not a diagonal line exists in a region sandwiched by the diagonal direction components in the periphery.

(Supplementary Note 6) In the presence / absence determining step of the oblique direction component, a step of calculating a pixel density in the unit area on a straight line connecting the vertices with a certain rectangular area as a unit area, and the pixel density The program according to Supplementary Note 5, further comprising: a step of determining that a diagonal component exists at the apex when a certain value or more.

(Supplementary Note 7) The extraction of the diagonal line in the region sandwiched by the vertices is performed by determining whether or not the diagonal direction component is included in the set of the unit regions on the estimated diagonal line. The program according to appendix 6, wherein the program is opened.

(Supplementary Note 8) The program according to Supplementary Note 5, wherein the apex is a rounded corner portion, and the diagonal direction components are detected at the center and both end points of the circular arc of the rounded corner portion.

(Supplementary Note 9) A ruled line position estimating step of estimating the position of a ruled line from a reduced image of the input image, a step of extracting a region surrounded by four ruled lines extracted from the input image, and a nested structure of the region. , The step of extracting the region by the recursive process and the recursive process are performed so that the extracted region is smaller than the size of the character inside or around the region. In this case, the pixel density of the ruled line at the estimated ruled line position is projected in the ruled line direction using the step of ending the recursive process and the high-resolution input image, and the ruled line is determined by the number of peaks of the pixel density. There is a ruled line determination step for determining how many lines are formed, and there are diagonal components in the opposite vertices of the area surrounded by the ruled lines extracted from the input image or around the vertices. Implementing a ruled line extraction method in an information processing device, the method comprising: determining whether or not there is a diagonal line in a region sandwiched by the diagonal direction components in the opposite vertices or in the periphery thereof. Program to let.

(Supplementary Note 10) The estimated ruled line position is compared by comparing the size of the character around the estimated ruled line position with the width between the lines when the ruled line is composed of a plurality of lines. 10. The program according to appendix 9, further comprising a step of determining whether the line in is a ruled line composed of a plurality of lines or a line in which separate lines are arranged.

(Supplementary Note 11) When the ruled line at the estimated ruled line position is determined to be a ruled line composed of a plurality of lines over a predetermined length or more, the entire ruled line is separated from the plurality of lines. 10. The program according to note 9, further comprising a step of determining that the ruled line is

(Supplementary Note 12) In the presence / absence determining step of the diagonal component, a certain rectangular area is set as a unit area.
Calculating a pixel density in the unit area on a straight line connecting the vertices, and determining that an oblique component exists at the vertices when the pixel density is a certain value or more. Supplementary Note 9 characterized by further comprising
The program described in.

(Supplementary note 13) The extraction of the diagonal line in the region sandwiched by the vertices is performed by determining whether or not the diagonal direction component is included in the set of the unit regions on the estimated diagonal line. 13. The program according to note 12, wherein the program is executed.

(Supplementary Note 14) The program according to Supplementary Note 9, wherein the apex is a rounded corner portion, and diagonal direction components are detected at the center and both end points of the circular arc of the rounded corner portion.

(Supplementary Note 15) A ruled line position estimating step of estimating the position of the ruled line from the reduced image of the input image, a step of extracting a region surrounded by four ruled lines from the input image, and a nesting structure of the region. , The step of extracting the region by the recursive process and the recursive process are performed so that the extracted region is smaller than the size of the character inside or around the region. In this case, the pixel density of the ruled line at the estimated ruled line position is projected in the ruled line direction using the step of ending the recursive process and the high-resolution input image, and the ruled line is determined by the number of peaks of the pixel density. There is a ruled line judgment step for judging how many lines are formed, and a diagonal direction component exists at the opposite vertices of the area surrounded by the ruled lines extracted from the input image or around the vertices. A step of determining Luke, ruled line extraction method comprising the steps of: determining whether said facing apex or diagonal lines in areas which sandwich the the oblique direction component in the surrounding exists.

(Supplementary Note 16) The estimated ruled line position is compared by comparing the size of the character around the estimated ruled line position with the width between lines when the ruled line is composed of a plurality of lines. 16. The ruled line extracting method according to appendix 15, further comprising a step of determining whether the line in 1 is a ruled line composed of a plurality of lines or a line in which separate lines are arranged.

(Supplementary Note 17) When the ruled line at the estimated ruled line position is determined to be a ruled line composed of a plurality of lines over a predetermined length or more, the entire ruled line is separated from the plurality of lines. 15. The ruled line extracting method according to appendix 15, further comprising a step of determining that the ruled line is

(Supplementary Note 18) In the presence / absence determining step of the oblique component, a certain rectangular area is set as a unit area.
Calculating a pixel density in the unit area on a straight line connecting the vertices, and determining that an oblique component exists at the vertices when the pixel density is a certain value or more. Additional Note 1 characterized by further comprising
The ruled line extraction method described in 5.

(Supplementary Note 19) The extraction of the diagonal line in the region sandwiched by the vertices is performed by determining whether or not the diagonal direction component is included in the set of the unit regions on the estimated diagonal line. The ruled line extracting method according to appendix 18, wherein the ruled line extracting method is performed.

(Additional remark 20) The apex is a rounded corner portion, and the diagonal line component is detected at the center and both end points of the circular arc of the rounded corner portion. Method.

(Supplementary Note 21) Ruled line position estimating means for estimating the position of the ruled line from the reduced image of the input image, and means for extracting a region surrounded by four sides by the ruled line extracted from the input image,
When the area has a nested structure, means for extracting the area by recursive processing, and the area to be extracted by the recursive processing rather than the size of characters inside or around the area When is smaller than, using the means to end the recursive processing and the high-resolution input image,
The pixel density of the ruled line at the estimated ruled line position is projected in the ruled line direction, and the ruled line judgment means for judging how many lines the ruled line is made from is extracted from the input image based on the number of peaks of the pixel density. Means for determining whether or not there is an oblique direction component in the opposite vertices of the area surrounded by the ruled lines or around the vertices, and an oblique line in the area between the opposite vertices or the oblique direction component in the periphery And a means for determining whether or not there is a ruled line extraction device.

(Supplementary Note 22) The estimated ruled line position is calculated by comparing the size of the character around the estimated ruled line position with the width between lines when the ruled line is composed of a plurality of lines. 22. The ruled line extracting apparatus according to appendix 21, further comprising means for determining whether the line in (1) is a ruled line composed of a plurality of lines or a line in which separate lines are lined up.

(Supplementary Note 23) When the ruled line at the estimated ruled line position is determined to be a ruled line composed of a plurality of lines over a predetermined length or more, the entire ruled line is separated from the plurality of lines. 22. The ruled line extracting apparatus according to appendix 21, further comprising means for determining that the ruled line is

(Supplementary Note 24) In the means for determining the presence / absence of the diagonal component, a unit for calculating a pixel density in the unit area on a straight line connecting the vertices with a certain rectangular area as a unit area, and the pixel The ruled line extraction device according to appendix 21, further comprising: a unit that determines that a diagonal component exists at the vertex when the density is equal to or higher than a certain value.

(Supplementary Note 25) The extraction of the diagonal line in the region sandwiched by the vertices is performed by judging whether or not the diagonal direction component is included in the set of the unit regions on the estimated diagonal line. The ruled line extracting device according to appendix 24, which is characterized in that:

(Supplementary note 26) The apex is a rounded corner portion, and the diagonal line components are detected at the center and both end points of the circular arc of the rounded corner portion. apparatus.

[0077]

According to the present invention, a ruled line is accurately extracted from a document image including a table and the like, which is taken in as image data, and not only character data but also table data is accurately reproduced. Can be made available on the device.

[Brief description of drawings]

FIG. 1 is a diagram schematically illustrating a labeling process.

FIG. 2 is a diagram illustrating a result of performing a line segment extraction process.

FIG. 3 is a diagram illustrating straight line extraction.

FIG. 4 is a diagram illustrating OR thinning processing.

FIG. 5 is a diagram illustrating a black pixel projection process.

FIG. 6 is a diagram (part 1) explaining a method of extracting a ruled line portion of a nested structure.

FIG. 7 is a diagram (part 2) explaining a method of extracting a ruled line portion of a nested structure.

FIG. 8 is a diagram illustrating another extraction method of a nested structure.

FIG. 9 is a diagram (part 1) for explaining the oblique line extraction processing.

FIG. 10 is a diagram (part 2) for explaining the oblique line extraction processing.

FIG. 11 is a diagram (part 3) for explaining the oblique line extraction processing.

FIG. 12 is a flowchart (part 1) of processing according to the embodiment of the present invention.

FIG. 13 is a flowchart (part 2) of the process according to the embodiment of the present invention.

FIG. 14 is a flowchart (No. 3) of processing according to the embodiment of the present invention.

FIG. 15 is a flowchart (part 4) of the process according to the embodiment of the present invention.

FIG. 16 is a flowchart (No. 5) of processing according to the embodiment of the present invention.

FIG. 17 is a flowchart (No. 6) of processing according to the embodiment of the present invention.

FIG. 18 is a flowchart (No. 7) of processing according to the embodiment of the present invention.

FIG. 19 is a flowchart (No. 8) of processing according to the embodiment of the present invention.

FIG. 20 is a diagram illustrating a hardware environment required when the method of the embodiment of the present invention is implemented by a program.

[Explanation of symbols]

10 bus 11 CPU 12 ROM 13 RAM 14 Communication interface 15 network 16 Information provider 17 Storage 18 Reader 19 Portable recording medium 20 I / O device

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Satoshi Naoi             4-1, Kamiodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa             No. 1 within Fujitsu Limited F term (reference) 5B029 BB02 CC27 EE12 EE16 EE18                 5L096 AA07 EA03 EA04 FA03 FA04                       FA10 FA12 FA13 FA16 FA18                       FA32 FA36 FA52 FA62 FA64                       FA66 FA67 FA69 GA10 GA15                       GA23 GA34 GA51 GA55

Claims (10)

[Claims] 1. A ruled line position estimating step of estimating the position of a ruled line from a reduced image of an input image, and using a high-resolution input image, projecting the pixel density of the ruled line at the estimated ruled line position in the ruled line direction, A program for causing an information processing apparatus to implement a ruled line extraction method, which comprises a ruled line determination step of determining how many lines a ruled line is composed of, based on the number of peaks of pixel density. 2. The estimated ruled line position is obtained by comparing the size of a character around the estimated ruled line position with the width between lines when the ruled line is composed of a plurality of lines. The program according to claim 1, further comprising a step of determining whether the line is a ruled line composed of a plurality of lines or a line in which separate lines are arranged. 3. The ruled line at the estimated ruled line position is
The method further comprising the step of determining that the entire ruled line is a ruled line including the plurality of lines when it is determined that the ruled line includes a plurality of lines over a predetermined length or more. The program according to item 1. 4. A step of extracting a region whose four sides are surrounded by ruled lines extracted from an input image, and a step of extracting the region by recursive processing when the region has a nested structure. , The recursive process is terminated when the region to be extracted is smaller than the size of the character inside or around the region, and the recursive process is terminated. A program to be implemented by the processing device. 5. A step of determining whether or not an oblique vertex component exists in the area surrounded by the ruled lines extracted from the input image on the four sides or around the vertex, and the facing vertex or the surrounding area of the vertex. A program for causing an information processing apparatus to realize a ruled line extraction method comprising: a step of determining whether or not a diagonal line exists in a region sandwiched by a certain diagonal component. 6. The step of determining the presence or absence of the diagonal direction component, the step of calculating the pixel density in the unit area on a straight line connecting the vertices with a certain rectangular area as the unit area, and the pixel density The program according to claim 5, further comprising: a step of determining that a diagonal component exists at the apex when the value is equal to or more than a certain value. 7. The extraction of diagonal lines in the region sandwiched by the vertices is performed by determining whether or not a diagonal component is included in the set of unit regions on the estimated diagonal line. The program according to claim 6, characterized in that. 8. A ruled line position estimating step of estimating a position of a ruled line from a reduced image of an input image, a step of extracting an area surrounded by four sides of the ruled line extracted from the input image, and the area having a nested structure. If the extracted area is smaller than the size of the character inside or around the area, the step of extracting the area by the recursive processing, and Then, the step of ending the recursive process is performed, the pixel density of the ruled line at the estimated ruled line position is projected in the ruled line direction using the high-resolution input image, and the number of ruled lines is determined according to the number of peaks of the pixel density. Ruled line determining step for determining whether or not there is a diagonal line component around the vertices of the area surrounded by the ruled lines extracted from the input image and surrounding the vertices. And determining, program for implementing the steps of determining whether the face vertices or diagonal lines in areas which sandwich the the oblique direction component in the surrounding exists, a ruled line extraction method comprising the information processing apparatus. 9. A ruled line position estimating step of estimating the position of a ruled line from a reduced image of an input image, a step of extracting a region surrounded by four sides by a ruled line extracted from the input image, and the region having a nested structure. If the extracted area is smaller than the size of the character inside or around the area, the step of extracting the area by the recursive processing, and Then, the step of ending the recursive process is performed, the pixel density of the ruled line at the estimated ruled line position is projected in the ruled line direction using the high-resolution input image, and the number of ruled lines is determined according to the number of peaks of the pixel density. Ruled line determining step for determining whether or not there is a diagonal line component around the vertices of the area surrounded by the ruled lines extracted from the input image and surrounding the vertices. Borders extraction method comprising the steps of determining, a step of determining whether the face vertices or diagonal lines in areas which sandwich the the oblique direction component in the surrounding exists, a. 10. A ruled line position estimating means for estimating a position of a ruled line from a reduced image of an input image, a means for extracting a region surrounded by four ruled lines extracted from the input image, and the region having a nested structure. When the extracted area is smaller than the size of the character inside or around the area, the means for extracting the area by the recursive processing In addition, the pixel density of the ruled line at the estimated ruled line position is projected in the ruled line direction by using the means for ending the recursive processing and the high-resolution input image, and the number of ruled lines is determined by the number of peaks of the pixel density. And a means for determining whether or not there is a diagonal component on the opposite vertices of the area surrounded by the ruled lines extracted from the input image or around the vertices. When, A ruled line extraction device comprising: a means for determining whether or not a diagonal line exists in the facing apex or in a region sandwiched by the diagonal direction components that surrounds the apex.