JP4208520B2 - Image processing apparatus, image processing method, program, and storage medium - Google Patents

Description

[0001]
[Technical field to which the invention belongs]
The present invention relates to an image processing apparatus, an image processing method, a program, and a storage medium.
[0002]
[Prior art]
In recent years, computerization of information has progressed, and there is an increasing demand for mutual conversion between paper documents and computerized documents. When a paper document is digitized, it is desirable not only to photoelectrically convert the image into a pixel data by a scanner or the like, but also to divide the area for each description content and convert it into a data that is optimal for the description content.
[0003]
When dividing the area, it recognizes the contents of the description and divides it into areas with different properties such as text area, symbol area, figure area, photo area, table area, etc. Line data and the table frame of the table area are converted into vector data, the photograph area is converted into pixel data, and the table contents are converted into structure data.
[0004]
Needless to say, the logical internal structure of tabular documents is important, but the shape of ruled lines has a great meaning as a visual representation of design effects and data structures. The need to extract faithfully is high.
[0005]
Further, when a form document is electronically created with reference to paper data, if the table information on the paper surface can be faithfully converted into data in advance, the time and labor required for document creation can be greatly reduced.
[0006]
Regarding the area division processing, for example, the processing described in US Pat. No. 5,680,478 has been proposed. A set of black pixel blocks and white pixel blocks in a document image is extracted, and characters, pictures, and the like are extracted from the shape, size, set state, and the like. And characteristic areas such as charts, tables, frames, and lines are extracted.
[0007]
There is also a method for extracting primitive line information from an image using run length or the like without using region division processing.
[0008]
[Problems to be solved by the invention]
However, the area dividing process described in US Pat. No. 5,680,478 recognizes the table area and the logical internal structure, but cannot extract information on the design effect and the visual representation of the data structure, such as the thickness and attribute of the table ruled line. It was.
[0009]
In particular, in order to speed up the processing, the processing target image may be reduced and divided into regions, and when the processing result is returned to the actual size, a quantization error occurs in the coordinate value.
[0010]
Extraction of line information by run length, etc. can only obtain data of a set of lines, not information of logical internal structure, etc., and images that contain information other than tables such as characters and diagrams However, the ruled line portion cannot be extracted correctly, and the extracted information contains many errors.
[0011]
The present invention has been made to solve such a conventional problem, and it is an object of the present invention to correctly extract information on ruled lines of a table from an image in which tables, characters, images, and the like are mixed.
[0012]
[Means for Solving the Problems]
An image processing apparatus according to the present invention includes an area extraction unit that extracts a table area from binary image data, and a boundary between white pixel blocks inside the extracted table area. Left and right edges and top and bottom Find the edge of the direction Left and right edges and top and bottom Partial ruled line information acquisition means for acquiring partial ruled line information based on edge histograms obtained from each edge in the direction, and ruled line information acquisition means for acquiring ruled line information of the table area based on the partial ruled line information . As a result, information regarding the ruled lines of the table can be correctly extracted.
[0013]
The image processing apparatus according to the present invention can further enhance the design expressive power of the table by further including ruled line frame corner processing means for obtaining the curvature radius of the curved line when the ruled line frame corner is a curve.
[0014]
In the image processing apparatus according to the present invention, the ruled line information acquisition unit may determine the position information of the partial ruled line included in the partial ruled line information acquired by the partial ruled line information acquisition unit based on the partial ruled line information of adjacent partial ruled lines. Correction may be performed and the ruled line information of the table area may be acquired based on the corrected partial ruled line information. Thereby, the partial ruled line information of the table can be extracted accurately, and the design expressive power can be further enhanced.
[0015]
In the image processing apparatus according to the present invention, the ruled line information acquisition unit groups thickness information included in the partial ruled line information acquired by the partial ruled line information acquisition unit, determines a representative value of each group, The thickness information included in the ruled line information may be corrected based on the representative value, and the ruled line information of the table area may be acquired based on the corrected partial ruled line information. Thereby, the partial ruled line information of the table can be accurately extracted, and the design expressive power can be further enhanced.
[0016]
In the image processing apparatus according to the present invention, the ruled line information acquisition unit integrates partial ruled lines that are equal in thickness information included in the partial ruled line information and are located on the same straight line, so that the ruled line of the table area is integrated. Information may be acquired. Thereby, the partial ruled line information of the table can be accurately extracted, and the design expressive power can be further enhanced.
In the image processing apparatus according to the present invention, the partial ruled line information acquisition means includes the Left and right edges and top and bottom Normal partial ruled line based on edge histogram obtained from each edge in direction Is there Double ruled line Is there , Partial ruled lines where characters touch Is present, or there is no ruled line, or one side is filled Or at least one of them may be determined, and the determination result may be acquired as the partial ruled line information.
[0017]
In the image processing apparatus according to the present invention, the region extracting unit extracts a black pixel block from the binary image data, the black pixel block is substantially square, and the internal white pixel block is well aligned. The black pixel block may be determined as a surface area. As a result, the table area can be accurately extracted.
[0018]
In the image processing apparatus according to the present invention, the ruled line frame corner processing means determines a vertex when the table is a rectangular region based on white pixels in the table region, and moves from the vertex toward the inside of the table region. Alternatively, a black pixel constituting a ruled line may be searched along a straight line having an inclination of about 45 degrees, and the radius of curvature may be obtained based on the position of the searched black pixel. Thereby, the radius of curvature in the digital image can be easily obtained.
[0019]
An image processing method according to the present invention includes: The area extraction means of the image processing device A region extracting step of extracting a table region from the binary image data; The partial ruled line information acquisition means of the image processing device In the vicinity of the boundary of the white pixel block inside the extracted table area Left and right edges and top and bottom Find the edge of the direction Left and right edges and top and bottom A partial ruled line information acquisition step for acquiring partial ruled line information based on an edge histogram obtained from each edge in the direction; Ruled line information acquisition means of the image processing device A ruled line information acquisition step of acquiring ruled line information of the table area based on the partial ruled line information. As a result, information regarding the ruled lines of the table can be correctly extracted.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
[Image processing device]
Next, an embodiment of an image processing apparatus according to the present invention will be described with reference to the drawings.
[0021]
FIG. 1 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention, and FIG. 2 is a diagram of a configuration example of the image processing apparatus of FIG.
[0022]
In FIG. 1, an image processing apparatus 100 photoelectrically converts an image recorded on a sheet and inputs it as pixel data, an area division processing unit 102 that extracts a table and a frame area from image data, a table area A table ruled line extraction unit 103 that extracts detailed information such as the position and thickness of the ruled line in the table, and an output unit 104 that outputs information on the table ruled line.
[0023]
In FIG. 2, in the device configuration example of the image processing apparatus 100, the photoelectric conversion operation of the input unit is executed by the scanner device 201, and the processing of the area division processing unit 102 and the table ruled line extraction unit 103 is executed by the computer device 204. The user controls the operation of the image processing apparatus 100 using the input device 203 such as a keyboard / mouse. In this embodiment, the scanner device 201 is used as an input unit. However, the input unit 101 may be any unit that inputs image data to the area division processing unit 102. For example, a network is used. The image data may be transmitted from an information processing apparatus connected via the network.
[0024]
The image processing apparatus 100 outputs and stores the detailed information extracted by the table ruled line extraction unit 103 to a memory (not shown) inside the image processing apparatus. Although the output unit 104 corresponds to the internal memory, the output unit 104 may include a display device 204 and a printer device 205, and information on the front ruled line is displayed on a screen or output to paper.
[0025]
[Image processing method]
Next, an embodiment of an image processing method according to the present invention will be described with reference to the drawings.
[0026]
FIG. 3 is a flowchart showing an embodiment of an image processing method executed by the image processing apparatus of FIG. 1, FIG. 4 is a flowchart showing details of the region division processing S302 in the image processing method of FIG. 3, and FIG. 3 is a flowchart showing details of ruled line extraction processing S303 in the image processing method of FIG. 3, FIG. 6 is a diagram showing an example of an image input to the image processing apparatus of FIG. 1, and FIG. FIG. 8 shows an example of an 8-connected black pixel block, FIG. 9 shows an example of a 4-connected white pixel block, and FIG. 10 shows a rectangular black pixel. FIG. 11 is a diagram showing an example of an alignment state of white pixel blocks inside a black pixel block, and FIG. 12 is a diagram showing an example of extraction of a logical internal structure of a table. FIG. 13 is a diagram showing an example of left and right edge peaks for partial ruled lines in the table. 14 is a diagram showing the relationship between the tangent position at an elevation angle of 45 degrees and the radius, FIG. 15 is a diagram showing an example of ruled line correction processing, FIG. 16 is a diagram showing an example of logical positions of ruled lines, and FIG. It is a figure which shows the other example of the area | region classification for every attribute by this area | region division process.
[0027]
The image processing method executed by the image processing apparatus of FIG. 1 includes the following steps.
[0028]
Step S301: A paper document is scanned to obtain an electronic binary image from the image described on the paper. Note that the scanner 201 may scan the paper document with multiple values (gray scale or full color) and execute binarization processing, or the scanner 201 may scan the paper document with multiple values and May be binarized in the computer device 204.
[0029]
Step S302: The area division processing unit 102 executes area division processing. In the area dividing process, for example, an image 17A including characters, figures, and tables in FIG. 17 is converted into character areas C17-1, C17-2, C17-3, C17-4, figure area P17, table area T17, and line area L17. Divide each area. In the present embodiment, the table area T17 among the areas obtained by the area division process will be particularly noted, and the process performed on the table area will be described. The details of the area division processing will be described later with reference to the flowchart of FIG.
[0030]
Step S303: The table area includes an area of white pixels 囲 ま (referred to as “matrix”) surrounded by ruled lines. Here, each ruled line surrounding each matrix (white pixel 隗 area) is referred to as a partial ruled line. In step S303, a partial ruled line is extracted and its position and thickness are specified. When a plurality of table areas are extracted by the area dividing process in step S302, partial ruled lines are extracted and specified for each table area. In the process of the partial ruled line, the partial ruled line in FIG. 13A, the character in contact with the upper and lower ruled lines in (B), the double ruled line in (C), and the partial ruled line in contact with the character in (D) are extracted. The process of partial ruled lines will be described later with reference to the flowchart of FIG.
[0031]
Step S304: When the corner of the ruled line frame surrounding the matrix is curved, the radius of curvature is detected. For example, the radius of curvature r of a corner portion of a ruled line frame as shown in FIG. 14 is calculated by an expression 1401 (see FIG. 14: r = n / (2- (2 ^ (1/2)))). If the radius of curvature r is accurately reproduced in the image output based on the table ruled line information extracted by the table ruled line extraction processing unit, the design effect of the table and the visual expression of the data structure are reproduced, and the persuasive power of the image is increased. . The process for obtaining the curvature radius r will be described later.
[0032]
Step S305: FIG. 15 shows a partial ruled line in contact with a character, which is erroneously recognized as a double ruled line. In order to correct such erroneous recognition, the information on the partial ruled lines is corrected with reference to the positions of the adjacent partial ruled lines. The partial ruled line correction process will be described later.
[0033]
Step S306: As shown in FIG. 16, in the 3 × 3 table, the partial ruled lines are arranged in series in the horizontal and vertical directions. If this serial state is detected, the partial ruled lines can be integrated so that the ruled lines of the entire matrix are represented by straight lines. This can enhance the design expressive power of the table. The partial ruled line integration process will be described later.
[0034]
Step S307: Based on the processing of steps S302 to S306, in addition to the ruled line position, thickness, internal logical structure, and corner radius of curvature, ruled line information including integrated ruled line start point and end point information is output. For example, in the ruled line on the uppermost side of the table of 3 rows and 3 columns of FIG. 16, the information of the start point and the end point is expressed as [0,0]-[3,0] using the logical position assigned to the intersection of the lines. The
[0035]
With the above processing, information regarding the ruled lines of the table can be correctly extracted from an image in which tables, characters, images, and the like are mixed.
[0036]
In FIG. 4, the area dividing process in step S302 is executed by the following steps. For high-resolution images, in order to increase the processing speed, it is common to perform region division processing after reducing the image. After the region division processing is completed, the coordinate value of the output region information is returned to the original coordinate system and restored to the region information at the original size. Here, it is assumed that area information of a table or a frame is used in the processing results, and detailed description of the processing for areas of other attributes such as characters and diagrams is omitted.
[0037]
Step S401: Extract eight connected contour blocks of all black pixels from the image. As shown in FIG. 8, the black pixel 8-connected contour block is a contour of a set of black pixels (hereinafter referred to as “black pixel block”) in contact with at least one of vertical and horizontal directions. Note that a black pixel block is that the contour is a black pixel, and even if the inside other than the contour is all white pixels, it is called a black pixel block.
[0038]
Step S402: The height and width are obtained for all black pixel blocks, and compared with thresholds based on a predetermined maximum character height and maximum character width. A black pixel block whose height and width are smaller than the threshold values is determined as a character element, and a symbol “CHAR” is attached. Among the black pixel blocks of “CHAR”, those close to each other are grouped and given a “TEXT” symbol. Since character processing is not directly related to the present invention, description thereof is omitted. For a black pixel block whose height or width is greater than or equal to the threshold value, the process proceeds to step S403.
[0039]
Step S403: A black pixel block is obtained with an aspect ratio, and a vertically or horizontally long black pixel block exceeding a predetermined value is determined to be a line, and is labeled with “LINE”. For black pixel blocks that are not vertically long or horizontally long, the process proceeds to step S404.
[0040]
Step S404: Paying attention to the outline of the black pixel block, when the outline is a thin diagonal line, the black pixel block is determined to be a diagonal line, and the sign of “LINE” is added as in step S403. If it is determined that the contour is not a thin diagonal line, the process proceeds to step S405.
[0041]
Step S405: It is determined whether the outline shape of the black pixel block is a quadrangle. If it is determined that the pixel is a quadrangle, the process proceeds to step S406. If it is determined that the pixel is not a quadrangle, the black pixel block is determined to be a figure, and a “PICTURE” symbol is assigned. For example, in FIG. 10, the image in the table of (A) is determined to be a square black pixel block, and the bird image of (B) is determined to be a black pixel block other than a square.
[0042]
Step S406: Four connected chunks of white pixels existing inside the black pixel chunk are extracted. As shown in FIG. 9, the four connected region block of white pixels is the outline of a set of white pixels (hereinafter referred to as “white pixel block”) in contact in either the vertical or horizontal direction (excluding the diagonal direction). .
[0043]
Step S407: It is determined whether or not the white pixel block extracted in step S406 is aligned. If the shape of the white pixel block is substantially square and the black pixel block is filled with almost no gap, the alignment of the white pixel block is determined to be good. A black pixel block in which the internal white pixel block is well aligned is determined to be a table, and is labeled “TABLE”. A black pixel block that is not well aligned is determined to be a figure and is given the symbol “PICTURE”. For example, in FIG. 11, in the tables of (a) and (b), the internal white pixel block is determined to be well-aligned, and is labeled “TABLE”. The area including the star shape in FIG. 11C is determined to be misaligned, and is given the symbol “PICTURE”.
[0044]
For example, if an image including a table, characters, and diagrams shown in FIG. 6 is divided into regions by the processes of steps S402 to S407, the data of FIG. “TABLE” is extracted.
[0045]
Step S408: The matrix information is extracted from the internal structure of the table region “TABLE” for the image divided into regions by the above processing. That is, the white pixel block in the table area is virtually arranged on a matrix that can logically represent the table structure. In this matrix, a table area is divided by horizontal and vertical boundaries, and two or more white pixel blocks do not exist in one cell (cell). In the table formed in this way, coordinates (x, y) are given to each cell. x indicates an in-row position, and y indicates an in-column position. The coordinates correspond to the arrangement on the matrix when each white pixel block logically represents the table structure. By grouping adjacent horizontal boundary lines and integrating them into an average position within the group, candidates for horizontal ruled lines in the table are extracted. Similarly, candidates for vertical ruled lines in a table are extracted by grouping adjacent ones of the boundary lines in the vertical direction and integrating them into an average position within the group.
[0046]
In FIG. 12, FIG. 12A shows a black pixel block labeled “TABLE”, and there are eight white pixel blocks inside. As shown in FIG. 12B, there are eight circumscribed square groups of these white pixel blocks. As shown in FIG. 12C, horizontal ruled lines are generated by integrating the horizontal boundary lines of these circumscribed squares. Is done. Similarly, the boundary lines in the vertical direction are integrated, and a cell is determined as shown in FIG. Each cell is given coordinates (x, y). FIG. 12E shows a state where white pixel blocks are applied to a matrix.
[0047]
Step S409: For each white pixel block in the table area, it is checked whether or not there is a black pixel inside. If there is, the black pixel block is added to the processing target.
[0048]
Step S410: If there is an unprocessed black pixel block in the document image data, the process returns to step S402, and when all the black pixel blocks have been processed, the region dividing process is terminated.
[0049]
The partial ruled line extraction process in step S303 will be described with reference to FIG. 5 (note that FIG. 5 shows a case where a vertical ruled line is extracted). Here, attention is paid to a white pixel block (matrix) existing in the table region, and it is assumed that a partial ruled line exists at the boundary of the white pixel 隗, and information on the accurate position and thickness is obtained. The area coordinates of the white pixel block may deviate from the actual ruled line due to characters or noise that come into contact with the ruled line.
[0050]
Note that when the original image is reduced and the region dividing process is performed, the obtained coordinate value is limited to a multiple of 4 or 2, for example, and therefore the following process needs to be performed on the image without the reduction. Through the processing in step S303, accurate partial ruled line information can be extracted.
[0051]
Step S501: The boundary line of the circumscribed rectangular area of the white pixel と し is set as the initial ruled line position candidate, and a search area of a predetermined width × white pixel block height is taken near the boundary line.
Step S502: A histogram of a right edge (a black pixel having a white pixel on the right) and a histogram of a left edge (a black pixel having a white pixel on the left) are acquired within the search area. The histogram of the right edge is obtained by extracting the x coordinate value of a black pixel having a white pixel on the right and counting the number of right edges at each x coordinate value. The histogram of the left edge is obtained by extracting the x coordinate value of a black pixel having a white pixel on the left and counting the number of left edges in each x coordinate value.
[0052]
Step S503: The right edge histogram and the left edge histogram peak are obtained, and the peak number, coordinate value, and peak value (number of times) are obtained. The acquired peak number (0 to k), its coordinate value, and peak value are stored.
[0053]
In order to facilitate maximum value extraction, a filter may be appropriately applied to the histogram. The maximum number of peaks may be limited by a number of 2 or more. In this example, it is assumed that a maximum of two peaks are acquired at each edge in the left and right directions, the number of peaks at the left edge is Lk, and the coordinate values and peak values are (Lx1, Lp1) and (Lx2, Lp2), respectively. The peak number of the right edge is Rk, and the coordinate value and peak value are expressed as (Rx1, Rp1) and (Rx2, Rp2). Note that Lp1> Lp2 and Rp1> Rp2.
[0054]
Further, a value forming a predetermined ratio with respect to the height of the search area is set as a threshold T, and the numbers of peaks when ignoring peaks below the threshold T are set as Ln and Rn.
[0055]
In FIG. 13, the state of the ruled line in the search area is classified as (A), (B), (C), (D) by the combination of Ln and Rn. (A) is Ln = Rn = 1 and is a partial ruled line. (B) is Ln = Rn = 0, and there is no partial ruled line. (C) is Ln = Rn = 2 and is a double ruled line. (D) is Ln = 0 or Rn = 0, and is a partial ruled line in contact with characters. A broken line 1301 in FIG. 13 indicates a search area.
[0056]
Step S504: It is determined whether Ln = Rn = 1. If Ln = Rn = 1, the process proceeds to step S505. Otherwise, the process proceeds to step S508.
[0057]
Step S505: It is determined whether Lx1 <= Rx1, and if Lx1 <= Rx1, the process proceeds to step S506.
[0058]
Step S506: It is determined that a ruled line having a thickness (Rx1-Lx1 + 1) exists at the position of (Lx1 + Rx1) / 2. Otherwise, the process proceeds to step S507.
[0059]
Step S507: It is determined that there is no ruled line.
[0060]
Step S508: It is determined whether Ln = Rn = 0. If Ln = Rn = 0, the process proceeds to step S509. Otherwise, the process proceeds to step S509.
[0061]
Step S509: It is determined whether or not Ln = Rn = 2. If Ln = Rn = 2, the process proceeds to step S510, and otherwise, the process proceeds to step S512.
[0062]
Step S510: Lx1 <= Rx1 <Lx2 <= Rx2, Lx1 <= Rx2 <Lx2 <Rx1, Lx2 <= Rx1 <Lx1 <= Rx2, or Lx2 <= Rx2 <Lx1 <= Rx1 is satisfied Judge. If it is any of these, the process proceeds to step S511, and otherwise, the process proceeds to step S516.
[0063]
Step S511: It is determined that a double ruled line having a thickness {max (Rx1, Rx2) -min (Lx1, Lx2) +1} exists at a position of {min (Lx1, Lx2) + max (Rx1, Rx2)} / 2. .
[0064]
Step S512: It is determined whether Ln = 0 or Rn = 0. If Ln = 0 or Rn = 0, the process proceeds to step S513. Otherwise, the process proceeds to step S516.
[0065]
Step S513: If Ln = 0, it is checked whether the left edge histogram has a peak below the threshold T, and if Rn = 0, it is checked whether the right edge histogram has a peak below the threshold T. If there is a peak, the process proceeds to step S514, and otherwise, the process proceeds to step S515.
[0066]
Step S514: In the search area, it is determined that a ruled line is present although a character or the like is in contact with one side and no clear edge appears. If there is no peak, the process proceeds to step S515.
[0067]
Step S515: It is determined that one side is a painted area.
[0068]
Step S516: If none of Steps S504, S508, S509, and S512 applies, that is, if Ln = 2 and Rn = 1, or Ln = 1 and Rn = 2, Step S516 is executed, and if Lx1 <= Rx1, Step S516 is executed. The process proceeds to S517. Otherwise, the process proceeds to step S518.
[0069]
Step S517: It is determined that a ruled line having a thickness (Rx1-Lx1 + 1) exists at the position of (Lx1 + Rx1) / 2. Otherwise, the process proceeds to step S518.
[0070]
Step S518: When Ln = 2 and Lx2 <= Rx1, the process proceeds to step S519. Otherwise, the process proceeds to step S520.
[0071]
Step S519: It is determined that a ruled line having a thickness (Rx1-Lx2 + 1) exists at the position of (Lx2 + Rx1) / 2.
[0072]
Step S520: When Rn = 2 and Lx1 <= Rx2, the process proceeds to step S521. Otherwise, the process proceeds to step S522.
[0073]
Step S521: It is determined that a ruled line having a thickness (Rx2-Lx1 + 1) exists at the position of (Lx1 + Rx2) / 2.
[0074]
Step S522: It is determined that there is no ruled line in the search area.
[0075]
The processes in steps S501 to S522 are applied to the left and right edges of all white pixel blocks. However, processing should not be repeated for the same ruled line. As a result, a vertical ruled line is extracted.
[0076]
Further, the same processing is executed for the vertical edge of the white pixel block to extract a horizontal ruled line. The processing contents are exactly the same except that the direction is 90 degrees different from the left-right direction edge, and the description is omitted.
[0077]
As shown in FIG. 14, when calculating the radius of curvature of the corner of the ruled line frame, a straight line L that touches the curve at the apex P of the corner and forms an angle of 45 degrees with respect to the horizontal ruled line is drawn. When the intersection of the straight line L and the horizontal ruled line is Q, there is an expression 1401 (see FIG. 14: r = n / (2- (2 ^ (1 / 2)))) is established. Therefore, the radius of curvature r can be obtained by detecting the distance n.
[0078]
Such geometric processing may not be easy on a real image. For example, the following processing is adopted.
[0079]
First, in the corner of interest (here, the case where the lower left corner of the table frame draws an arc will be described as an example), in order to simplify the explanation, the origin of the horizontal coordinate axis (x-axis in FIG. 14) is set. A coordinate system is defined that corresponds to the left end of the white pixel block and the origin of the horizontal coordinate axis (y-axis in FIG. 14) corresponds to the bottom end of the white pixel block (not limited to this coordinate system, but arbitrary coordinates) System can be processed). In this coordinate system, the pixels on the straight line M in FIG. 14 are searched from the origin P toward the center of the arc. When the height of the white pixel block is h, the coordinates (i, j) of the pixel to be searched are searched in the order of i = j = 0, 1, 2, 3,. The coordinates of the black pixel are regarded as the contact S where the straight line L and the arc of the ruled line contact. Assuming that the coordinates of the contact S at this time are (is, is), the distance n between PQs is obtained by n = 2 * is.
[0080]
Even when a black pixel is found, it is checked whether the black pixel is an isolated point, and if it is determined to be an isolated point, it is processed as noise, and further searching is continued to be robust against noise. Become.
[0081]
In FIG. 15, when the ruled line of interest is in contact with the character “outside”, the processing in step S303 is (Lx1, Lp1), (Lx2, Lp2) as the peak in the left direction edge, and ( Rx1, Rp1) and (Rx2, Rp2) are extracted. Here, it is erroneously recognized that there is a double ruled line between Lx1 and Rx2. However, in step S305, a pair of peaks that most closely matches the left and right ends of the next lower ruled line is searched again to obtain Lx2 and Rx2. As a result, the position and thickness information of the upper partial ruled line is corrected to be accurate.
[0082]
In FIG. 16, a 3 × 3 table with a high design effect is generated by integrating the partial ruled lines in series. When integrating partial ruled lines, ruled line information is organized by integrating partial ruled lines that are on the same line (in series) and have the same thickness.
[0083]
The position of the partial ruled line obtained by the processing up to step S305 includes an error due to noise, attached pixels, a quantization error caused by reading resolution, or the like. Therefore, the thickness of the partial ruled lines is averaged. In the averaging, the distribution of thicknesses of all partial ruled lines is obtained, and the groupable thicknesses are grouped. A representative value of the thickness is set from the average value of the thickness of each group, and the thickness of the partial ruled lines belonging to each group is unified to the representative value. Double ruled lines may be grouped separately from other partial ruled lines.
[0084]
Thereafter, the partial ruled lines having the same uniform thickness and adjacent on the same line are integrated into one ruled line. Similarly, the position of the ruled line is also adjusted to be an average position.
[0085]
The ruled line information output by the above processing can be used as basic information to be displayed on an image or output again on paper. In addition, the physical information of the ruled lines can be obtained with respect to noise and image state fluctuations in terms of position and thickness, etc., and is compatible with logical table structure layout, so it is compatible with various applications. High information.
[0086]
[program]
The control program in the image processing apparatus of FIGS. 1 and 2 includes a program for executing the embodiment of the image processing method. This program includes a routine for executing the processes of steps S301 to S307, steps S401 to S410, and steps S501 to S522.
[0087]
[Storage medium]
The image processing method according to the present invention can also be implemented by an apparatus other than the image processing apparatuses of FIGS. 1 and 2, for example, a general-purpose computer. In this case, a control program provided on a storage medium or the like is stored in an external storage of the general-purpose computer. It may be set in the apparatus and executed by a CPU or the like according to an instruction from an operator or the like.
[0088]
【The invention's effect】
According to the present invention, it is possible to correctly extract information related to the ruled line of the table.
Further, the ruled line information can be correctly extracted, so that the table can be accurately reproduced.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention.
FIG. 2 is a diagram illustrating an example of a device configuration of the image processing device in FIG. 1;
FIG. 3 is a flowchart showing an embodiment of an image processing method executed by the image processing apparatus of FIG. 1;
4 is a flowchart showing region division processing in the image processing method of FIG. 3;
5 is a flowchart showing ruled line extraction processing in the image processing method of FIG. 3;
6 is a diagram illustrating an example of an image input to the image processing apparatus in FIG. 1. FIG.
7 is a diagram illustrating an example of region classification for each attribute by the region division processing of FIG. 4; FIG.
FIG. 8 is a diagram illustrating an example of eight connected black pixel blocks;
FIG. 9 is a diagram illustrating an example of a 4-connected white pixel block.
FIG. 10 is a diagram illustrating an example of a rectangular black pixel block and other black pixel blocks.
FIG. 11 is a diagram illustrating an example of an alignment state of white pixel blocks inside a black pixel block.
FIG. 12 is a diagram illustrating an example of extracting a logical internal structure of a table.
FIG. 13 is a diagram showing an example of left and right edge peaks for partial ruled lines in a table.
FIG. 14 is a diagram showing a relationship between a tangent position at an elevation angle of 45 degrees and a radius.
FIG. 15 is a diagram illustrating an example of ruled line correction processing;
FIG. 16 is a diagram illustrating an example of logical positions of ruled lines.
FIG. 17 is a diagram illustrating another example of area classification for each attribute by the area division processing in FIG. 4;
[Explanation of symbols]
100 Image processing apparatus
101 Input section
102 Region division unit
103 Table ruled line extraction unit
104 Output section

Claims (12)

Area extraction means for extracting a table area from binary image data;
Obtains the vertical edges and horizontal direction of the edge near the boundary of the extracted table region inside the white pixel block, based on the edge histogram calculated from each edge of the horizontal direction of the edge in the vertical direction, part ruled line information Partial ruled line information acquisition means for acquiring
Ruled line information acquisition means for acquiring ruled line information of the table area based on the partial ruled line information;
An image processing apparatus comprising:   2. The image processing apparatus according to claim 1, further comprising ruled line frame corner processing means for obtaining a radius of curvature of the curved line when the ruled line frame corner is a curve.   The ruled line information acquisition unit corrects the position information of the partial ruled line included in the partial ruled line information acquired by the partial ruled line information acquisition unit based on the partial ruled line information of the adjacent partial ruled line, and the corrected partial ruled line The image processing apparatus according to claim 1, wherein ruled line information of the table area is acquired based on the information.   The ruled line information acquisition unit groups thickness information included in the partial ruled line information acquired by the partial ruled line information acquisition unit, determines a representative value of each group, and determines the thickness information included in the partial ruled line information. The image processing apparatus according to claim 1, wherein the image processing apparatus corrects based on the representative value, and acquires ruled line information of the table area based on the corrected partial ruled line information.   The ruled line information acquisition means acquires the ruled line information of the table region by integrating partial ruled lines that are equal in thickness information included in the partial ruled line information and are present on the same straight line. The image processing apparatus according to claim 1. The partial ruled line information acquisition unit is configured to determine whether a normal partial ruled line exists, a double ruled line exists, or a character contacts based on edge histograms obtained from the left and right edges and the vertical edges. 6. The method according to claim 1, wherein at least one of a ruled line is present, a ruled line is not present, and one side is a filled area is determined, and the determination result is acquired as the partial ruled line information. An image processing apparatus according to any one of the above.   The region extracting means extracts a black pixel block from the binary image data, and when the black pixel block is substantially square and the internal white pixel block is well aligned, The image processing apparatus according to claim 1, wherein the image processing apparatus is determined as a table area.   The ruled line frame corner processing means determines a vertex when the table is a rectangular region based on the white pixel wrinkles inside the table region, and makes a straight line with an inclination of approximately 45 degrees from the vertex toward the inside of the table region. The image processing apparatus according to claim 2, wherein a black pixel that forms a ruled line is searched for, and the curvature radius is obtained based on a position of the searched black pixel. The left and right edge is a right edge indicating a black pixel having a white pixel on the right and a right edge indicating a black pixel having a white pixel on the left,
The image processing according to any one of claims 1 to 8, wherein the partial ruled line information acquisition unit acquires partial ruled line information of a vertical ruled line based on an edge histogram obtained from the left and right edges. apparatus. A region extracting step in which a region extracting unit of the image processing device extracts a table region from the binary image data;
Part ruled line information acquisition unit of the image processing apparatus, the vertical edges and horizontal direction of the edge near the boundary of the extracted table region inside the white pixel blocks determined, with the horizontal direction of the edge from the vertical edge, respectively Based on the obtained edge histogram, a partial ruled line information acquisition step for acquiring partial ruled line information;
Ruled line information acquisition means for acquiring ruled line information of the table area based on the partial ruled line information , the ruled line information acquiring unit of the image processing apparatus ;
An image processing method comprising: A region extracting step of extracting a table region from the binary image data;
Obtains the vertical edges and horizontal direction of the edge near the boundary of the extracted table region inside the white pixel block, based on the edge histogram calculated from each edge of the horizontal direction of the edge in the vertical direction, part ruled line information Partial ruled line information acquisition step for acquiring,
Ruled line information acquisition step for acquiring ruled line information of the table area based on the partial ruled line information;
For causing a computer to execute the computer-executable program. A computer-readable storage medium in which the program according to claim 11 is stored.

Images