JP3380136B2 - Apparatus and method for identifying format of table image - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition device for converting a document or drawing into image data by an input device such as a scanner and recognizing the image data, and a format identification device for identifying a ruled line structure in an image. And its method.

[0002]

2. Description of the Related Art Currently, a large amount of tabular documents, such as slips, whose format is structured by ruled lines, are in wide circulation, and there is a great demand for automatically identifying the format of tabular documents. In the automatic format identification processing, the feature quantity of the format of the tabular document is registered in the dictionary as a model in advance. Then, when an unknown tabular document is input, the feature amount is calculated from the image and matched with the feature amount of the model of the dictionary to identify the most similar model.

In the conventional model matching method, after normalizing the entire table, the position and length of each ruled line or the position of the center point of each rectangular cell surrounded by the ruled lines on the four sides is determined between the input document and the model. Then, it was judged whether or not they overlap each other, and the similarity was calculated based on the result. Here, the normalization of the table means an adjustment process such as converting the entire image so that the aspect ratio of the table becomes 1: 1.

However, the success or failure of the model matching largely depends on the success or failure of the normalization process, and thus this method cannot be said to be robust against the blur or noise of the image of the input document.

Therefore, the present applicant has developed a technique for adaptively changing the reference position and length when comparing the position and length of ruled lines between an input document and a model. This method is based on the prior application “Management Information Extraction Device and Method for Extracting Management Information from Image” (Japanese Patent Application No. 08-351).
346), it is described as a form identification process using a matching table.

[0006]

However, in the developed model matching method, in order to compare the relative relationship between the ruled lines in the input document with the relative relationship between the ruled lines in the model, a certain processing time is required. Requires.
Since this processing time is linearly proportional to the number of models to be compared, there arises a problem that the processing time increases as the number of models increases. Therefore, it is desirable to reduce the number of models to be applied without applying this method to all the given models.

An object of the present invention is to provide a format identification apparatus and method for reducing the number of models to which detailed model matching is applied in the format identification of a tabular document and accelerating the processing.

[0008]

FIG. 1 is a block diagram of a format identification device according to the first and second principles of the present invention.

The format identification device according to the first principle of the present invention comprises a storage means 1, a comparison means 2 and a classification means 3, searches for formats of a plurality of tables, and formats corresponding to given table images. Ask for.

The storage means 1 stores information regarding the characteristics of the formats of the plurality of tables. The comparing means 2 compares one or more features extracted from the input image with one or more features included in one of the plurality of formats,
Find combinations of features that may correspond. The classifying unit 3 obtains the number of the features corresponding to the combination among the features of at least one of the plurality of formats and the input image, and narrows down the search target from the plurality of formats based on the number.

What are the characteristics of the input image and format?
It represents arbitrary information that describes the input image and the table structure of the format, and indicates, for example, ruled lines, intersections of the ruled lines, or rectangular cells surrounded by the ruled lines.

The comparing means 2 compares the feature quantities such as the size and position of the feature between the input image and the format, and determines that the features having similar feature quantities can correspond to each other. When ruled lines are used as a feature, for example, a combination of ruled lines having similar sizes and positions is obtained as a compatible combination.

The classifying means 3 has at least one of the characteristics of the input image and the characteristics of the format in the storage means 1,
The number of objects determined to be compatible with any of the characteristics of the comparison partner is counted. Then, for example, a format whose number is a certain ratio or more with respect to the total number of features is extracted as a search target.

The format identification device performs more detailed model matching only on the format group thus extracted to specify the format corresponding to the input image. The process of counting the number of features determined to be compatible does not take much time, so the number of models subject to detailed model matching is efficiently reduced,
The processing speed is increased.

Next, the format discriminating apparatus according to the second principle of the present invention comprises the comparing means 2 and the calculating means 4, and in order to match the two table images, the format extracted from the two table images. Compare the features of.

The comparing means 2 compares one or more features included in one of the two table images with one or more features included in the other, and combines features that may be associated with each other. Ask for. The calculating means 4 obtains the number of the features corresponding to the combination among at least one of the features of the two table images, and calculates the degree of similarity between the two table images based on the number.

The comparing means 2 compares the features such as ruled lines extracted from the two given table images, determines the possibility of correspondence, and obtains a combination of features that can be dealt with. The calculating means 4 counts the number of those determined to be able to correspond to any of the characteristics of the comparison partner in at least one of the front images. Then, for example, the ratio of the number to the total number of features is calculated, and the result is used as the degree of similarity between the two table images.

The format identification device considers that the formats of these front images match each other, for example, when the obtained similarity is a certain value or more. Since the process of counting the number of features determined to be compatible does not take much time, the degree of similarity can be obtained relatively easily, and the matching process can be performed efficiently.

For example, the storage means 1 of FIG. 1 corresponds to a memory 12 or an external storage device 15 of FIG. 2 described later, and the comparison means 2, the classification means 3 and the calculation means 4 are provided with a CPU (central processing unit) 11 And the memory 12.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. Even in the case of the model matching process of the prior application described above, the models in the dictionary are roughly classified in advance based on the number of intersections of the horizontal ruled lines and the vertical ruled lines, and the model candidates to which the detailed identification process should be applied to some extent. Limited by percentage.

In the present embodiment, a new large classification process is inserted in the model matching described as a kind of the detailed identification process in the prior application to further reduce the number of models. Therefore, similar to the detailed identification process of the prior application, first, by using the relative size and position of the ruled line with respect to the entire table as the feature amount, the possibility of correspondence between the ruled line included in the image of the input document and the model ruled line is examined, and correspondence is performed. Find possible combinations of ruled lines.

Here, by expanding the condition of the possibility of correspondence, a plurality of ruled lines can correspond to one ruled line. Therefore, even if the extraction of the feature amount for the entire table fails, the correct correspondence is not overlooked if the failure is within a certain range.

Next, with respect to each ruled line of the input image and the model, the number of combinations that can be handled is generated, and the ratio of the obtained number to the total number of ruled lines is calculated. Then, a model having a ratio equal to or higher than a certain threshold is recognized as a large classification candidate. By appropriately setting this threshold value, it is possible to narrow down the large classification candidates.

The format identification device of this embodiment is realized by an information processing device (computer) as shown in FIG. 2, for example. The information processing apparatus shown in FIG.
Memory 12, input device 13, output device 14, external storage device 15, medium drive device 16, network connection device 1
7 and a photoelectric conversion device 18, which are connected to each other by a bus 19.

The CPU 11 executes the program while utilizing the memory 12 to perform the format identification processing.
As the memory 12, for example, a ROM (read only memor)
y), RAM (random access memory), etc. are used.

The input device 13 corresponds to, for example, a keyboard, a pointing device, etc., and is used for inputting requests and instructions from the user. The output device 14 corresponds to a display device (printer), a printer, etc., and is used for inquiring of the user and outputting processing results.

The external storage device 15 is, for example, a magnetic disk device, an optical disk device, a magneto-optical disk device, etc., and can store programs and data. It is also used as a database to store images and model dictionaries.

The medium driving device 16 drives the portable storage medium 20 and accesses the stored contents. Portable storage medium 2
0 means memory card, floppy disk, CD
Any computer-readable storage medium such as a ROM (compact disk read only memory), an optical disk, a magneto-optical disk can be used. This portable storage medium 20
In addition to data, the program stores a program for performing the above-described format identification processing.

The network connection device 17 is a LAN (lo
It is connected to any communication network such as a cal area network) and performs data conversion accompanying communication. The format identification device can receive necessary data and programs from an external database or the like via the network connection device 17. The photoelectric conversion device 18 is, for example, an image scanner, and inputs an image such as a document or drawing to be processed.

FIG. 3 is a flowchart of the format identification processing of this embodiment. When the processing is started, the format identification device first inputs an image (step S
1) Extract ruled lines from the input image (step S
2).

A rectangle surrounding the ruled line (ruled line rectangle)
Regarding the coordinates of the upper left apex, the horizontal ruled lines are rearranged in the ascending order of the y coordinate, and the vertical ruled lines are arranged in the ascending order of the x coordinate (step S3). Here, the x-axis of the coordinates is in the horizontal direction and the y-axis is in the vertical direction. If the y-coordinates of the horizontal ruled lines are the same, sort them in ascending order of their x-coordinates,
If the coordinates are the same, the y-coordinates are sorted in ascending order.

Next, general information is extracted for each vertical and horizontal ruled line (step S4). The outline information is a relative value of the length and position of the ruled line with respect to the entire table, and is represented by a set of three integers. Further, considering all combinations of two ruled lines in each of the vertical and horizontal directions, detailed information regarding each combination is extracted (step S5). The detailed information represents the relative relationship between the length and the position between the two ruled lines.

The outline information and the detailed information of the front image of the model to be collated with the input image are extracted in advance and stored in the dictionary 2.
It is stored in 1. Therefore, next, the outline information and the detailed information of the input image are collated with the outline information and the detailed information of the model to perform model matching (step S).
6). Here, the outline information is used for large classification model matching, and the detailed information is used for detailed classification model matching. Then, the optimum model is output as the identification result (step S7), and the process ends.

Next, the processes of steps S4, S5 and S6 will be described in detail with reference to FIGS. Hereinafter, the outline information will be referred to as global information.

In step S4, the reference width W, the reference height H, the reference x coordinate x0, and the reference y coordinate y0 of the table are obtained as preprocessing for obtaining the global information. First, the maximum lengths of horizontal ruled lines are calculated, and the first and last horizontal ruled lines having a length greater than a certain threshold (for example, 0.8) relative to that length. , And use them as the outer standard horizontal ruled lines.

Regarding the vertical ruled lines, the maximum length is obtained, and two outer reference vertical ruled lines are obtained in the same manner as in the case of the horizontal ruled lines. Then, consider the circumscribed rectangle of the obtained four outline reference ruled lines, its width is the reference width W, its height is the reference height H, and its upper left apex is the reference point, and its coordinates are the reference coordinates (x0, y0). And

For example, in the tabular document as shown in FIG. 4, the horizontal ruled lines 31 and 32 are extracted as the outer reference horizontal ruled lines, and the vertical ruled lines 33 and 34 are extracted as the outer reference vertical ruled lines. The width of the circumscribed rectangle is the reference width W, and its height is the reference height H. Further, the coordinates of the upper left apex 35 of the circumscribed rectangle are the reference coordinates (x0, y0).

Here, by selecting the outline reference ruled line from ruled lines having a predetermined length or more calculated from the maximum length, for example,
Short ruled lines such as the horizontal ruled lines 36 and 37 are excluded from the candidates for the outer reference ruled lines. Instead of using the upper left vertex of the circumscribed rectangle of the four outline reference ruled lines as the reference point, the lower left vertex, the upper right vertex,
An arbitrary point on the periphery of the circumscribed rectangle, such as the lower right apex, can be used as the reference point. Even in that case, the following processing is basically the same.

Next, based on the size of the obtained table and the reference coordinates, from the information of the length and center position of each ruled line rectangle, 3
Two feature quantities (global information) length1, twist,
Request the position. In the case of a horizontal ruled line, as shown in FIG. 5, the length l1 of the ruled line rectangle 41 and the center coordinates (x1, y
From 1), these feature quantities are calculated by the following equation. length1 = integer part of [(l1 / W) × 100] twist = [((x1-x0) / W) × 100] integer part (1) position = [((y1-y0) / H) × 100] In the case of vertical ruled lines, as shown in FIG.
From the length l1 and the center coordinates (x1, y1), these feature quantities are calculated by the following equation. length1 = integer part of [(l1 / H) × 100] twist = [((y1-y0) / H) × 100] integer part (2) position = [((x1-x0) / W) × 100] In the obtained integer, the length1 represents the relative ratio of the length of the ruled line to the size of the table, and the twist and the position represent the relative position of the ruled line to the reference point of the table.

Next, in step S5, detailed information indicating the relative relationship between the two ruled lines is obtained. here,
When the length of one ruled line rectangle is 1, the length length2 of the other ruled line rectangle, the length difference in the x direction between the centers of the respective ruled line rectangles, and the difference in the y direction between the centers Three values of the length height will be obtained as detailed information.

First, all combinations of two horizontal ruled lines are extracted. Then, in each combination, as shown in FIG. 7, one ruled line rectangle (the one having the earlier sort order) 43
Is l1, the center coordinate is (x1, y1), and the length of the other ruled line rectangle (the one with the slower sorting order) 44 is l
2 and its center coordinates are (x2, y2). At this time,
With the center of the ruled line rectangle 43 as a reference, a shift dw in the x direction and a shift in the y direction between the centers of these ruled line rectangles are defined by the following equation. dw = x2-x1 dh = y2-y1 (3) According to this definition, the center of the ruled line rectangle 44 is the ruled line rectangle 43.
If it is to the right of the center of, the dw becomes positive and the ruled line rectangle 44
If the center of is to the left of the center of the ruled line rectangle 43, dw is negative. Similarly, if the center of the ruled line rectangle 44 is below the center of the ruled line rectangle 43, dh is positive, and if the center of the ruled line rectangle 44 is above the center of the ruled line rectangle 43, dh is negative.

Next, the above-mentioned three feature quantities length
2, differ and height are calculated by the following equation. length2 = 12 / l1 differ = dw / l1 (4) height = dh / l1 Similarly, for vertical ruled lines, all combinations of two ruled lines are extracted. In each combination, as shown in FIG. 8, the length of the ruled line rectangle 45 having the earlier sort order is l1, its center coordinates are (x1, y1), and the length of the ruled line rectangle 46 having the slower sort order is Is 12 and its center coordinates are (x2, y2). Then, dw and dh are obtained by the equation (3), and the detailed information length is obtained by the following equation.
Calculate h2, differ, and height. length2 = 12 / l1 differ = dh / l1 (5) weight = dw / l1 In the formula (5), the definitions of differ and height are opposite to those of the formula (4).

Next, in step S6, the horizontal and vertical ruled lines are divided into two, and the global information and detailed information of the input image are compared with the global information and detailed information of each model to determine the degree of format similarity. To calculate. The large classification processing of the present invention is included in the model matching in step S6, and a flowchart showing this in more detail is shown in FIG. The model matching will be described below with reference to the flowchart of FIG.

The format identification device first determines the number of large classification candidates (step S11). When the number of models in the dictionary 21 is n, the number of major classification candidates cand is
It is determined by the following formula. cand = max {n / 10, inf} (6) Here, inf represents the minimum number of candidates given in advance by the user, and for example, inf = 50.
The right side of the equation (6) represents the maximum value of n / 10 and inf.

Next, the large classification model matching is performed to compare the global information of the input image with the global information of each model (step S12). This large classification model matching is performed twice for each model, that is, horizontal ruled lines and vertical ruled lines. In the large-class model matching, first, a p × m table as shown in FIG. 10 is created with the number of horizontal ruled lines of the input image of the unknown document being p and the number of horizontal ruled lines of the model being m.

Here, p = 12 and m = 15, and both the row number and the column number of the table start from 0.
The element (item) at the i-th row and the j-th column in this table is data representing the correspondence between the i-th ruled line of the input image and the j-th ruled line of the model. Hereinafter, such a table will be referred to as a matching table.

Next, based on the global information, the i of the input image is
The th horizontal ruled line IP (i) is the jth horizontal ruled line M of the model.
Whether or not it corresponds to O (j) is determined, and if there is a possibility of correspondence, a node is arranged in the element at the i-th row and the j-th column of the matching table. As a result, the horizontal ruled line IP (i)
And the horizontal ruled line MO (j) are described on the matching table as ruled line correspondence candidates. The condition of the possibility of correspondence at this time is set to be sufficiently loose to allow a plurality of ruled lines to correspond to one ruled line in an overlapping manner.

Here, the global information of the ruled line IP (i) (l
(length1, twist, position) are respectively set to (ipl, ipt, ipp), and the ruled line MO is set.
The global information of (j) is (mol, mot, mo)
In p), when the difference between the respective values is smaller than a certain value, it is considered that the ruled line IP (i) and the ruled line MO (j) may correspond to each other.

A concrete condition representing the possibility is given by the following equation with the parameters α, β and γ depending on the number of horizontal ruled lines and vertical ruled lines in the table as threshold values. | Ipl-mol | <α | ipt-mot | <β (7) | ipp-mop | <γ The parameters α, β, and γ that depend on the number of these ruled lines are positive integers, and the smaller the number of ruled lines, the larger. It takes a value, and conversely, the larger the number of ruled lines, the smaller the value. At this time, the condition of Expression (7) has the effect of expanding the matching search range if the density of the ruled lines in the table is sparse, and narrowing the search range if the density is high. A vertical ruled line matching table is also created in the same manner.

Next, in the matching table in which the nodes are arranged, the projection of the nodes is taken in the horizontal direction and the vertical direction, and the global similarity is calculated. Here, regarding the horizontal ruled lines of the input image and the model, the number of ruled lines corresponding to the row and the column in which the node exists is determined, and these numbers are set as iphit and mohit, respectively. Then, the ratio of each number to the total number of horizontal ruled lines is calculated, and the sum thereof is taken as the global similarity of the horizontal ruled lines, as shown in the following expression. Global similarity of horizontal ruled line = iphit / p + mohit / m (8) For example, in the matching table of FIG. 10, a row and a column in which one or more nodes represented by the symbol ● exist are indicated by the symbol ★. There is. By counting the numbers of these symbols, it can be seen that ipit = 8 and mohit = 10. Therefore, the global similarity of equation (8) is 8
/12+10/15=1.33. The global similarity of the vertical ruled lines is calculated in the same manner, and the global similarity of one model is given by the sum of the global similarity of the horizontal ruled lines and the global similarity of the vertical ruled lines.

Next, a model whose global similarity is equal to or higher than a certain threshold value or a model whose global similarity is up to the top cand is set as a large classification candidate (step S13). Alternatively, among the models whose global similarity is equal to or higher than the threshold value, the top c
The ones up to the and position may be the major classification candidates.

Next, the detailed classification model matching is performed by applying the processing described in the prior application to the model of the large classification candidate, and the detailed similarity is calculated (step S14). Then, the format of the table corresponding to the input image is determined based on the obtained detailed similarity (step S15), and the process ends.

In this embodiment, by appropriately adjusting the threshold value and the value of cand used in step S13, the number of large classification candidates can be effectively reduced, and the detailed classification model matching can be performed at high speed. Can be converted.

Further, if the threshold values of the equation (7) are set to be large in step S12, the condition of the possibility of correspondence between the ruled lines becomes loose. Therefore, even if there is a portion where ruled line extraction does not work well due to the effect of blurring or noise, a model similar to the input document can be included in the major classification candidates without exception. Therefore, the target of the detailed classification model matching can be limited to the large classification candidate group including the model having the same ruled line structure as the input document among all the models in the dictionary 21.

Next, the processing of the large classification model matching in step S12 of FIG. 9 will be described in detail with reference to FIGS. 11, 12, 13, 1
4 is a flowchart showing an example of the large classification model matching performed for one matching table.

Of these, FIGS. 11 and 12 show the process of setting the window 51 as shown in FIG. 15 in the matching table. This window 51 represents a region in which a node may be arranged in the j-th column of the matching table. The line number at the top end is described as st [j] and the line number at the bottom end is It is described as ed [j]. 13 and 14 show a process of counting the number of arranged nodes.

When the process is started, the format identification device firstly controls the control variable j representing the row of the matching table.
Is set to 0 (step S21), and the j-th ruled line of the model is set to mo (step S22). Next, global information po
A variable k representing the number of ruled lines of an unknown document whose position is similar to the j-th ruled line of the model is set to 0 (step S23).

Next, the control variable i representing the row of the matching table is set to 0 (step S24), and i of the input document is set.
The th ruled line is set to ip (step S25). And
Absolute value of difference between global information ipp and mop | ipp-mop
| Is compared with the threshold value γ (step S26).

If | ipp-mop | is smaller than γ,
Next, the window 51 is set by performing the processing of FIG. Here, first, k is compared with 0 (step S3
5) If k is 0, st [j] = i is set and 1 is added to k (step S37). If k is not 0,
Since st [j] has already been set, the process of step S37 is performed as it is.

Next, the number of ruled lines of the unknown document is set to ipsuu, and i is compared with ipsuu-1 (step S3).
8). This ipsu corresponds to the number of rows in the matching table, and ipsu-1 represents the number of the last row in the matching table.

If i reaches ipsu-1, then ed
Setting [j] = i (step S39), 1 is added to j (FIG. 11, step S33), and j is the number m of ruled lines in the model.
It is compared with osuu (step S34). Then, if j is smaller than mosuu, the processing from step S22 is repeated.

In step S38 of FIG. 12, i is i
If it is smaller than psu-1, add 1 to i (see FIG.
1, step S28), i is compared with ipsu (step S29). If i is smaller than ipsuu, the processing from step S25 is repeated until i is ips
When uu is reached, k is again compared with 0 (step S3).
0).

If k remains 0, the global information posi
It means that there is no ruled line of the unknown document whose tion is similar to the j-th ruled line of the model. Then st
[J] = ed [j] = − 1 (step S31),
The processing after step S33 is performed. st [j], ed
When [j] is -1, the window 51 is not set.

In step S26, | ipp
If −mop | is γ or more, then k is compared with 0 (step S27). And if k is 0, then s
Since t [j] is not set, the processing from step S28 is performed. If k is greater than 0, already st
Since [j] is set in any row, next, e
With d [j] = i−1, the processing from step S33 is performed.

Further, in step S30, k is 0.
If it is larger, st [j] and ed [j] have already been set, and therefore the processing from step S33 is performed. If j reaches mosuu in step S34, it means that the possible windows 51 have been set for all the columns of the matching table. Therefore, node placement processing is performed next.

Here, first, a variable ipit representing the number of rows in which the nodes are arranged and a variable mohit representing the number of columns in which the nodes are arranged are set to 0 (FIG. 13, step S4).
0), ipline [i] representing the number of nodes in the i-th row
And 0 for moline [j] representing the number of nodes in the jth column
(Step S41). However, i = 0,
1 ,. ． ． , Ipsu-1, j = 0, 1 ,. ． ． , M
osuu-1.

Next, the control variable j representing the column of the matching table is set to 0 (step S42), and the j-th ruled line of the model is set to mo (step S43). Further, the control variable k representing the row of the matching table is set to st [j] (step S44), and its value is checked (step S4).
5).

If k is 0 or more, since the window 51 is set in the jth column, the kth ruled line of the input document is set to ip (step S46), and the global information ipl is set.
Absolute value | ipl-mol |
And the absolute value of the difference between the global information ipt and mot | ip
t-mot | is compared with the threshold value β (step S4).
7).

If | ipl-mol | is smaller than α and | ipt-mot | is smaller than β, the position of the k-th row in the set window 51 satisfies the node arrangement condition. So, next, ipline
[K] is compared with 0 (FIG. 14, step S53), and ip
If line [k] = 0, 1 is added to iphit to indicate that a node exists in the kth row (step S54).

Next, 0 is compared with moline [j] (step S55), and if moline [j] = 0, mo is shown to indicate that a node exists in the j-th column.
1 is added to hit (step S56). And i
1 is added to each of plane [k] and moline [j] (steps S57 and S58), and 1 is added to k to check the next row (FIG. 13, step S4).
8), k is compared with ed [j] (step S49).

In step S53, ipline
If [k]> 0, the k-th row has already been counted, and therefore the processing from node step S55 is performed. Similarly, in step S55, moline [j]> 0
If so, since the j-th column has already been counted, the processing from step S57 is performed.

In step S49, k is ed [j]
If the following is true, the processing from step S46 is repeated.
When k exceeds ed [j], 1 is added to j in order to check the next column (step S50), and j is mo.
It is compared with Suu (step S51).

If j is smaller than mosuu, the processing from step S43 is repeated. When j reaches mosuu, the increment of iphit, mohit, ipline [k], and moline [j] is completed.
At this point, the total number of rows in which at least one element satisfying the condition of Expression (7) exists is recorded in ipit, and
In t, the total number of columns in which at least one element satisfying the condition exists is recorded.

Therefore, the global similarity between the input document and the model is calculated by the following equation similar to the equation (8) (step S5).
2), the process ends. Global similarity = iphit / ipsuu + mohit / mosuu (9) The global similarity of Eq. (9) is obtained for each matching table of horizontal ruled lines and vertical ruled lines, and the sum is calculated to obtain the final global similarity. To be

Next, FIG. 16 is a flow chart showing an example of the large classification candidate determination processing in step S13 of FIG. This processing is performed using the above-mentioned global similarity. When the processing is started, the format identification device first sets a set of all models as Smo (step S6).
1), the control variable i indicating the element of Smo is set to 1 (step S62), and the global similarity of the i-th model is compared with the threshold value TH (step S63).

If the global similarity is smaller than TH, the i-th model is removed from Smo (step S64), 1 is added to i (step S65), and i is compared with the total number n of models (step S66). . If i is equal to or less than n, the processing from step S63 is repeated. If the global similarity is equal to or higher than TH in step S63,
The processing from step S65 onward is performed as it is.

When i exceeds n in step S66, the number of elements of Smo is then compared with the number cand of the large classification candidates determined in step S11 of FIG. 9 (step S67).

If the number of elements of Smo is larger than cand,
The elements of Smo are rearranged in the order of global similarity (step S
68), only the upper cand elements are set as Smo (step S69), and the process is ended. If the number of elements of Smo is equal to or less than cand, the process is terminated as it is.

In this way, when the set Smo of large classification candidates is obtained, the detailed classification model matching in step S14 of FIG. 9 is performed on each model included in Smo. FIG. 17 is a flowchart of this detailed classification model matching. When the process is started, the format identification device first creates a matching table as shown in FIG. 10 as in the case of the large classification model matching (step S71).

Next, based on the global information, the i of the input image is
The th ruled line IP (i) is the j th ruled line MO of the model.
Whether or not it corresponds to (j) is judged, and if there is a possibility of correspondence, a node is arranged in the element at the i-th row and the j-th column of the matching table (step S72).

Next, based on the detailed information, among the arranged nodes, those which satisfy a specific relationship with each other, that is, those which are compatible with each other are searched for (step S73). Then, those nodes are regarded as nodes belonging to the same group and are connected by a path. In the case of the matching table of FIG. 10, for example, a path as shown in FIG. 18 is generated.

Here, the fact that the node n (i, j) at the i-th row and the j-th column and the node n (k, l) at the k-th row and the l-th column satisfy a specific relationship means that the i-th ruled line and the k-th line of the input image are satisfied. Indicates that the relative relationship of the ruled line of is proportional to the relative relationship of the j-th ruled line and the l-th ruled line of the model. In other words, when the i-th ruled line of the input image and the j-th ruled line of the model are exactly overlapped, the k-th ruled line of the input image and the l-th ruled line of the model are exactly overlapped.

By connecting such nodes with one path, the nodes are classified into several groups. It is considered that a group with a larger number of nodes strongly indicates that the input document and the model are more similar. Therefore, by targeting such a group, the detailed similarity calculation in model matching can be performed efficiently. It can be carried out.

When searching for a node compatible with a certain node, the node located in the lower right area of the former node is always set as the search target in order to improve the processing efficiency. As a result, it becomes possible to quickly obtain a path having a large number of nodes.

Next, among the obtained sets of paths, combinations of paths that do not contradict each other are obtained, and a combination having the largest number of nodes is searched from some combinations (step S74). Then, the combination of the paths is set as the optimum path set. The two paths do not conflict with each other, that is, the ranges of the ruled line sets corresponding to the nodes in the respective paths do not overlap each other.

Next, the number of ruled lines in the input image is ph, the number of ruled lines in the model is mh, and the number of nodes included in the optimum path set is ma.
xh is the detailed similarity SH between the input image and the model ruled line
Is calculated by the following equation (step S75), and the process is terminated. SH = maxh / ph + maxh / mh (10) This similarity SH is the sum of the ratio of the ruled lines of the input image corresponding to the optimum path set and the ratio of the model ruled lines corresponding to the optimum path set. In general, the larger the characteristic of the ruled line of the input image and that of the model, the larger the value. The final detailed similarity is obtained by obtaining the detailed similarity of the expression (10) for each of the matching tables of the horizontal ruled line and the vertical ruled line and taking the sum of them.

In step S15 of FIG. 9, the format identification device compares the detailed similarities of the models and sets the model having the largest detailed similarity as the format of the table corresponding to the input image.

In the embodiment described above, the position and length of the ruled line are used as the feature amount to create the matching table of the input image and the model. However, in the model matching, any other arbitrary table structure can be used. Can be used.

For example, when the coordinates of the intersection of the vertical ruled line and the horizontal ruled line are (x, y), the feature amount (d
x, dy) is defined by the following equation. dx = (x−x0) × 100 / W dy = (y−y0) × 100 / H (11) Then, a matching table of the intersection on the input document side and the intersection on the model side is created, and the corresponding intersections are possible. A combination of is represented by a node. In this case, when the intersection on the input document side is Ip and the intersection on the model side is Mp, Ip
If the Euclidean distance between Mp and Mp is less than or equal to a predetermined threshold value th, it is determined that these intersections can correspond.

Alternatively, a matching table of the rectangular cells on the input document side and the rectangular cells on the model side is created by using the size and position of the rectangular area (rectangular cell) surrounded by ruled lines on all four sides as the feature amount. Good. In this case, it is determined that rectangular cells having similar sizes and positions can correspond to each other, and the combination thereof is represented by a node.

No matter which feature quantity is used, if the number of rows including one or more nodes and the number of columns including one or more nodes arranged on the matching table are counted, these numbers are obtained. Based on this, it is possible to calculate the global similarity as expressed by equation (9). Therefore, the large classification candidates can be narrowed down using the global similarity.

[0092]

According to the present invention, when a ruled line structure is extracted from an unknown tabular document and matching is performed with a model format registered in a dictionary in advance, a model group to which detailed matching should be applied is effective. It becomes possible to narrow it down. As a result, a stable processing speed can be realized even when the number of dictionary models increases.

[Brief description of drawings]

FIG. 1 is a principle diagram of a format identification device of the present invention.

FIG. 2 is a configuration diagram of an information processing device.

FIG. 3 is a flowchart of a format identification process.

FIG. 4 is a diagram showing a reference width, a reference height, and a reference point.

FIG. 5 is a diagram showing horizontal ruled lines.

FIG. 6 is a diagram showing vertical ruled lines.

FIG. 7 is a diagram showing detailed information of horizontal ruled lines.

FIG. 8 is a diagram showing detailed information of vertical ruled lines.

FIG. 9 is a flowchart of model matching.

FIG. 10 is a diagram showing a matching table.

FIG. 11 is a flowchart (No. 1) of large classification model matching.

FIG. 12 is a flowchart (No. 2) of large classification model matching.

FIG. 13 is a flowchart (part 3) of large classification model matching.

FIG. 14 is a flowchart (No. 4) of large classification model matching.

FIG. 15 is a diagram showing a window of a matching table.

FIG. 16 is a flowchart of a large classification candidate determination process.

FIG. 17 is a flowchart of detailed classification model matching.

FIG. 18 is a diagram showing paths on a matching table.

[Explanation of symbols]

1 storage means 2 comparison means 3 classification means 4 calculation means 11 CPU 12 memories 13 Input device 14 Output device 15 External storage device 16 Medium drive 17 Network connection device 18 Photoelectric conversion device 19 bus 20 Portable storage medium 21 dictionary 31, 32, 33, 34, 36, 37 Ruled lines 35 vertices 41, 42, 43, 44, 45, 46 Ruled line rectangle 51 windows

─────────────────────────────────────────────────── ─── Continuation of the front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06K 9/00-9/82 G06F 17/30 G06F 19/00 G06T 11/60

Claims (12)

(57) [Claims] 1. A format identification device that searches for a format of a plurality of tables to obtain a format of a table corresponding to a given table image, wherein each rule included in each of the formats of the plurality of tables.
For lines, the length and position of the ruled line relative to the entire table
The storing means for storing the outline information indicating the relative value and the outline information of each ruled line extracted from the input image
Summary information for each ruled line included in each of the table formats
Compared to broadcast, and comparing means for obtaining a combination of ruled lines between which a potential corresponding, for each format of said plurality of tables, the table
The number of ruled lines included in the combination of the ruled lines among at least one ruled line of the format and the input image is obtained, and the search target is narrowed down from the format of the plurality of tables based on the obtained number. means and, format to be searched that has been narrowed down by the classifying means
For the format of one table included in the group
Based on the information of the combination of the ruled lines, the input image
Each ruled line included in the image and included in the format of the one table
Detailed matching for detailed matching with each ruled line
Format identification apparatus, comprising a means. 2. The classification means is a method for forming a list of the plurality of tables.
For each of the mats
Ratio of ruled lines included in the combination of ruled lines
And a combination of the ruled lines of the input image
Calculate the sum with the percentage of ruled lines included in
Using the format of the table and the similarity of the input image
2. The search target is narrowed down.
Format identification device. 3. The detailed matching means is configured to input the input image.
Relative length and position between two ruled lines in the image
Information showing the relationship and the format of the one table
Of the length and position between the two ruled lines contained in
The detailed matching is performed using the detailed information indicating the relationship.
The format identification according to claim 1, wherein
Another device. 4. The detailed matching means is configured to input the input image.
Format of p ruled lines included in the image and the one table
P-by-m matrix that represents the correspondence between the m ruled lines included in
Generates a teaching table and the i-th ruled line of the input image is
Corresponding to the jth ruled line in the format of the one table
When the matching table has the capability, the i-th row and the j-th column of the matching table are required.
A node is arranged at the base, and detailed information of the input image and the one
Connect multiple nodes using detailed information in the table format.
Path is generated, and the input image is generated using the generated path.
To obtain the detailed similarity of the format of the one table
The format identification device according to claim 3, which is characterized in that. 5. A format for a plurality of tables is searched for and given.
The format of the table corresponding to the obtained table image is calculated.
A format identification device, Each vertical included in each of the plurality of table formats
Position of intersection of vertical and horizontal ruled lines for ruled lines and each horizontal ruled line
Storage means for storing the outline information representing The outline information of each intersection extracted from the input image
Summary information of each intersection included in each of the table formats
Combining the intersections that have a possibility of correspondence compared to the report
A comparison means for seeking For each of the multiple table formats, the table
Of the intersection of the format and at least one of the input images
The number of intersections included in the combination of the intersections
The format of the plurality of tables is calculated based on the obtained number.
Classification means to narrow down the search target from The format of the search target narrowed down by the classification means.
For the format of one table included in the group
A detailed matrix of the input image and the format of the one table.
It is equipped with detailed matching means for
Format identification device to collect. 6. The format of a plurality of tables is searched for and given.
The format of the table corresponding to the obtained table image is calculated.
A format identification device, Ruled lines included in each of the plurality of table formats
For each rectangular area enclosed by, the size and position of the rectangular area
Storage means for storing schematic information representing the position, The outline information of each rectangular area extracted from the input image is
Each rectangular area contained in each of the number table formats
The quadrangle that may Shape region same
Comparison means to find the combination of the For each of the multiple table formats, the table
A rectangular area of at least one of the format and the input image
Rectangle included in the combination of the rectangular regions
The number of areas is calculated, and the plurality of tables are calculated based on the obtained number.
Classification means to narrow down the search target from the format of The format of the search target narrowed down by the classification means.
For the format of one table included in the group
A detailed matrix of the input image and the format of the one table.
It is equipped with detailed matching means for
Format identification device to collect. 7. A storage medium storing a program for a computer that searches a plurality of table formats and obtains a table format corresponding to a given table image ,
The program is for each rule included in each of the formats of the plurality of tables.
For lines, the length and position of the ruled line relative to the entire table
Storing general information indicating a pair of values, the general information of each ruled line extracted from the input image
Summary information for each ruled line included in each of the table formats
Compared to broadcast, determined combination of ruled lines between which a potential corresponding, for each format of said plurality of tables, the table
The number of ruled lines included in the combination of the ruled lines among at least one ruled line of the format and the input image
Determined, based on the obtained number narrow down the search target from the format of said plurality of tables, one included in the format group narrowed search target
Combining the ruled lines with each other for the table format
Each ruled line included in the input image based on the matching information
And details of each ruled line included in the format of the one table
It is necessary to let the computer perform the process of performing the matching.
Characteristic computer-readable storage medium. 8. A method for searching a format of a plurality of tables,
A table for calculating the table format corresponding to the obtained table image.
A storage medium storing a program for a computer,
The program is Each vertical included in each of the plurality of table formats
Position of intersection of vertical and horizontal ruled lines for ruled lines and each horizontal ruled line
Stores summary information representing The outline information of each intersection extracted from the input image
Summary information of each intersection included in each of the table formats
Combining the intersections that have a possibility of correspondence compared to the report
Ask for For each of the multiple table formats, the table
Of the intersection of the format and at least one of the input images
The number of intersections included in the combination of the intersections
Seeking, From the format of the multiple tables based on the number obtained
Narrow down the search target, One included in the narrowed down search target format group
The input image and the 1
To perform detailed matching with one table format
Computer read-out characterized by causing the computer to perform
Storage media. 9. The format of a plurality of tables is searched for and given.
A table for calculating the table format corresponding to the obtained table image.
A storage medium storing a program for a computer,
The program is Ruled lines included in each of the plurality of table formats
For each rectangular area enclosed by, the size and position of the rectangular area
Stores the summary information that represents the The outline information of each rectangular area extracted from the input image is
Each rectangular area contained in each of the number table formats
Comparing the outline information of
Seeking a combination of For each of the multiple table formats, the table
A rectangular area of at least one of the format and the input image
Rectangle included in the combination of the rectangular regions
Find the number of regions, From the format of the multiple tables based on the number obtained
Narrow down the search target, One included in the narrowed down search target format group
The input image and the 1
To perform detailed matching with one table format
Computer read-out characterized by causing the computer to perform
Storage media. 10. Searching for multiple table formats,
A format identification method for obtaining a table format corresponding to a given table image, wherein each ruled line included in each of the plurality of table formats
For lines, the length and position of the ruled line relative to the entire table
The outline information indicating the opposite value is stored in advance, and the outline information of each ruled line extracted from the input image is stored in the plurality of
Summary information for each ruled line included in each of the table formats
Compared to broadcast, determined combination of ruled lines between which a potential corresponding, for each format of said plurality of tables, the table
Obtains the number of ruled lines contained in the combination of the ruled lines between the at least one of the borders of the format and the input image, narrow down the search target from the format of said plurality of tables based on the obtained number, the search narrowed down One included in the target format group
Combining the ruled lines with each other for the table format
Each ruled line included in the input image based on the matching information
And details of each ruled line included in the format of the one table
A format identification method characterized by various matching . 11. Searching for formats in multiple tables,
Find the table format for a given table image
A format identification method, Each vertical included in each of the plurality of table formats
Position of intersection of vertical and horizontal ruled lines for ruled lines and each horizontal ruled line
Is stored in advance, The outline information of each intersection extracted from the input image
Summary information of each intersection included in each of the table formats
Combining the intersections that have a possibility of correspondence compared to the report
Ask for For each of the multiple table formats, the table
Of the intersection of the format and at least one of the input images
The number of intersections included in the combination of the intersections
Seeking, From the format of the multiple tables based on the number obtained
Narrow down the search target, One included in the narrowed down search target format group
Format of the table The input image and the 1
A detailed match with one table format
A method for identifying a format. 12. Searching for multiple table formats,
Find the table format for a given table image
A format identification method, Ruled lines included in each of the plurality of table formats
For each rectangular area enclosed by, the size and position of the rectangular area
The outline information indicating the location is stored in advance, The outline information of each rectangular area extracted from the input image is
Each rectangular area contained in each of the number table formats
Comparing the outline information of
Seeking a combination of For each of the multiple table formats, the table
A rectangular area of at least one of the format and the input image
Rectangle included in the combination of the rectangular regions
Find the number of regions, From the format of the multiple tables based on the number obtained
Narrow down the search target, One included in the narrowed down search target format group
The input image and the 1
A detailed match with one table format
A method for identifying a format.