JP7458600B2 - Information processing device and program - Google Patents

Description

The present invention relates to an information processing device and a program for processing an image of a form.

2. Description of the Related Art A technology has been put into practical use that reads information written in each entry box from a form that includes a plurality of information entry boxes. In this technology, for each type of form, coordinate information that specifies the area of the entry frame and information that indicates what kind of information the area specified by the coordinate information corresponds to the entry frame are stored in advance. It is associated and stored as area information.

When reading information, the type of form is specified, the area information stored for the specified type of form is referred to, and the information within the information entry frame on the form is read.

Japanese Patent Application Publication No. 2011-169451

However, the above area information is not necessarily prepared. For example, even in a new form or a similar form, the position of the entry frame may have been changed. In such cases, it is desired to simplify the generation of area information by recognizing the area of the entry frame on the read form without any manual operation.

The present invention has been made in consideration of the above-mentioned circumstances, and one of its objectives is to provide an information processing device and program that can recognize the area of the entry frame of an optically read document without manual operation.

In addition, in order to perform optical character recognition, Patent Document 1 discloses a technique for finding an area that is assumed to contain characters by stretching the area of characters that is determined not to have superimposed noise in the row or column direction. Disclosed. However, the technique disclosed in Patent Document 1 cannot recognize the area of the entry frame.

One aspect of the present invention that solves the problems of the above-mentioned conventional examples is an information processing device, comprising: a means for receiving image data of a form; a means for extracting circumscribing rectangles of candidates for entry frames on the form from the received image data of the form; a first recognition means for recognizing at least one first rectangle subgroup including a plurality of the circumscribing rectangles arranged in a first axis direction of the image data of the form; a first grouping means for setting, for each first rectangle subgroup recognized by the first recognition means, a first group rectangle that further circumscribs the plurality of circumscribing rectangles included in the first rectangle subgroup; The image data includes a second recognition means for recognizing at least one second rectangular subgroup including a plurality of the circumscribing rectangles arranged in a second axis direction intersecting the first axis direction, a second grouping means for setting a second group rectangle that further circumscribing the plurality of circumscribing rectangles included in the second rectangular subgroup for each of the second rectangular subgroups recognized by the second recognition means, and a means for recognizing the intersection range between the first group rectangle and the second group rectangle as an entry frame, extracting image data within the recognized entry frame, and providing the image data for predetermined processing.

According to the present invention, it is possible to recognize the entry frame area of an optically read form without any manual operation.

1 is a block diagram illustrating a configuration example of an information processing device according to an embodiment of the present invention. FIG. FIG. 2 is an explanatory diagram illustrating an example of a form to be processed by the information processing apparatus according to the embodiment of the present invention. FIG. 1 is a functional block diagram illustrating an example of an information processing device according to an embodiment of the present invention. FIG. 3 is an explanatory diagram illustrating an example of processing for detecting fill-in box candidates by the information processing device according to the embodiment of the present invention. FIG. 2 is an explanatory diagram illustrating an example of grouping processing performed by the information processing apparatus according to the embodiment of the present invention. FIG. 3 is an explanatory diagram illustrating an example of recognition processing of a fill-in frame by the information processing device according to the embodiment of the present invention. FIG. 7 is an explanatory diagram showing another example of a table format included in a form to be processed by the information processing apparatus according to the embodiment of the present invention.

Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the information processing device 1 according to the embodiment of the present invention includes a control section 11, a storage section 12, an operation section 13, a display section 14, and an interface section 15. This is a general computer device connected to a device such as a scanner 2.

The control unit 11 is a control device such as a processor, and operates according to a program stored in the memory unit 12. In this embodiment, the control unit 11 receives image data of a form obtained by optically reading the form using the scanner 2, for example. In this embodiment, the form read here includes a table (T) in which multiple frames (writing frames) for writing information are aligned in the row and column directions, as shown in FIG. 2.

When the control unit 11 receives the image data of the form, it recognizes a candidate for an entry frame on the form from the received image data, and extracts a circumscribed rectangle for the candidate for the entry frame. The control unit 11 recognizes at least one first rectangular subgroup that is arranged in the first axis direction of the image data of the form and includes the plurality of circumscribed rectangles extracted above, and further recognizes the first rectangular subgroup that is arranged in the first axis direction of the image data of the form. For each, a first group rectangle that further circumscribes the plurality of circumscribed rectangles included in the first rectangle subgroup is set.

The control unit 11 also recognizes at least one second rectangular subgroup of the image data of the form that is arranged in the second axis direction and includes the plurality of circumscribed rectangles, and for each of the recognized second rectangular subgroups. Then, a second group rectangle that further circumscribes the plurality of circumscribed rectangles included in the second rectangle subgroup is set. Here, it is assumed that the first axial direction and the second axial direction intersect with each other (are non-parallel).

The control unit 11 recognizes the intersection range of the first group rectangle and the second group rectangle set here as a writing frame, extracts image data within the recognized writing frame, and subjects it to predetermined processing. The detailed operation of this control section 11 will be described later.

The storage unit 12 is a disk device, a memory device, or the like, and holds programs executed by the control unit 11. This program may be provided by being stored in a computer-readable, non-transitory recording medium, and may be stored in this storage unit 12. Further, this storage section 12 also operates as a work memory for the control section 11.

The operation unit 13 is a keyboard or the like, and accepts a user's operation and outputs the content of the operation to the control unit 11. The display unit 14 is a display or the like, and displays and outputs information according to instructions input from the control unit 11.

The interface unit 15 includes, for example, a USB interface. In one example of the present embodiment, a device such as the scanner 2 is connected to the information processing device 1 via this interface unit 15.

The scanner 2 connected to the information processing device 1 of this embodiment optically reads characters and figures formed on a paper medium and outputs them as image data.

Next, a processing example of the control unit 11 will be described. In the present embodiment, the control unit 11 includes, as illustrated in FIG. It is configured to include a recognition section 25, a second grouping section 26, a frame recognition section 27, and an information output section 28.

The receiving unit 21 receives image data of a form read by the scanner 2 from the scanner 2 via the interface unit 15 . This image data is so-called bitmap image data.

Further, as described above, an example of a form that is a target of processing by the information processing device 1 of the present embodiment has multiple frames for entering information (input frames) in the row direction, as illustrated in FIG. and tables (T) arranged in the column direction. In the example in Figure 2, the table (T) has a plurality of entry frames A arranged in the row direction for entering the name of the person in charge for each time period, and the column direction for writing the name of the person in charge for each time zone. It is assumed that the entry boxes A for entering the names of the persons are arranged in a line. In other words, in this table (T), the name of the person in charge is entered for each location and each time period. In addition, in this table (T), the first row is an arrangement of entry frames (D) in which information identifying the time zone is written, and the first column is filled with information identifying the place in charge. It is assumed that the entry frames (P) are arranged.

Further, the information processing device 1 of the present embodiment sets a first axis direction and a second axis direction intersecting the first axis direction for the image data of the received form. Here, the first and second axial directions may be orthogonal to each other. In the following example, the first axial direction is the row direction (the sub-scanning direction of the scanner 2), and the second axial direction is the column direction (the main scanning direction of the scanner 2).

The entry box candidate extraction unit 22 recognizes entry box candidates on the form from the image data of the form accepted by the acceptance unit 21, and extracts a circumscribing rectangle that circumscribes the area of the recognized entry box candidate. As a specific example, in this embodiment, since the entry boxes are represented by rectangular figures, the entry box candidate extraction unit 22 rotates the image data of the form accepted by the acceptance unit 21 so that the direction of straight lines is aligned in the row direction or column direction by the Hough transform.

Then, the frame candidate extracting unit 22 performs a process (binarization process) in which significant pixels (for example, pixels whose luminance is below a predetermined threshold value are set as black, and pixels that are not otherwise set as white) for each row and each column. The number of significant pixels (pixels that turn black) is counted, and rows or columns in which the number of significant pixels is greater than a count threshold determined by a predetermined method are found. The frame candidate extraction unit 22 extracts a range in which significant pixels are included in the found row or column, and if the size of the range is larger than a predetermined continuity threshold, the continuous range is extracted. is recognized as a ruled line that separates entry frames.

The writing frame candidate extracting unit 22 recognizes a rectangular portion between a pair of ruled lines adjacent to each other in the row direction and partitioned by a pair of ruled lines adjacent to each other in the column direction as a candidate area for a writing frame. . The entry frame candidate extracting unit 22 outputs information representing a rectangle circumscribing the area of each recognized entry frame candidate as information representing a circumscribing rectangle.

In this example, as illustrated in FIG. 4(a), candidates for entry frames that are adjacent to each other share one ruled line. That is, the ruled line on the right side of entry frame A1 is also the ruled line on the left side of entry frame A2, and the ruled line on the lower side of entry frame A1 is also the ruled line on the upper side of entry frame A3.

In another method, the entry frame candidate extracting unit 22 recognizes a rectangle (character rectangle) that circumscribes a significant pixel block (character) in the image data of the form. Here, since a widely known method can be adopted for the process of recognizing character rectangles circumscribing significant pixels constituting a character, a detailed explanation will be omitted here.

The entry frame candidate extraction unit 22 groups the plurality of recognized character rectangles based on predetermined character grouping conditions. Here, the character grouping condition may be, for example, that the distance between adjacent character rectangles is less than a predetermined character spacing threshold. If a ruled line can be separately recognized, the conditions are set such that the above-mentioned character spacing threshold value is satisfied and the character is not divided by the recognized ruled line.

In this case, the entry box candidate extraction unit 22 generates a character string circumscribing rectangle that further circumscribing the multiple character rectangles grouped according to the character grouping conditions, and the area within this generated character string circumscribing rectangle is set as the area of the entry box candidate.

In this example, as illustrated in FIG. 4B, candidate areas A'1, A'2, and A'3 for the entry frame are included in an area partitioned by ruled lines. In this example, the rectangle (circumscribed rectangle) circumscribing the candidate region of the entry frame may be the character string circumscribing rectangle itself.

The first recognition unit 23 selects any one of the plurality of circumscribed rectangles extracted by the entry frame candidate extraction unit 22, and selects one of the plurality of circumscribed rectangles extracted by the entry frame candidate extraction unit 22, and performs a process in the row direction (first direction) of the image data of the form with respect to the selected circumscribed rectangle. Detect other circumscribed rectangles that exist in the axial direction).

The first recognition unit 23 sets a first rectangle subgroup including the selected circumscribed rectangle and other detected circumscribed rectangles.

If there is a circumscribed rectangle that does not belong to any first rectangle subgroup (and a circumscribed rectangle that is not set as a first isolated rectangle, which will be explained later), the first recognition unit 23 selects one of the circumscribed rectangles. Select more and repeat the above process. Note that when there is no other circumscribed rectangle existing in the row direction (corresponding to the first axis direction) of the image data of the form with respect to the selected circumscribed rectangle, the first recognition unit 23 recognizes the selected circumscribed rectangle. is set as the first isolated rectangle.

5, when the first recognition unit 23 selects a circumscribing rectangle A, it virtually sets line segments AH, AB extending the top and bottom sides of the circumscribing rectangle A, and finds other circumscribing rectangles B, C, D, etc. that overlap this range and are different from the circumscribing rectangle A (other circumscribing rectangles). For each of the other circumscribing rectangles it finds, the first recognition unit 23 compares the area RIN between the line segments AH, AB with the area ROUT of the portion outside the line segments AH, AB, and determines whether the area satisfies the condition:
RIN>ROUT, and
ROUT < Rrow_th
Here, Rrow_th is an empirically determined threshold value that determines the extent of the area of the portion outside the line segments AH and AB that is to be included in the first rectangle subgroup to which the selected circumscribing rectangle A belongs.

The first recognition unit 23 then groups the circumscribed rectangles that are determined to satisfy the above conditions and the circumscribed rectangle A into one first rectangle subgroup. In the example of FIG. 5, circumscribed rectangles A, B, and C belong to one first rectangle subgroup, and since RIN>ROUT does not hold, circumscribed rectangle D belongs to the first rectangle subgroup to which circumscribed rectangle A belongs. are not grouped.

For at least one first rectangle subgroup set by the first recognition unit 23, the first grouping unit 24 sets a first group rectangle that further circumscribes the multiple circumscribed rectangles included in the first rectangle subgroup.

The second recognition unit 25 selects any one of the plurality of circumscribed rectangles extracted by the entry frame candidate extraction unit 22, and applies the column direction (second direction) of the image data of the form to the selected circumscribed rectangle. Detect other circumscribed rectangles that exist in the axial direction).

The second recognition unit 25 sets a second rectangle subgroup including the selected circumscribed rectangle and other detected circumscribed rectangles.

If there is a circumscribed rectangle that does not belong to any second rectangle subgroup (and a circumscribed rectangle that is not set as a second isolated rectangle to be described later), the second recognition unit 25 selects one of the circumscribed rectangles. Select more and repeat the above process. Note that when there is no other circumscribed rectangle existing in the column direction (corresponding to the second axis direction) of the image data of the form with respect to the selected circumscribed rectangle, the second recognition unit 25 recognizes the selected circumscribed rectangle. is set as the second isolated rectangle.

As an example, when the second recognition unit 25 selects the circumscribed rectangle A illustrated in FIG. Find circumscribed rectangles (other circumscribed rectangles) E, F, G, etc. that overlap with the circumscribed rectangle A and are different from the circumscribed rectangle A. For each of the other circumscribed rectangles found, the second recognition unit 25 compares the area RIN between the line segments AL and AR with the area ROUT of the portion outside the line segments AL and AR, and calculates the following conditions:
RIN>ROUT, and
ROUT＜Rcolumn_th
Check whether it is satisfied or not. Here, Rcolumn_th is a threshold determined empirically, and to what extent the area outside the line segments AL and AR should be included in the second rectangle subgroup to which the selected circumscribed rectangle A belongs. This is to establish the following.

The second recognition unit 25 then groups the circumscribed rectangles that are determined to satisfy the above conditions and the circumscribed rectangle A into one second rectangle subgroup. In the example of FIG. 5, circumscribed rectangles A, E, and F belong to one second rectangle subgroup, and since RIN>ROUT does not hold, circumscribed rectangle G belongs to the second rectangle subgroup to which circumscribed rectangle A belongs. are not grouped.

The second grouping unit 26 sets, for each at least one second rectangle subgroup set by the second recognition unit 25, a second group rectangle that further circumscribes the plurality of circumscribed rectangles included in the second rectangle subgroup. do.

The entry box recognition unit 27 recognizes the intersection range of the first group rectangle set by the first grouping unit 24 and the second group rectangle set by the second grouping unit 26 as an entry box. In this embodiment, the first grouping unit 24 generally sets a plurality of n first group rectangles. These first group rectangles are rectangles that extend in the row direction and are arranged in the column direction. The second grouping unit 26 generally sets a plurality of m second group rectangles. These second group rectangles are rectangles that extend in the column direction and are arranged in the row direction.

Therefore, there are n×m intersection ranges between the first group rectangle and the second group rectangle recognized by the fill-in frame recognition unit 27, and the fill-in frame recognition unit 27 recognizes n×m write frames. The entry frame recognition unit 27 issues a unique identifier to each recognized entry frame, and also provides coordinate information (the coordinates of the upper left and lower right of the rectangle) specifying the rectangle circumscribing each of the entry frames. The information is obtained and stored in the storage unit 12 as a fill-in frame database in association with the corresponding identifier.

The information output unit 28 refers to the entry box database stored in the memory unit 12, and subjects the image data within the area on the image data of the form represented by the unique identifier and the coordinate information associated therewith (the image data within each entry box) to a predetermined process. Here, the predetermined process may be, for example, character recognition processing based on the image data within each entry box. The information output unit 28 outputs the results of the predetermined process.

[motion]
The information processing device 1 of this embodiment has the above configuration and operates as follows. In the example below, the form read by the scanner 2 has entry frames (portions indicated by A, B, C, D, E, and F) spaced at predetermined intervals, as illustrated in FIG. 6(a). It is assumed that the characters are arranged in a matrix (in the row and column directions), and character strings AAA, BB, CC, DDD, EE, and FFF (the number of characters may be different from each other) are written in each. Furthermore, in the example of FIG. 6(a), it is assumed that there are no ruled lines.

When the control unit 11 of the information processing device 1 receives image data of a form including the table format shown in FIG. 6(a) from the scanner 2 via the interface unit 15, the control unit 11 sets a row direction as a first axis direction and a column direction as a second axis direction intersecting the first axis direction for the received image data of the form.

The control unit 11 may attempt to recognize the ruled lines that separate the entry frames from the image data of this form, but since the ruled lines cannot be recognized, a method that does not use ruled lines, such as a method that recognizes character areas, may be used. do. Specifically, the control unit 11 recognizes a rectangle (character rectangle) circumscribing a significant pixel block (character) from the image data of the received form. Further, the control unit 11 groups the recognized character rectangles under the character grouping condition that the distance to mutually adjacent character rectangles is less than a predetermined character interval threshold. Through this process, the character strings AAA, BB, CC, DDD, EE, and FFF are grouped into different groups.

Then, the control unit 11 generates a character string circumscribing rectangle that further circumscribes the plurality of character rectangles grouped according to the character grouping condition (A, B, C, D, E, F in FIG. 6(a)), The area within the generated character string circumscribing rectangle is set as a candidate area for the entry frame.

Next, the control unit 11 selects one of the plurality of circumscribed rectangles circumscribing each of the candidate regions of the entry frame. For example, assume that the control unit 11 selects a circumscribing rectangle that circumscribes the character string AAA. The control unit 11 detects other circumscribed rectangles existing in the row direction of the image data of the form with respect to the selected circumscribed rectangle. Here, in contrast to the circumscribed rectangle that circumscribes the character string AAA, there is a circumscribed rectangle that circumscribes the character string BB in the row direction of the image data of the form. Therefore, the control unit 11 sets a first rectangle subgroup including the selected circumscribed rectangle (the circumscribed rectangle of the character string AAA) and the other detected circumscribed rectangle (the circumscribed rectangle of the character string BB). ), a first group rectangle X1 is set that circumscribes the circumscribed rectangle included in the first rectangle subgroup.

The control unit 11 also selects one of the circumscribed rectangles that does not belong to any of the first rectangle subgroups, for example, the circumscribed rectangle that circumscribes the character string CC, and sets the image data of the form to the selected circumscribed rectangle. Detect other circumscribed rectangles that exist in the row direction. Here, in contrast to the circumscribed rectangle that circumscribes the character string CC, there is a circumscribed rectangle that circumscribes the character string DDD in the row direction of the image data of the form. Therefore, the control unit 11 sets a first rectangle subgroup that includes the selected circumscribed rectangle (the circumscribed rectangle of the character string CC) and the other detected circumscribed rectangle (the circumscribed rectangle of the character string DDD), and A first group rectangle X2 that circumscribes the circumscribed rectangle included in the subgroup is set.

Similarly, the control unit 11 sets a first rectangle subgroup including a circumscribed rectangle of the character string EE and a circumscribed rectangle of the character string FFF, and sets a first rectangle subgroup that circumscribes the circumscribed rectangle included in the first rectangle subgroup. A group rectangle X3 is set (FIG. 6(b)).

In addition to the process of setting the first group rectangle, the control unit 11 selects one of the multiple circumscribing rectangles that circumscribing each of the candidate regions for the entry frame. For example, assume that the control unit 11 selects a circumscribing rectangle that circumscribing the character string AAA. The control unit 11 detects other circumscribing rectangles that exist in the column direction of the image data of the form for the selected circumscribing rectangle. In this case, for the circumscribing rectangle that circumscribing the character string AAA, there are a circumscribing rectangle that circumscribing the character string CC and a circumscribing rectangle that circumscribing the character string EE in the row direction of the image data of the form. Therefore, the control unit 11 sets a second rectangle subgroup that includes the selected circumscribing rectangle (the circumscribing rectangle of the character string AAA) and the other detected circumscribing rectangles (the circumscribing rectangle of the character string BB and the circumscribing rectangle of the character string EE), and sets a second group rectangle Y1 that circumscribs the circumscribing rectangle included in the second rectangle subgroup, as illustrated in FIG. 6C.

The control unit 11 then selects one of the circumscribing rectangles that does not belong to any of the second rectangle subgroups and repeats the above process. Here, for example, the control unit 11 selects a circumscribing rectangle that circumscribing character string BB, and detects, for the selected circumscribing rectangle, a circumscribing rectangle that circumscribing character string DDD and a circumscribing rectangle that circumscribing character string FFF as other circumscribing rectangles that exist in the column direction of the image data of the form. The control unit 11 sets a second rectangle subgroup that includes the selected circumscribing rectangle (the circumscribing rectangle of character string CC) and the other detected circumscribing rectangles (the circumscribing rectangle of character string DDD and the circumscribing rectangle of character string FFF), and sets a second group rectangle Y2 that circumscribs the circumscribing rectangle included in the second rectangle subgroup.

The control unit 11 recognizes the intersection ranges RA, RB, RC, RD, RE, RF (FIG. 6(d)) of the first group rectangles X1, X2, X3 and the second group rectangles Y1, Y2 as entry frames. Then, information representing a circumscribed rectangle that circumscribes these intersecting ranges is recorded as information on a circumscribed rectangle that specifies each entry frame.

Using this record, the control unit 11 assumes that the image data within each circumscribed rectangle specified by the information included in the record is the image data within each entry frame, and, for example, sets the image data within each entry frame. Performs character recognition processing based on the text and outputs the result of character recognition processing for each entry frame.

[Circumscribing rectangle selection criteria]
In the explanation up to this point, the control unit 11 selects one of the circumscribed rectangles in the processing as the first recognition unit 23, and performs the line direction (or The other circumscribed rectangles existing in the column direction) are detected and grouped. In this case, other circumscribed rectangles that overlap the area obtained by extending the selected circumscribed rectangle in the row direction (or column direction), and the area of the overlapping range is a predetermined area with respect to the area of the entire other circumscribed rectangle. Although the condition is that the grouping is performed when the ratio is larger than the ratio, the grouping conditions are not limited to this.

For example, the control unit 11 employs a criterion based on the distance between a pair of circumscribed rectangles included in a plurality of circumscribed rectangles or a criterion based on the shape of a plurality of circumscribed rectangles as a condition for grouping. Good too.

Here, the criterion based on the distance between a pair of circumscribed rectangles included in a plurality of circumscribed rectangles is, for example, based on the distance between a selected circumscribed rectangle and another circumscribed rectangle that exists in the row direction (or column direction) of the image data of the form. and the distance (shortest distance) to the selected circumscribed rectangle is less than a predetermined grouping threshold, or belongs to the same first rectangle subgroup (or second rectangle subgroup) as the selected circumscribed rectangle. The condition may be that the distance (shortest distance) from the circumscribed rectangle is less than a predetermined grouping threshold.

In this example, the control unit 11 initially exists in the row direction of the image data of the form with respect to the selected circumscribed rectangle, and the distance (shortest distance) to the selected circumscribed rectangle is a predetermined grouping threshold. When finding another circumscribed rectangle that is less than , a first rectangle subgroup to which the other circumscribed rectangle and the selected circumscribed rectangle belong is set. The control unit 11 then selects one of the other circumscribed rectangles that exist in the row direction of the image data of the form with respect to the selected circumscribed rectangle and that belongs to the first rectangle subgroup to which the selected circumscribed rectangle belongs. Each time another circumscribed rectangle whose distance (shortest distance) is less than a predetermined grouping threshold is found, the other circumscribed rectangle found is added to the first rectangle subgroup to which the selected circumscribed rectangle belongs. repeat.

As a result, when two or more tables are arranged consecutively in the first axis direction, the distance is less than the distance between the tables and exceeds the distance of the entry frame within each table. If this is set as the grouping threshold, each entry frame in each table will be identified separately. Note that the control unit 11 may perform similar processing in the column direction as well. In this case, when two or more tables are arranged consecutively in the second axis direction, the distance is less than the distance between the tables and exceeds the distance of the entry frame within each table. If this is set as the grouping threshold, each entry frame in each table will be identified separately.

Further, the control unit 11 may use a criterion based on the shape of a circumscribed rectangle as a condition for grouping. In this case, the control unit 11 clusters the circumscribed rectangles obtained by the entry frame candidate extraction unit 22 using the size (width and height) and the ratio of width to height (aspect). As this clustering method, a widely known method such as the k-means method can be used, so a detailed explanation will be omitted.

For each cluster obtained by clustering, the control unit 11 selects one of the circumscribed rectangles belonging to the cluster, and moves the selected circumscribed rectangle in the row direction (first axis direction) of the image data of the form. Detect and group other circumscribed rectangles existing in the same cluster as the selected circumscribed rectangle (which may also be weighted according to the conditions already described), and form the first rectangle subgroup. It may also be set.

In this example, the table set T illustrated in FIG. 7 is processed as follows. Here, the table T in FIG. 7 has a column header HC that includes N entries for item names in the column direction (N is an integer of 1 or more), and M entries for item names in the row direction (M is 1 It is assumed that the table format is divided into a row header HR, which includes (an integer greater than or equal to)), and a table main body TB, which includes N×M entry frames. Further, the entry frames included in the column header HC each have a width CW and the height CH, the entry frames in the main body TB each have a width CW and a height RH, and the entry frames included in the row header HR each have a width CW and a height CH. Suppose that the height is RW and the height is RH.

Furthermore, it is assumed here that RH<CH and RW>CW. In other words, the height CH of the entry frame for the item name in the column direction is greater than the height RH of the entry frame in the main body TB, and the width RW of the entry frame for the item name in the row direction is the entry frame in the main body TB. is larger than the width CW of.

The control unit 11 extracts each circumscribed rectangle that circumscribes the candidate area of the entry frame, and converts the extracted circumscribed rectangle into three-dimensional vector information of its width, height, and width-to-height ratio (aspect). Clustering based on. As a result of this clustering, in the table set T in the example of FIG. It is classified into a rectangular cluster (cluster II) and a rectangular cluster circumscribing the entry frame in the main body part TB (cluster III).

For each cluster, the control unit 11 selects one of the circumscribed rectangles belonging to the cluster, and selects another circumscribed rectangle existing in the row direction of the image data of the form with respect to the selected circumscribed rectangle. Those belonging to the same cluster as the circumscribed rectangle (which may also be weighted according to the conditions already described) are detected and grouped, and a first rectangle subgroup is set.

As a result, the circumscribed rectangle in cluster I, the circumscribed rectangle in cluster II, and the circumscribed rectangle in cluster III each belong to different first rectangle subgroups.

In addition, as in the row direction, the control unit 11 selects one of the circumscribed rectangles belonging to the cluster for each cluster, and selects one of the circumscribed rectangles belonging to the cluster, and selects one of the circumscribed rectangles belonging to the cluster, and other Circumscribed rectangles that belong to the same cluster as the selected circumscribed rectangle (furthermore, the conditions described above may be weighted) are detected and grouped, and a second rectangle subgroup is set.

As a result, the circumscribed rectangle in cluster I, the circumscribed rectangle in cluster II, and the circumscribed rectangle in cluster III each belong to different second rectangle subgroups.

Further, while sequentially selecting clusters, the control unit 11 recognizes the intersecting ranges of the first rectangular subgroup to which the circumscribed rectangle of the selected cluster belongs and the second rectangular subgroup of the selected cluster as respective entry frames.

In the example of FIG. 7, when selecting cluster I to which the circumscribed rectangle of the column-direction item name entry frame belongs, the control unit 11 selects the first group that includes all the circumscribed rectangles of the column-direction item name entry frames. Recognizing each column-direction item name entry frame as an entry frame, which is the intersection range of the rectangle and the second group rectangle that includes one circumscribing rectangle for each of the column-direction item name entry frames. Become.

Similarly, when selecting cluster II to which the circumscribed rectangle of the entry frame for the item name in the row direction belongs, the control unit 11 selects a first group rectangle that includes one circumscribing rectangle for each entry frame for the item name in the row direction. The entry frame for the item name in the row direction, which is the intersecting range of the second group rectangle that includes all the circumscribed rectangles of the entry frame for the item name in the row direction, is recognized as the entry frame.

Further, when cluster III is selected, the control unit 11 recognizes the intersecting ranges of the first group rectangle and the second group rectangle, each of which includes the circumscribed rectangle of cluster III, as a writing frame.

[Modified example]
In another example of this embodiment, the control unit 11 may recognize the table as follows. The control unit 11 recognizes a rectangular area from the image data of the form received from the scanner 2 etc. using processing such as binarization processing, gradation inversion, and outline extraction, and extracts the rectangular area from the recognized rectangular area. Issue unique identification information for each.

The control unit 11 controls the height (main scanning direction H of the scanner 2), width (sub scanning direction W of the scanner 2), and coordinate information representing the position (for example, the upper left corner of the rectangular area) of each recognized rectangular area. (which may be vertex coordinates) are stored in the storage unit 12 in association with the identification information of the corresponding rectangular area.

The control unit 11 refers to the information on the rectangular area stored in the storage unit 12, classifies the recognized rectangular area into clusters based on the height and width information, and obtains the classification result. Based on this classification result, rectangular areas having substantially the same height and width are classified as clusters, and a list of identification information of rectangular areas belonging to the cluster is obtained for each cluster. As this cluster classification method, a widely known method such as the K-means method can be used.

The control unit 11 sequentially selects the clusters resulting from the classification, and when a plurality of rectangular regions are classified into the selected cluster, the control unit 11 selects identification information of each rectangular region classified into the selected cluster. and select one of the identification information. Then, the control unit 11 acquires coordinate information of the rectangular area specified by the selected identification information.

The control unit 11 selects a rectangle specified by the identification information (including the selected identification information) classified into the selected cluster, which exists in the width direction (assumed to be the X-axis direction) of the point represented by the acquired coordinate information. Find the shape region. This processing is performed, for example, by searching for identification information associated with coordinate information whose difference from the value of the acquired coordinate information in the height direction (assumed to be the Y-axis direction) is less than a predetermined threshold.

The control unit 11 also controls, for each of the rectangular areas (that is, rectangular areas arranged in a line in the X-axis direction) found in this process, the height of the point represented by the coordinate information of the rectangular area. A rectangular area specified by the identification information classified into the selected cluster that exists in the direction (Y-axis direction) is found. This process is performed, for example, by searching for identification information associated with coordinate information whose difference from the value of the coordinate information in the height direction (X-axis direction) is less than a predetermined threshold.

Through this process, the control unit 11 finds rectangular regions arranged in a row in the Y-axis direction for each of the rectangular regions arranged in a row in the X-axis direction. Then, the control unit 11 sorts the found Y coordinate values of the coordinate information of the rectangular areas arranged in a line in the Y-axis direction in descending order or ascending order, and arranges the Y coordinate values obtained by sorting. It is checked whether the values of two Y coordinates adjacent to each other in the list are less than a predetermined threshold. This threshold value is, for example, a value determined by a predetermined method on the height of the rectangular area belonging to the selected cluster (for example, a constant, or a value obtained by multiplying the height by a predetermined real number coefficient).

Then, the list of Y coordinate values is divided at a position where it is determined that the values of two adjacent Y coordinates are not below a predetermined threshold value.

The control unit 11 generates a rectangle corresponding to the coordinate information related to the Y coordinate value included in the list of Y coordinate values obtained by dividing (or one list of Y coordinate values if not divided). Shape regions are grouped, and among the coordinate information of rectangular regions belonging to the group, the coordinate information with the smallest Y coordinate value is set as the coordinate information of the group.

The control unit 11 obtains the above-mentioned groups for each of the rectangular regions arranged in a line in the X-axis direction, and determines the groups corresponding to each other (the difference in Y coordinate values of the coordinate information of the groups is a predetermined threshold). (groups below the value) are combined into one group.

The control unit 11 then finds a group of rectangular areas that are aligned in a line in the X-axis direction and arranged with no space between them in the vertical direction. The control unit 11 then recognizes these rectangular areas as the cells of a table, and recognizes the group of rectangular areas that it has found as the range of the table.

Note that the control unit 11 also arranges the X coordinate values of the coordinate information of the rectangular areas arranged in a row in the X-axis direction in descending order or ascending order. The list of X coordinate values obtained by rearranging and rearranging is divided into groups (hereinafter referred to as width direction groups for differentiation) at positions where two adjacent X coordinate values exceed a predetermined threshold. It may also be a thing. In this case, the threshold value may be, for example, a value determined by a predetermined method for the width of the rectangular region belonging to the selected cluster (for example, a constant, or a value obtained by multiplying the width by a predetermined real number coefficient). In this case, for each divided width direction group, for each of the rectangular regions belonging to one width direction group, rectangular regions arranged in a line in the Y-axis direction are found and divided into groups. You can also use it as

Note that the width and height of all cells in a table are not necessarily uniform (for example, some columns may be wider), so clusters may be generated by width or height.

In this example, the control unit 11 refers to the information on the rectangular area stored in the storage unit 12, classifies the recognized rectangular area into clusters based on the height information, and uses the results of the classification as clusters. 1 is stored in the storage unit 12 as the classification result. The control unit 11 also refers to the information on the rectangular area stored in the storage unit 12, classifies the recognized rectangular area into clusters based on the width information, and uses the classification result as the second classification result. It is stored in the storage unit 12 as .

Then, the control unit 11 refers to the first classification result and determines whether the rectangular areas belonging to the same classification (rectangular areas having substantially the same height) are in the width direction of the rectangular area, that is, in the sub-scanning direction of the scanner 2. Determine whether or not there are multiple arrays in the direction. That is, the control unit 11 sequentially selects the clusters of the second classification results that are the classification results, and when a plurality of rectangular regions are classified into the selected clusters, the control unit 11 selects the clusters of the second classification results that are the classification results. The identification information of each rectangular area classified into clusters is obtained, and one of the pieces of identification information is selected. Then, the control unit 11 acquires coordinate information of the rectangular area specified by the selected identification information.

The control unit 11 controls the identification information (including the identification information selected above) that is classified into the cluster of the selected second classification result that exists in the width direction (assumed to be the X-axis direction) of the point represented by the acquired coordinate information. Find a rectangular area specified by the information.

The control unit 11 also sequentially selects rectangular areas found in this process (i.e., rectangular areas arranged in a row in the X-axis direction) and finds rectangular areas that exist in the height direction (Y-axis direction) of the point represented by the coordinate information of the selected rectangular area and that belong to the same cluster of the first classification result as the selected rectangular area.

Through this process, the control unit 11 controls the rectangular regions arranged in a row in the X-axis direction and having substantially the same height to each other, and in the Y-axis direction in a row in the Y-axis direction. Find substantially common rectangular areas. Then, the control unit 11 sorts the found Y coordinate values of the coordinate information of the rectangular areas arranged in a line in the Y-axis direction in descending order or ascending order, and arranges the Y coordinate values obtained by sorting. It is checked whether the values of two Y coordinates adjacent to each other in the list are less than a predetermined threshold. This threshold value is, for example, a value (for example, a constant) determined by a predetermined method based on the height of a rectangular area belonging to the selected cluster.

Then, the list of Y coordinate values is divided at positions where it is determined that two Y coordinate values adjacent to each other do not fall below a predetermined threshold.

The control unit 11 generates a rectangle corresponding to the coordinate information related to the Y coordinate value included in the list of Y coordinate values obtained by dividing (or one list of Y coordinate values if not divided). Shape regions are grouped, and among the coordinate information of rectangular regions belonging to the group, the coordinate information with the smallest Y coordinate value is set as the coordinate information of the group.

The control unit 11 obtains the above-mentioned groups for each of the rectangular regions arranged in a line in the X-axis direction, and determines the groups corresponding to each other (the difference in Y coordinate values of the coordinate information of the groups is a predetermined threshold). (groups below the value) are combined into one group.

Thereby, the control unit 11 finds a group of rectangular areas arranged in a line in the X-axis direction and without any distance in the height direction. The control unit 11 then recognizes the rectangular area as each cell of the table, and recognizes the found group of rectangular areas as the range of the table.

Here too, the control unit 11 may perform a process of dividing into groups in the X-axis direction as well. This process is similar to the process already described, and the explanation will be repeated, so the explanation will be omitted.

Through this processing, the control unit 11 finds a part in which rectangular areas having a common height and width are arranged consecutively in a grid pattern (without any spacing), recognizes it as a table, and also recognizes it as a table. The rectangular areas arranged in a shape are recognized as table cells.

Alternatively, the control unit 11 finds a part where rectangular regions having a common height are arranged consecutively (without any interval) in the X-axis direction, and each of the rectangular regions included in the found arrangement , a portion where rectangular regions having a common width are arranged consecutively (without any interval) in the Y-axis direction is found, and the found portion is recognized as one table. Further, a rectangular area included in the portion recognized as the table is recognized as a cell of the table.

With this method as well, the area of the entry frame of the optically read form can be recognized without any manual operation and with a relatively low processing load.

[Effects of embodiment]
According to this embodiment, the area of the entry frame of the optically read form can be recognized without any manual operation, regardless of the presence or absence of ruled lines. Furthermore, the area of the entry frame can be recognized by distinguishing between the item name and the main body of the table.

1 Information processing device, 2 Scanner, 11 Control unit, 12 Storage unit, 13 Operation unit, 14 Display unit, 15 Interface unit, 21 Acceptance unit, 22 Entry frame candidate extraction unit, 23 First recognition unit, 24 First grouping section, 25 second recognition section, 26 second grouping section, 27 entry frame recognition section, 28 information output section.

Claims (4)

means for accepting image data of a form;
means for extracting a circumscribed rectangle of a candidate entry frame on the form from the image data of the accepted form;
A first circumscribing rectangle of a plurality of candidates for the entry frame arranged in the first axis direction of the image data of the form, the first circumscribing rectangle including a circumscribing rectangle of the plurality of candidates for the entry frame that satisfies a predetermined criterion. first recognition means for recognizing at least one rectangular subgroup;
a first grouping means for setting, for each first rectangular subgroup recognized by the first rectangular subgroup, a first group rectangle that further circumscribes a circumscribed rectangle of a plurality of entry frame candidates included in the first rectangular subgroup; and,
A plurality of circumscribed rectangles of candidates for the plurality of entry frames arranged in a second axis direction intersecting the first axis direction of the image data of the form, the plurality of entries satisfying a predetermined criterion. a second recognition means for recognizing at least one second rectangle subgroup including the circumscribed rectangle of the frame candidate ;
For each of the second rectangle subgroups recognized by the second rectangle subgroup, a second grouping rectangle is set that further circumscribes the circumscribed rectangles of the plurality of entry frame candidates included in the second rectangle subgroup. means and
Recognizing the intersecting range of the first group rectangle and the second group rectangle as a writing frame, extracting image data within the recognized writing frame and subjecting it to predetermined processing;
The information processing device wherein the predetermined criteria used by the first recognition unit or the second recognition unit include criteria based on the shapes of the plurality of circumscribed rectangles. The information processing device according to claim 1 ,
The predetermined criterion used by the first recognition means or the second recognition means may further be a criterion based on a distance between a pair of circumscribed rectangles included in the circumscribed rectangles of the plurality of entry frame candidates , or Criteria based on the shape of the circumscribed rectangle of the frame candidate ,
An information processing device that is a standard that includes at least one of the following. 3. The information processing device according to claim 1 ,
The predetermined processing includes character recognition processing based on image data included in the entry box. computer,
means for accepting image data of a form;
means for extracting a circumscribed rectangle of a candidate entry frame on the form from the image data of the accepted form;
A first circumscribing rectangle of a plurality of candidates for the entry frame arranged in the first axis direction of the image data of the form, the first circumscribing rectangle including a circumscribing rectangle of the plurality of candidates for the entry frame that satisfies a predetermined criterion. first recognition means for recognizing at least one rectangular subgroup;
A first grouping means for setting, for each first rectangular subgroup recognized by the first rectangular subgroup, a first group rectangle that further circumscribes a circumscribed rectangle of a plurality of entry frame candidates included in the first rectangular subgroup. and,
A plurality of circumscribed rectangles of candidates for the plurality of entry frames arranged in a second axis direction intersecting the first axis direction of the image data of the form, the plurality of entries satisfying a predetermined criterion. a second recognition means for recognizing at least one second rectangle subgroup including the circumscribed rectangle of the frame candidate ;
For each of the second rectangular subgroups recognized by the second rectangular recognition means, a second grouping is performed in which a second group rectangle is further circumscribed to a circumscribed rectangle of a plurality of entry frame candidates included in the second rectangular subgroup. means and
means for recognizing the intersecting range of the first group rectangle and the second group rectangle as a writing frame, extracting image data within the recognized writing frame and subjecting it to predetermined processing;
function as
The predetermined standard used by the first recognition means or the second recognition means includes a standard based on shapes of the plurality of circumscribed rectangles.

Images