JP2012141670A - Apparatus, method and program for recognizing form - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus for recognizing an unspecified number of kinds of forms without using a form defined in advance, while increasing binarization processing speed without deteriorating character recognition accuracy.SOLUTION: The form recognition apparatus recognizes a character written on a form including a table composed of cells partitioned by ruled lines and including or not including a character or a character string. The form recognition apparatus defines a table structure for determining arrangement of cells of the table, extracts a title which is a predetermined character or character string from an image, identifies a position of a cell including the title in the table structure, detects a filled-in cell filled with any character or character string among the cells in the defined table structure, and selects only the detected filled-in cell as a binarization cell. The binarization cell in the table is binarized to generate a binary image, thereby recognising a character written in the form with the use of the binary image.

Description

  The present invention relates to a form recognition apparatus, method, and program used for recognizing characters on a form in a document having a table such as a form, and particularly includes binarization processing of image data having a table such as a form. The present invention relates to a form recognition apparatus, method, and program.

  In today's business activities, especially corporate activities, there are many cases where many types of forms are handled simultaneously. For example, when a customer makes a transaction with a financial institution, the customer fills in one of a plurality of forms and submits it to the financial institution. The financial institution that has received the completed form from the customer performs a predetermined process and a response to the customer based on the description content of the form. The contents of the form include numbers such as amounts. Even if such multiple types of forms are different in size of the form itself, the ruled lines defining the table cells are different, or the ruled line structure is the same, the items to be entered in the table cells are different. It ’s different. In addition, for example, in a form for requesting processing for a plurality of different objects, such as a transfer request form used in a financial institution, a table entry area may differ depending on a customer.

  Many of these forms have been processed as digitized documents. For this reason, an image recognition technique has attracted attention as a technique for converting a paper document into an electronic document. Conventionally known image recognition techniques include OCR (Optical Character Recognition Recognition).

2. Description of the Related Art In recent years, an apparatus that automatically reads the contents described in an electronic form for processing an unspecified number of types of forms with a single machine has come to be known.
In such an apparatus, in order to process an unspecified number of types of forms, the ruled lines of the table on the form are recognized, the structure of the table cell is defined, the table header (item) is recognized, It is necessary to infer the format of the form by automatically inferring the contents to be included in each cell from the type of heading. In this case, the method of recording the format of the form in advance and specifying the format of the form by reading the identification symbol attached to the form is not applicable in this case.

  At this time, generally, each cell of the table included in the form is filled with a headline character, a character indicating the content of the headline, a number, a symbol, or the like. It is necessary to automatically associate letters, numbers, symbols, etc. related to the heading. For example, if the headline is “amount”, the item related to the headline is “number” such as yen or dollar. If the form format is predefined, such a correspondence can be stored in the database. However, if the form format cannot be defined in advance, it is necessary to automatically generate a correspondence.

  In order to read a paper form, recognize a ruled line of a table, and perform character recognition using OCR or the like, it is necessary to generate a binary image of the entire form before performing recognition processing. Character recognition such as OCR is performed using the binary image. Generally, in order to process an unspecified number of types of forms, it is necessary to binarize the entire form.

  For example, for a plurality of headings (items), a form in which an entry field for entering data characters such as numbers and kanji related to the headings and preprinted item names are arranged in a table is optically provided. An optical character recognition device that recognizes data characters (keywords) entered in an entry field from a read image is known (Patent Document 1). In an example of such an apparatus, OCR processing is performed on the entire form image using a system dictionary to recognize table structures and characters in the cells of the table, and whether or not OCR processing is necessary again for each heading is determined in advance. By referring to the defined re-OCR processing designation information, specifying the entry field to be subjected to OCR again, and performing partial OCR processing for each entry using a suitable dictionary for the specified entry field. Included is an apparatus that can cope with forms whose layouts are not registered in advance and whose items have different layouts. For example, in the OCR process for the entire form image, the layout of each heading cell of the form is recognized, and in the partial OCR, a user dictionary suitable for the target entry field, for example, a number is entered for the entry field where the amount is to be entered. A stored dictionary can be used. In this method, since the recognition method is changed, such as changing the dictionary used in the OCR process according to the attribute of the item name, the recognition accuracy of the data characters can be improved.

In addition, in order to speed up the processing in the apparatus as described above, an apparatus that preferentially searches an area having a high probability that an item name character is written is known (Patent Document 2).
Furthermore, there is an image processing apparatus that improves the accuracy of character recognition by generating a binary image suitable for character recognition of a form.

  As one example of such an apparatus, there is known an apparatus that determines the presence or absence of characters in a certain area and fills the area with white pixels when it is determined that no character exists in the area (Patent Literature). 3). This device determines the presence or absence of a character by paying attention to the number of edges of connected components in a predetermined area, for example, the number of black and white inversions when scanning in the horizontal or vertical direction within the area. If it is determined that there is no pixel, processing for making all the areas white pixels is performed.

  In addition, it is an image processing device that can contribute to the subsequent character recognition and other processing by removing the influence of background noise from the image in which characters are entered, and the density distribution of blank areas that are not filled in or areas of blank forms prepared in advance And comparing the density distribution of the small area obtained by dividing the arbitrarily designated area to determine whether or not there is a character in the small area, and the density of the area determined to have no character input, There is an apparatus that removes noise from an original image by replacing it with a predetermined background density (Patent Document 4).

Japanese Patent No. 4347777 JP 2009-93305 A JP 2000-331118 A Japanese Patent No. 4393411

  However, in general, forms have different table structures and different table entry areas depending on customers. When a form is recognized using the above-described conventional technique, there is a problem that it takes time to generate a binarized image used for character recognition from an electronic document.

  Also, in the device that determines the presence / absence of characters in each character region and determines that there are no characters in the region, the area for performing the process of calculating the binarization threshold in the device that fills the region with white pixels is Since there is no change, there is a problem that it is difficult to significantly reduce the time required for the binarization processing.

  Therefore, in a form recognition apparatus that recognizes an unspecified number of types of forms without using a predefined form, there is a demand for an apparatus that speeds up the binarization process without degrading the character recognition accuracy. .

  The form recognition apparatus according to one aspect of the present invention is a cell delimited by ruled lines, and includes characters written in a form including a table including or not including a character or a character string. A form recognition apparatus for recognizing a table structure defining means for defining a table structure for determining an arrangement of cells in the table, and extracting a headline that is a predetermined character or character string from the image Among the cells of the table structure defined by the means and the table structure defining means, the position of the header cell that is a cell including the heading is specified in the table structure. A filled cell detecting means for detecting filled cells in which characters or character strings are written, and only the filled cells detected by the filled cell detecting means are selected as binarization target cells. Binarization target cell selection means, binarization means for performing binarization processing of the binarization target cells in the table and generating a binary image, and being described in the form from the binary image Character recognition means for recognizing a character being present.

  A form recognition method and program according to another aspect of the present invention are a method for realizing means used in the form recognition apparatus, and a program including computer-recognizable instructions for realizing the function of the means used in the form apparatus by a computer. is there.

  In such a form recognition method and apparatus, a form can be recognized without using a predefined form, and only a partial area of the table on the form is binarized to perform binary processing. The time required for the valuation process can be reduced. As a result, the speed can be increased without degrading the recognition accuracy of the free forum form recognition.

It is a block diagram of the apparatus of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of this invention. It is a figure for demonstrating the process in a table structure information extraction part. It is a figure which shows the example of the form used as the process target by the apparatus of this invention. FIG. 5 is a diagram illustrating a part of the example of the form illustrated in FIG. 4. It is a block diagram of the apparatus of Example 1 of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of Example 1 of this invention. It is an example of the binary image which left the ruled line. It is a figure which shows the example of the form which the apparatus of Example 1 of this invention makes into a process target. It is a block diagram of the apparatus of Example 2 of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of Example 2 of this invention. It is a figure which shows a part of example of the form which the apparatus of Example 2 of this invention makes into a process target. It is a figure which shows the one part non-binarization object range of the example of the form shown in FIG. It is a block diagram of the apparatus of Example 3 of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of Example 3 of this invention. It is a figure which shows a part of example of the form which the apparatus of Example 3 of this invention makes into a process target. It is a block diagram of the apparatus of Example 4 of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of Example 4 of this invention. It is a figure explaining extraction of the ladder frame in the processing method of the keyword recognition technique applied to the apparatus of Example 4 of this invention. It is a figure which shows the binarization non-target range of the example of the form which the apparatus of Example 4 of this invention makes into a process target. It is a figure explaining the grouping of the ladder frame in the processing method of the keyword recognition technique applied to the apparatus of Example 4 of this invention. It is a block diagram of the apparatus of Example 5 of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of Example 5 of this invention. It is a figure which shows the binarization non-target range of a part of the example of the form which the apparatus of Example 5 of this invention processes. It is a block diagram of the apparatus of Example 6 of this invention. It is a flowchart which shows the processing method of the keyword recognition technique applied to the apparatus of Example 6 of this invention. It is a figure explaining the check of the ruled line which divides | segments the cell up and down in the processing method of the keyword recognition technique applied to the apparatus of Example 6 of this invention. It is a figure which shows the process which makes some Kana item flags 1 of the example of the form which the apparatus of Example 6 of this invention processes as a non-binarization object range.

(General explanation)
The form recognition apparatus according to one aspect of the present invention is a cell delimited by ruled lines, and includes characters written in a form including a table including or not including a character or a character string. A form recognition apparatus for recognizing a table structure defining means for defining a table structure for determining an arrangement of cells in the table, and extracting a headline that is a predetermined character or character string from the image Among the cells of the table structure defined by the means and the table structure defining means, the position of the header cell that is a cell including the heading is specified in the table structure. A filled cell detecting means for detecting filled cells in which characters or character strings are written, and only the filled cells detected by the filled cell detecting means are selected as binarization target cells. Binarization target cell selection means, binarization means for performing binarization processing of the binarization target cells in the table and generating a binary image, and being described in the form from the binary image Character recognition means for recognizing a character being present.

Here, the image of the form may be a color image.
Here, the table structure demarcating means performs binarization processing for the entire digitized form image and performs processing for leaving only ruled lines from the binarized image. This processing may be performed, for example, by extracting horizontal lines and vertical lines on the form image. Further, the horizontal and vertical lines on the form image may be extracted, and the remaining range may be overcoated with a background color, for example, white. When the ruled line of the table is determined, cells delimited by the ruled line are determined, and a column number and a row number can be assigned to each cell. Further, as described below, the table structure defining means may extract a combined cell including a plurality of cells such as a ladder frame cell by extracting a line that is not a solid line such as a dotted line or a broken line.

  Further, the table structure defining means detects the point in the image by detecting a point where at least two lines extend from the point, that is, an intersection point, or a line segment where at least one end contacts another line. The ruled line of the table may be detected. Further, the outer frame of the table may be demarcated by extracting the intersection point closest to the outermost side, that is, the top, bottom, left and right frames of the form.

  Here, the headline extraction means refers to a character string while referring to a headline dictionary in which headlines are stored in advance from a binarized image obtained by binarizing all or part of an electronic form image. Recognize and extract the headline on the form. In general, headings are often written in the upper row of the table, for example, in the first row. Therefore, character recognition in the upper row of the table may be performed with priority by referring to the table structure defined by the table structure defining means.

  Here, the heading position specifying means specifies information related to the position of the heading, that is, the row number and column number in the table. More specifically, the table structure defined by the table structure defining unit is compared with the position extracted by the headline extracting unit and its position, for example, the distance from the upper left corner of the form, and the position information in the table of the headline is compared. That is, the column number and the row number are specified.

  Here, in the binarization target cell selection means, the area of the table relating to the specific heading is excluded from the binarization target. For example, a column with a specific heading is excluded from binarization. In other words, only a row or column area of a table related to a specific heading is set as a binarization target, and cells included in the area are selected as binarization target cells.

  However, the binarization target cell selection unit does not always have to select the binarization target area in units of columns in the table. For example, in the case where two entry fields containing the same information are prepared in one row, such as “name” and “phonetic”, only two cells where “phonetic” is written are stored. It is also possible to make it a target for valuation. In this case, the binarization target cell may be selected every other row.

  Here, the binarization means binarizes the cell selected by the binarization target cell selection means. It is desirable that the binary image obtained by using the binarization means is suitable for the character recognition technique used by the character recognition means.

Here, the characters include numbers, symbols, alphabets, katakana, hiragana, kanji, and the like. In some cases, a symbol representing a character such as a barcode may be used.
The ruled line may be a solid line, a dotted line, a broken line, a one-dot chain line, or the like. For example, ladder frame cells for entering a string of numbers such as monetary amounts are separated by dotted lines. As described above, the ladder frame may be extracted by the table structure defining means.

The filled-in cell detection means determines whether or not a character is written in each cell of the table defined by the table structure defining means, and detects a filled-in cell in which the character is written.
Whether or not a character is written in a certain cell may be determined by checking the number of black pixels in the cell. Alternatively, the color of an entire cell may be detected, and it may be determined from the color whether or not characters are entered in the cell. More specifically, it may be determined from the brightness of the color. In this way, in the method of determining the presence / absence of the entry of a character in a certain cell from the brightness, it is only necessary to measure the color once for one cell, compared to other methods such as detection of the number of edges. Thus, it is possible to determine whether or not a character in a certain cell has been entered at high speed.

  In addition, when the form image is a color image, the filled-in cell detecting means fills in the cell using a vector in the HSV color space composed of three components of hue, saturation and brightness of the cell color. It may be determined whether or not.

  In general, in the HSV space vector, the range of the hue axis representing the color is 0 ° to 360 °, the saturation representing the vividness of the color is 0 to 100%, and the lightness representing the brightness is 0 to 100%. If the brightness value is small, it is close to black. For example, a cell in which a cell including a predetermined number or more of pixels in a cell having a lightness less than 20% is entered may be determined.

The filled-in cell detection means may determine whether or not a cell has been filled using a vector in the RGB space instead of a vector in the HSV space of the cell color.
Only the cells detected by the entered cell detection means are set as binarization target cells, and binarization is performed by the binarization means to generate a binary image.

The character recognition means recognizes characters described in the form from the binary image. As the character recognition means, a known technique such as OCR can be used.
Thus, in the form recognition apparatus according to this aspect, it is possible to recognize a form without using a predefined form, and furthermore, by performing only a partial area of the table on the form, binarization processing is performed. The time required for the binarization process can be reduced. As a result, the speed can be increased without degrading the recognition accuracy of the free forum form recognition.

  In the form recognition apparatus according to another aspect of the present invention, the binarization target cell selecting unit is further positioned at a relative position corresponding to the heading among the cells of the table structure defined by the table structure defining unit. When a cell to be processed is not a completed cell, a cell in a predetermined relative position with respect to a cell that is not the completed cell is excluded from the binarization target cell.

  In the case of a form, when there are many unfilled cells in the detail line of the table, it is highly possible that the detail line is an unfilled detail. Therefore, when the number of unfilled cells in the detail row is a certain number or more, the time required for the binarization processing of the form can be reduced by not binarizing the cells in the entire row.

  The form recognition apparatus according to another aspect of the present invention further includes a plurality of cell defining means for defining, in the table structure, a plurality of cells located at relative positions corresponding to the headings, and the binarization target cell selection The means recognizes the plurality of cells as the binarization target cells.

  Here, the plurality of cells defined by the plurality of cell defining means include ladder frame cells. When checking the number of black pixels in a cell, the ladder frame cell detects a cell row in which the number of black pixels above a certain threshold is periodically represented by a ruled line such as a dotted line that divides the ladder frame in the table. Can be recognized by. The ladder frame cell may be regarded as a single cell as a plurality of cells as a binarization target.

  In a form, a ladder frame is often used for entering numbers. In general, in the case of a form, there is often a subtotal column in the bottom line. In the bottom row of the table, the subtotal column is extracted by extracting the cell column in which the number of black pixels equal to or greater than a certain threshold appears periodically in the table, and the subtotal column is set as a single cell. You may consider it.

Further, the plurality of cells may simply be a collection of a plurality of cells.
In this way, for example, by recognizing a ladder frame as a plurality of cells and considering it as a binarization target cell, the number of binarization processes can be reduced, and binarization of a form whose format is not given in advance. The time required for processing can be further reduced.

  Further, for example, in the case where a column for entering a number of digits is not a ladder frame cell but a form that is a single cell, characters are not always written from end to end of the cell. In such a case, the binarization process may be omitted by setting an area other than the area where characters are entered as a non-binarization target area.

  The form recognition apparatus according to another aspect of the present invention further includes a filled-in multiple-cell detecting unit that detects filled-in cells in which cells in which characters are written are continuous among the plurality of cells, The binarization target cell selection means detects the filled-in plural cells among the plurality of cells located at a relative position corresponding to the heading with respect to the position of the cell including the heading specified by the heading position specifying means. Only the plurality of cells detected by the means are selected as binarization target cells.

  For example, the amount entry column in the form table has a large number of digits so as not to cause an overflow of the amount. In the form, the subtotal column at the bottom of the table is the maximum number of digits. Then, a number smaller than the maximum number of digits is entered in the line above it. Therefore, cells with digits larger than the maximum number of digits are blank. If the binarization process for the region can be omitted, the area for the binarization process can be reduced.

  The form recognition apparatus according to another aspect of the present invention further includes a merged cell defining means for defining, in the table structure, a plurality of cells located at relative positions corresponding to the headings, and the binarization target cell selection The means, with respect to the position of the cell including the heading specified by the heading position specifying means, only a part of the merged cell located at a relative position corresponding to the heading is used as the binarization target cell. It is characterized by selecting.

  Here, the “combined cell” is also referred to as a ladder frame cell, a ladder-type cell, or the like, and refers to a cell in which each digit is divided by a dotted line or a broken line to enter a number of digits. When not only numbers but also hiragana and katakana are entered, cells in which each digit is delimited by a dotted line or a broken line are included in the combined cells in order to process a cloud point or the like as one character.

  In this aspect, a filled cell detection for detecting a filled cell in which any character or character string is written in the cell from at least some of the cells of the table structure defined by the table structure defining means. Means for defining the maximum number of written cells in the merged cell, and the binarization target cell selecting means is further configured for a cell position including the heading. The maximum number of cells may be selected as the binarization target cell from the plurality of cells included in the merged cell located at the relative position corresponding to the heading depending on the heading.

  In general, a form including a monetary amount may include a subtotal column or a total column in the bottom column of the table. Then, the number entered in each row of the column has a smaller number of digits than the number of digits entered in the subtotal column. Therefore, the number of digits described in the subtotal column is extracted as the maximum number of digits, and the binarization processing of cells with digits exceeding the maximum number of digits is omitted in the binarization processing of cells in each row of the column. It is possible.

  Further, the binarization means may start the column of the written cells in the junction cell in the merged cell from the rightmost cell to the left or from the leftmost cell to the right depending on the heading. The binarization process may be skipped until it is found.

  In the form recognition apparatus according to another aspect of the present invention, the binarization means, depending on the heading, in each of the plurality of cells from the leftmost cell to the right or from the rightmost cell to the left. The binarization process is omitted until an end cell of a plurality of completed cells is found.

According to this aspect, the binarization process of the blank portion of one cell can be omitted, and the area to be binarized can be reduced.
In addition, the form recognition method according to one aspect of the present invention is described in a form including a table composed of cells that are separated by ruled lines and include or do not include characters or character strings. A form recognition method for recognizing characters, in the table, defining a table structure that determines the arrangement of the cells, extracting a headline that is a predetermined character or character string from the image, A position of a heading cell that is a cell including a heading is specified in the table structure, and among the cells of the table structure defined by the table structure defining means, any character or character string is entered in the cell. Detecting a completed cell, selecting only the completed cell detected by the completed cell detection means as a binarization target cell, and selecting the binary cell in the table. Of performs binarization of the target cell, characterized by comprising generating a binary image, a step recognizes characters that are described in the form of the binary image.

  In the form recognition method according to this aspect, a form can be recognized without using a pre-defined form, and further, binarization is performed by performing only a part of the table area on the form. The time required for processing can be reduced. As a result, the speed can be increased without degrading the recognition accuracy of the free forum form recognition.

  The form recognition method according to another aspect of the present invention provides a predetermined recognition method for a non-filled cell when a specific cell defined by the heading is not a filled cell among the cells of the table structure. The method includes a step of excluding cells in relative positions from the binarization target cell.

  The method for binarizing a form image according to another aspect of the present invention further includes the step of defining in the table structure a plurality of cells located at relative positions corresponding to the headings, The step of selecting a cell to be binarized recognizes the plurality of cells as the binarization cell.

  The form recognition method according to another aspect of the present invention further includes a step of detecting a plurality of filled cells in which cells in which characters are written are continuous among the plurality of cells, and the binarization target cell is selected. The step of detecting is detected by the filled-in multiple cell detecting means among the plurality of cells located at a relative position corresponding to the heading with respect to the position of the cell including the heading specified by the heading position specifying means. Only the plurality of cells are selected as binarization target cells.

  In the form recognition method according to another aspect of the present invention, when a cell located at a relative position corresponding to the heading is not a completed cell among the cells of the table structure defined by the table structure defining means, The method includes a step of excluding, from the binarization target cell, a cell at a predetermined relative position with respect to a cell that is not the completed cell.

  The form recognition method according to another aspect of the present invention further includes a step of detecting a filled cell in which any character or character string is written in the cell from at least some of the cells of the defined table structure. And defining a maximum number of the completed cells in the combined cell, wherein selecting the binarization target cell is further defined by the heading relative to a cell location including the heading. The maximum number of cells is selected as a binarization target cell from the plurality of cells included in the combined cell located at a relative position, depending on the heading.

  The form recognition method according to another aspect of the present invention further includes the step of defining a merged cell including a plurality of cells located at relative positions corresponding to the heading in the table structure, and In the selecting step, only a part of the merged cell located at a relative position corresponding to the heading with respect to the position of the cell including the heading specified by the heading position specifying means is binarized. It is characterized by selecting as a cell.

  The form recognition method according to another aspect of the present invention further includes a step of detecting a filled cell in which any character or character string is written in a cell from at least some of the cells of the table structure, Defining a maximum number of filled cells in a merged cell, and selecting the binarized cell further comprises relative to the position of the cell containing the heading relative to the heading. Selecting the maximum number of cells as binarization target cells from the plurality of cells included in the merged cell located at a position depending on the heading.

  The form recognition method according to another aspect of the present invention provides, depending on the heading, the left column from the rightmost cell or the right column from the leftmost cell in the merged cell, and the column of the filled cells in the merged cell. The method includes a step of skipping the binarization process until the beginning of is found.

  The form recognition method according to another aspect of the present invention provides, depending on the heading, the leftmost cell of the plurality of cells recognized by the binarization target cell selecting unit, or the leftward direction from the rightmost cell. The method includes a step of omitting the binarization processing until a cell at the end of the plurality of cells is found.

  Further, the program according to one aspect of the present invention includes a cell that is delimited by a ruled line and includes a character or a character string that is included in a cell including a table that includes or does not include a character or a character string. In a computer that can be used as a form recognition device for recognizing, in the table, a table structure defining function for defining a table structure for determining the arrangement of the cells, and a heading that is a character or a character string predetermined from the image Of the table structure cells defined by the table structure defining means, a headline extraction function for extracting, a heading position specifying function for specifying the position of the header cell that is a cell including the headings in the table structure, and A filled cell detection function for detecting a filled cell in which any character or character string is written in the cell, and the filled cell detected by the filled cell detection means. A binarization target cell selection function for selecting only as a binarization target cell, a binarization function for performing binarization processing of the binarization target cell in the table and generating a binary image, A character recognition function for recognizing characters described in a form from the binary image is realized.

With reference to FIGS. 1 to 5, the image data binarization processing method according to the present invention as described above and the form recognition apparatus 10 using the same will be described in general.
FIG. 1 is a block diagram of a form recognition apparatus 10 of the present invention.

  The form recognition apparatus 10 according to the present invention includes a scanner 100, a ruled line binarization processing unit 110, a table structure information extraction unit 120, a heading binarization processing unit 130, a heading / item extraction unit 140, a simple character recognition processing unit 142, A non-valuation target processing unit 150, a ladder frame grouping processing unit 152, an entry presence / absence determination unit 154, a partial binarization processing unit 160 and a character recognition processing unit 170 are included. These are electrically connected to each other. These processing units may be, for example, a processor that specializes each processing, or a combination of a general-purpose computer and a medium that includes a computer-recognizable instruction for performing each processing. When each processing unit is realized by a general-purpose computer, the function of each processing unit is realized by using a program including instructions for causing a computer to execute the function.

  In addition, the apparatus 10 includes an original image storage unit 200, a ruled line binary image storage unit 202, a table structure information storage unit 204, a headline binarization storage unit 206, a headline / item information storage unit 208, a headline dictionary unit 210, a binary A non-target range storage unit 212 and a threshold storage unit 214 are included. These may be memory such as RAM or ROM. The various processing units may be arranged in the same device, or may be prepared as separate devices.

In addition, these storage units can receive and store data output from the various processing units.
The original image storage unit 200 is communicably connected to the scanner 100 and stores an image captured by the scanner. The ruled line binary image storage unit 202 is communicably connected to the ruled line binarization processing unit 110, and the result of the ruled line binarization processing unit 110 is stored. The table structure information storage unit 204 and the headline binarization storage unit 206 are communicably connected to the table structure information extraction unit 120 and the headline binarization processing unit 130, respectively, and the results of the respective processing units are stored. Both the headline / item information storage unit 208 and the headline dictionary unit 210 are communicably connected to the headline / item extraction unit 140. The headline / item extraction unit 140 extracts a headline / item from the form image while referring to the headline dictionary unit 210, and the result is output to the headline / item information storage unit 208. The headline dictionary unit 210 stores character information of a preset headline. The non-binarization target range storage unit 212 is communicably connected to the non-binarization target processing unit 150 and stores a range in which the binarization processing selected by the non-binarization target processing unit 150 is not performed. The threshold value storage unit 214 is communicably connected to the entry presence / absence determination unit 154. The entry presence / absence determination unit 154 uses the threshold value stored in the threshold value storage unit 214 to determine whether or not there is a character entry in a cell. Determine whether.

  The entry presence / absence determination unit 154 may determine whether characters or the like are entered in the cell based on the number of black pixels in the cell or the distribution thereof. In this case, whether or not the pixel is a black pixel can be determined using the magnitude and / or orientation of a vector in the HSV color space composed of the three components of hue, saturation, and brightness. Alternatively, by detecting the number of connected components in a certain cell, it may be determined whether or not a character or the like is written in the cell.

The partial binarization processing unit 160 performs binarization processing on a region of the form image excluding the binarization target non-selected range selected by the binarization target non-processing unit 150.
The character recognition processing unit 170 recognizes characters in the image binarized by the partial binarization processing unit 160 by using known character recognition means such as OCR (Optical Character Recognition Recognition).

The table structure defining means includes a ruled line binarization processing unit 110, a table structure extracting unit 120, a ruled line binary image storage unit 202, and a table structure information storage unit 204.
The headline extraction means includes a headline binarization processing unit 130, a headline / item extraction unit 140, a simple character recognition processing unit 142, a headline binarization storage unit 206, a headline / item information storage unit 208, and a headline dictionary unit 210. . The heading position specifying means includes a table structure extraction unit 120, a ruled line binary image storage unit 202, a table structure information storage unit 204, a heading binarization processing unit 130, a heading / item extraction unit 140, a simple character recognition processing unit 142, a heading. A binarization storage unit 206, a headline / item information storage unit 208, and a headline dictionary unit 210 are included. The binarization target cell selection unit includes a binarization target non-processing unit 150 and a binarization target non-range storage unit 212. The binarization means includes a partial binarization processing unit 160. The entered cell detection means includes an entry presence / absence determination unit 154 and a threshold storage unit 214, and the multiple cell demarcation means includes a ruled line binarization processing unit 110, a table structure extraction unit 120, a ruled line binary image storage unit 202, and a table structure. An information storage unit 204 and a ladder frame grouping processing unit 152 are included. The character recognition means includes a character recognition processing unit 170. The maximum digit number detecting means includes an entry presence / absence determining unit 154. The combined cell delimiting means includes a ruled line binarization processing unit 110, a table structure extraction unit 120, and a ruled line binary image storage unit 202. Each of the above means may be defined by a program including instructions that can be recognized by a computer that implements the functions of the respective means. For example, the table structure defining means may define a computer-recognizable program or a part of a program that realizes a table structure defining function corresponding to the function of the means.

FIG. 2 is a flowchart showing a keyword recognition technique processing method applied to the apparatus 10 of the present invention.
In step S1000, a form image reading process is performed using the scanner 100. In the form recognition apparatus 10 according to the present invention, the form is preferably a color image. By using a color image, it becomes easy to separate characters and noise, or characters and background colors, and form recognition with high accuracy is possible. However, the form handled by the form recognition apparatus 10 according to the present invention may not be color. An image read using the scanner 100 is stored in the original image storage unit 200. When the processing of S1000 ends, the process proceeds to S1005.

  In S1005, the ruled line binarization processing unit 110 performs the binarization processing of the color image stored in the original image storage unit 200 in S1000, and outputs a binary image that leaves the ruled line. More specifically, in S1005, the input color image is binarized, and only a ruled line is left, and a binary image with all other background colors (for example, white) is output. The binary image that leaves the ruled line is output to the ruled line binary image storage unit 202. Since the binarization processing at this time is intended to extract ruled lines, the unit size of the binarization processing range composed of a plurality of pixels is large for recognizing characters, The processing speed may be minimized by making it sufficiently small for recognizing.

A binary image in which only ruled lines are left from a color image is generated by extracting linear components such as horizontal lines and vertical lines on a form. When the processing of S1005 ends, the process proceeds to S1010.
In S1010, the binary image with the ruled lines stored in the ruled line binary image storage unit 202 is referred to, and the table structure extracting unit 120 uses the information on the intersection of the horizontal and vertical lines to determine the cell and the final linear (table). Form), and information obtained by adding column number and row number information (coordinates) to each cell is output to the table structure information storage unit 204 as table structure information. The position of each cell is not represented by the two components of the column number and the row number. For example, from the cell in the upper left corner toward the cell in the lower right corner, the cell is numbered from left to right for the row and from top to bottom for the row. It may be attached and represented by its number.

  The processing of the table structure extraction unit 120 in S1010 will be described with reference to FIG. A point P1 represented by a large circle in FIG. 3 is a temporary end point of the table having ruled lines shown in FIG. Further, by detecting an intersection P3 where a vertical line and a horizontal line that are represented by small circles intersect, a quadrangle having the intersection P3 as a vertex is recognized as an individual cell. When the table structure extraction unit 120 defines a quadrangle having the point P1 as a vertex as a table shape, a protruding intersection appears as a point P2. Therefore, a rectangle including the point P2 as a vertex is also recognized as a table cell, and a final table shape is extracted.

  In S1010, the table structure extraction unit 120 may extract a “ladder frame” or a “combined cell” as described later. A ladder frame is a table frame that defines a series of cells in which each digit is separated by a dotted line to enter a number of digits. In the example of the form to be processed by the apparatus 10 shown in FIG. 4, the columns of the headings “Account No.”, “Amount”, and “Fees” are each 10 consecutive cells, 10 The cell is a ladder frame cell composed of five cells, and the ruled line defining the ladder frame cell is a ladder frame. When the process of S1010 ends, the process proceeds to S1015. The position of the ladder frame cell, the number of included cells, and the like may be stored in the table structure information storage unit 204 as ladder frame information.

  In S1015, using the table structure information extracted by the table structure extraction unit 120, a binary image is generated with the characters remaining for the top row of the table, and is output to the heading binarization storage unit 206. To do. More specifically, in the color image in which the ruled lines are deleted from the top row of the table using the table structure information, the binarization process is performed on the top row region of the table and the characters are left. A value image may be generated. S1015 is processed by the headline binarization processing unit 130. When the process of S1015 ends, the process proceeds to S1020.

  In the binarization processing by the heading binarization processing unit 130 in S1015, the unit size of the binarization processing range (generally including a plurality of pixels and the binarization threshold is defined in the region) is The size used in the binarization processing performed by the ruled line binarization processing unit 110 is preferably equal to or smaller than the size used. In this way, the binarization processing of the form is performed by changing the unit size of each binarization processing range in the binarization processing of the ruled line binarization processing unit 110 and the headline binarization processing unit 130. Speed can be improved.

  In S1020, character recognition is performed on the binary image in which the characters in the top row of the table created in S1015 are left, and the extracted character information is compared with information stored in the index dictionary unit 210. The character information regarded as the headline is output to the headline / item information storage unit 208.

  In S1025 subsequent to S1020, if there is no character in the binary dictionary image 210 in which the character in the top row of the table is left in the headline dictionary unit 210, the headline item is not recognized and the process proceeds to S1030. On the other hand, if the character in the binary image in which the character in the top row of the table is left is in the heading dictionary unit 210, the process proceeds to S1035. Steps S1020 and S1025 are processed by the headline / item extraction unit 140.

In the branch in the flowchart of FIG. 2, “Y” represents yes (affirmative) and “N” represents no (denied). The same applies to the flowcharts appearing in the drawings after FIG.
In S1030, the column of the table to which the non-binarization heading selected in S1025 belongs is excluded from the binarization target, and is output to the binarization non-target range storage unit 212. The processing of S1030 is processed by the binarization target non-processing unit 150. When the process of S1030 ends, the process proceeds to S1035.

  In S1035, the right attribute or the left attribute is assigned to the heading extracted in S1020 and S1025. This processing is performed by the headline / item extraction unit 140, and the result is output to the headline / item information storage unit 208. When the process of S1035 ends, the process proceeds to S1040.

This position attribute information may be stored in the headline dictionary unit 210 together with characters representing the headlines extracted in S1020 and S1025.
In S1040, the entry presence / absence determination unit 154 specifies a blank portion in which no character is entered for each heading column based on the position attribute information for each heading extracted in S1030. , And output to the binarized non-target range storage unit 212. When the process of S1040 ends, the process proceeds to S1045.

  In S1045, based on the table structure information extracted in S1010 by the entry presence / absence determining unit 154 and the preset threshold for each cell, threshold for each heading, and the like stored in the threshold storage unit 214, It is determined whether or not a character is entered in each cell of the table from the row.

  In S1050 subsequent to S1045, it is determined whether or not there is a line (detail line) that can be regarded as a blank in the table. This process is performed by the entry presence / absence determination unit 154. If there is a line that can be regarded as a blank, the process proceeds to S1055, and the line that can be regarded as a blank is output to the binarization-untargeted range storage unit 212 by the binarization-untargeted processing unit 150 as a binarization-untargeted range. The Thereafter, the process proceeds to S1060. If there is no line that can be regarded as a blank, the process directly proceeds to S1060.

  In S1060, a ladder frame in the table is extracted based on the ladder frame information extracted in S1010, and is output to the table structure information storage unit 204. When the process of S1060 ends, the process proceeds to S1065. The method of extracting the ladder frame may be performed by extracting dotted ruled lines that divide the area surrounded by the solid line in the table, in addition to a method using a histogram as described later.

In S1065, it is determined whether there is a ladder frame in the table with reference to the result of S1060. The processing of S1060 and S1065 is performed by the table structure information extraction unit 120.
If the result of determination in S1065 is “Yes”, the process proceeds to S1070. If the result is “None”, the process proceeds to S1095. In S1070 and S1095, the simple character recognition processing unit 142 performs character recognition for two rows from the bottom of the table using the table structure information stored in the table structure information extracting unit 120. The resulting character information is matched with the data stored in the heading dictionary unit 210 and the heading character information regarded as a subtotal column is output to the heading / item information storage unit 208. In S1070, since a ladder frame cell exists, when extracting the subtotal description column, the ladder frame cell may be preferentially searched as a candidate for the subtotal description column. When the processes of S1070 and S1095 are finished, the process proceeds to S1075 and S1100, respectively.

In S1075 and S1100, it is determined whether the subtotal column has been extracted.
If the subtotal field is extracted in S1075, the process proceeds to S1080, and if the subtotal field is not extracted, the process proceeds to S1085. If the subtotal field is extracted in S1100, the process proceeds to S1105. If the subtotal field is not extracted, the process proceeds to S1115.

In S1080, since the subtotal column is a ladder frame, the entry presence / absence determination unit 154 calculates the number of digits to be entered.
In S1105, since the subtotal column is not a ladder frame, the entry presence / absence determination unit 154 calculates coordinates indicating the entry range. Then, from the coordinates indicating the entry range, a subtotal entry length that is a length in which characters such as numbers are described in the subtotal entry field is calculated.

  In S1085 subsequent to S1080, a cell range in which, for example, a monetary amount other than the number of digits calculated in S1080 may be entered is set as a binarization non-target range, and is output to the binarization target non-target range storage unit 212. If the subtotal column is not a ladder frame, in S1110, for example, a cell range other than the subtotal entry length calculated in S1105 is entered as a non-binarization target range, and a binarization non-target range storage unit It outputs to 212.

When the processing of S1085 and S1110 ends, the process proceeds to S1090 and S1115, respectively.
In S1090, the ladder frame grouping processing unit 152 groups the binarization non-target range specified in S1085, and outputs the grouping result to the binarization non-target range storage unit 212. Grouped cells may also be referred to as “multiple cells”.

  In step S1115, based on the table structure information extracted in step S1010 and the heading information extracted in step S1020, the top cell of the recipient item is checked for a ruled line that is divided vertically. In this example, the recipient item is described as an example. However, it is also possible to target a cell in which ruled lines that are divided vertically are used in the cell in order to shake the ruby, such as a combination of name and furigana.

In S1120 subsequent to S1115, it is determined whether or not the recipient item has a ruled line. If there is a ruled line, the process proceeds to S1125, and if there is no ruled line, the process proceeds to S1135.
In S1125 and S1135, the simple character recognition processing unit 142 checks whether there is a kana item above or below the ruled line of the recipient item. As the kana item, in addition to hiragana and katakana, characters used in foreign languages such as alphabet may be used.

If it is determined in step S1125 that a kana item exists, the process advances to step S1145. If the result does not exist, the process advances to step S1130.
If it is determined in step S1135 that a kana item exists, the process advances to step S1150. If the result does not exist, the process advances to step S1140.

In S <b> 1130 and S <b> 1140, the binarization target processing unit 150 regards a column other than the kana item of the recipient item as a binarization non-target range and outputs it to the binarization target non-target range storage unit 212.
In S1130 and S1140, a column other than the Kana item of the recipient item is regarded as a non-binarization target range. However, if the recipient item has only a Kanji entry field, a column other than the Kana item of the recipient item is selected. You may leave it as the object of a binarization process, without setting it as the range which is not a binarization object.

When the processing of S1130 and S1140 ends, the process proceeds to S1150.
On the other hand, in S1145, since there is a Kana item in the recipient item, based on the table structure information extracted in S1010, the position of a cell in which an item that is not a Kana item is entered is specified. After being regarded as a binarization non-target range by the binarization non-target processing unit 150, it is output to the binarization non-target range storage unit 212. When the process of S1145 ends, the process proceeds to S1150.

  In S1150, it is determined by referring to the binarization non-target range storage unit 212 whether there is a binarization non-target range. If there is a non-binarization target range, the process proceeds to S1155, and if not, the process proceeds to S1160.

  In S <b> 1155, the binarization non-target processing unit 150 reads information related to the binarization non-target range from the binarization target non-target storage unit 212, and defines the final binarization target range. Thereafter, the process proceeds to S1160.

In S1160, the binarization processing unit 160 performs binarization processing in units of cells on the cells in the final binarization target range defined in S1155. When the process ends, the process proceeds to S1165. In the binarization processing in S1160, the unit size of each binarization processing range may be equal to or smaller than the size used in the binarization processing performed by the ruled line binarization processing unit 110. desirable. By doing so, it is possible to improve the speed of the binarization process without sacrificing the accuracy at the time of character recognition of the generated binary image.
In S1165, the character recognition processing unit 170 performs character recognition on the binarization target range of the table on the form.

  FIG. 4 is a diagram showing an example of a form to be processed by the apparatus of the present invention. In the case of FIG. 4, the table included in the form has a structure of 22 rows and 32 columns. Here, the column of the “Account Number” heading is counted as 10, the column of the “Money” heading is counted as 10, and the column of the “Fees” heading is counted as 5. In the case of FIG. 4, 22 rows × 32 columns = 704 cells are included.

FIG. 5 shows cells, ladder format cells (ladder frame cells), headings, and recognition target character data in the form shown in FIG.
A region delimited by a ruled line that is a straight line such as a solid line or a broken line is a cell. A cell is also a unit element constituting a table. The ladder format cell includes a plurality of cells divided by dotted ruled lines or the like. This ladder type cell is also referred to as a plurality of cells. In the example of FIG. 5, the entry columns for the account number, amount, and fee are ladder-type cells.

  In FIG. 5, “No.”, “Financial institution name”, “Branch name”, “Deposit line item”, “Account number”, “Recipient”, “Amount”, “Transfer category” in the first row of the table. , “Fee” and “Summary” are headings. “Client” is also included in the heading. Character data to be recognized that is symmetrical to character recognition is a character or a character string that is entered in each row of the heading column in the table.

  By adopting the configuration as described above, the binarization process has been speeded up without degrading the character recognition accuracy in a device that recognizes an unspecified number of types of forms without using a predefined form. A form recognition device can be obtained.

(Example 1)
Hereinafter, with reference to some examples, the image data binarization processing method according to the present invention and the form recognition apparatus using the method will be described in more detail. Hereinafter, elements having the same or similar functions, or steps for performing the same or similar processing are given the same reference numerals as those of the apparatus 10 described above, and detailed description thereof is omitted.

This example relates to a method for reducing the range to be partially binarized by setting cells related to items that are not to be recognized from the heading information as binarization-free ranges, and an apparatus using the method.
Hereinafter, this example will be described with reference to FIGS. FIG. 6 is a block diagram of the apparatus of Example 1 of the present invention. FIG. 7 is a flowchart showing the processing method of the keyword recognition technique applied to the apparatus 20 of this example of the present invention. FIG. 8 is an example of a binary image with ruled lines left. FIG. 9 is a diagram showing an example of a form to be processed by the apparatus of Example 1 of the present invention.

  As shown in FIG. 6, the form recognition apparatus 20 according to this example includes a scanner 100, a ruled line binarization processing unit 110, a table structure information extraction unit 120, a heading binarization processing unit 130, a heading / item extraction unit. 140, and a binarized non-processing unit 150. These are electrically connected to each other.

  In addition, the apparatus 20 includes an original image storage unit 200, a ruled line binary image storage unit 202, a table structure information storage unit 204, a headline binarization storage unit 206, a headline / item information storage unit 208, a headline dictionary unit 210, and a binary dictionary. A non-valuation target range storage unit 212 is included. These storage units are connected to the corresponding processing units in the same manner as the apparatus 10 described above.

FIG. 7 is a flowchart showing processing executed by the apparatus 20.
In the apparatus 20 of this example, the binarized area is reduced by excluding items other than the recognition target items from the binarization target in the table on the form as shown in FIG.

After processing steps S1000 to S1015 executed by the apparatus 10, in this example, simple character recognition is performed in S2000.
An example of a binary image leaving the ruled line obtained in S1005 is shown in FIG. FIG. 8 is a part of a binary image in which the ruled lines of the table on the form of FIG. 4 are left.

In S2000, the headline / item extraction unit 140 is used to perform character recognition on the binary image in which the characters in the top row of the table created in S1015 are left, and the process proceeds to the next S2005.
In S2005, the extracted character information is matched with information stored in the heading dictionary unit 210, and it is determined whether the extracted character information is a heading item.

  If the character information is a heading item, the process proceeds to S2020. In S2020, the position of the heading, for example, the column number and the row number is defined by comparing with the table structure information, and the character information regarded as the heading item is output to the heading / item information storage unit 208 together with the column number and the line number information. To do.

If there is no character information in the heading item, it is output to the heading / item information storage unit 208 together with the column number and line number information as a heading item that is not a recognition target, and the process proceeds to S2025.
In S2025, it is determined whether there is a heading item that is not a recognition target, and if not, the process of reducing the binarized area ends.

  If there is a heading item that is not the recognition target, in S2030, the entire column including the heading item that is not the recognition target specified in S2025 is searched while referring to the table structure information stored in the table structure information storage unit 204. A non-recognized flag is added in units of columns as a non-valued range.

FIG. 9 shows an example of a column to which the non-recognition flag 1 is assigned and is set as a binarization non-target range.
Finally, in S2010 after S2030, the binarization target non-range storage unit 212 binarizes the column of heading items that are not recognized based on the flag set in S2030 and excludes the column from the binarization target. The coordinates are output to the non-target range storage unit 212, and the process of reducing the binarized area ends.

  After performing the process of reducing the binarized area in this example, the forms can be recognized by performing S1150 to S1165 in FIG.

The effect of the apparatus 20 of this example will be described using the form shown in FIG. 4 as an example.
If binarization processing is performed on all cells in the table shown in FIG. 4 without performing binarization processing on the binarized non-target range as shown in this example, the number of processing is Number 22 × number of columns 32 = 704 times. On the other hand, the number of binarization processes by using the apparatus 30 of this example can be reduced to 22 rows × 3 columns = 66 times.

(Example 2)
Example 2 relates to a method for reducing the range of partial binarization by assigning a position attribute to the description position of data of each item, specifying a blank part for each cell, and excluding it, and an apparatus using the same .

  This example will be described with reference to FIGS. FIG. 10 is a block diagram of the apparatus 30 of Example 2 of the present invention. FIG. 11 is a flowchart showing the processing method of the keyword recognition technique applied to the apparatus of Example 2 of the present invention. FIG. 12 is a diagram showing a part of an example of a form to be processed by the apparatus of Example 2 of the present invention. FIG. 13 is a diagram showing a non-binarization target range in the example of the form shown in FIG.

  As shown in FIG. 10, the form recognition apparatus 30 according to this example includes a table structure information extraction unit 120, a heading / item extraction unit 140, a binarization target non-processing unit 150, and an entry presence / absence determination unit 154. It is out. These are electrically connected to each other.

  The apparatus 30 also includes an original image storage unit 200, a table structure information storage unit 204, a headline / item information storage unit 208, a headline dictionary unit 210, a binarized non-target range storage unit 212, and a threshold storage unit 214. Yes.

  In the apparatus 30, the headline / item information storage unit 208 and the headline dictionary unit 210 are in the headline / item extraction unit 140, the table structure information storage unit 204 is in the table structure information extraction unit 120, the threshold storage unit 214 and the original image storage unit 200. Is connected to the entry presence / absence determination unit 154.

FIG. 11 is a flowchart showing processing executed by the device 30.
In the apparatus 30 of this example, position attribute assignment is performed on the heading item to determine whether the data description position is left-justified or right-justified, and a check is performed on a cell in which a blank part exists for each item column, and the specified blank part is partially The binarization processing area is reduced as the binarization out-of-range.

  In the process shown in FIG. 11, in S3000, as in S2000 of Example 1, character recognition is performed on a binary image in which the characters in the top row of the table are left, and the extracted character information is found in a dictionary. Matching the information stored in the unit 210, the character information regarded as the headline is output to the headline / item information storage unit 208.

  Next, in S1025, if there is no character in the binary dictionary image 210 in which the character in the top row of the table is left in the index dictionary unit 210, the process of reducing the range to be partially binarized is terminated. On the other hand, if the character in the binary image in which the character in the top row of the table is left is in the heading dictionary unit 210, the process proceeds to S1035.

  In S1035, the heading information as the recognition target stored in the heading / item information storage unit 208 is referred to, and the position attribute flag is added to the item to which the unrecognized flag is not added. As an example of the flag, 0 is left justified and 1 is right justified. The processing of S1035 is performed by the headline / item extraction unit 140.

  FIG. 12 shows an example of adding a position attribute flag. A position attribute flag 0 is added to the heading “financial institution name”, and “AAA bank”, “BBB bank”, etc., which are recognition target character data, are left-justified in the cell. On the other hand, the position attribute flag 1 is added to the heading “account number”, and the account number is written right-justified.

In step S3005 subsequent to step S1035, a blank portion is checked.
The process of S3005 will be described with reference to FIG.
With reference to the color image of the form read by the scanner in the original image storage unit 200, the entry presence / absence determination unit 154 generates a histogram for each column of the table on the form based on the flag set as described above. The black pixel used is determined for each dot, and a blank portion is specified. Here, the histogram indicates the black pixel region distribution in the combined cell (ladder frame cell) as shown in FIG. In the histogram in FIG. 13, the horizontal axis represents the distance from the left end in the cell, and the vertical axis represents the number of black pixels. The histogram may be a distribution obtained as a result of determining whether each unit area to be binarized is a black pixel or not from a vector in the HSV space of the color of the area. For example, a unit area having a lightness of less than 20% may be determined as a black pixel. When a histogram is created with ladder frame cells, a sharp peak of the number of black pixels appears on a ruled line that distinguishes digits. Using this peak information, ladder frame cells may be extracted or confirmed. Moreover, this peak is recognized as a blank part.

Then, the specified blank portion is output to the binarized non-target range storage unit 212.
In the heading “financial institution name”, the position attribute flag “0” in the column indicating that the chuck processing is performed from the right side of the cell to determine whether or not there is a black pixel for specifying a blank portion is included in the heading “account number”. ”Is added with a column position attribute flag“ 1 ”indicating that the chuck processing for determining whether or not there is a black pixel is performed from the left of the cell in order to specify a blank portion.

  In S2010 subsequent to S3005, the binarization target non-range storage unit 212 sets the coordinates of the blank part of the column of the heading item that is not the recognition target to the binarization target based on the flag set in S1030. The coordinates are output to the non-target range storage unit 212.

  After performing the process of reducing the binarized area in this example, the forms can be recognized by performing S1150 to S1165 in FIG.

The effect of the apparatus 30 of this example will be described using the form shown in FIG. 4 as an example.
By using the apparatus 30 of this example, the binarization processing area is 19%, which is 4% in the column of “No.”, 8% in the “Transfer” column, and 7% in the “Summary” column. Can be reduced.

(Example 3)
This example relates to an apparatus for reducing the range and the number of times of partial binarization by checking the presence or absence of entry of cell information in a table.

  This example will be described with reference to FIGS. FIG. 14 is a block diagram of the apparatus 40 of Example 3 of the present invention. FIG. 15 is a flowchart showing a keyword recognition technique processing method applied to the apparatus of Example 3 of the present invention. FIG. 16 is a diagram showing a part of an example of a form to be processed by the apparatus of Example 3 of the present invention.

As shown in FIG. 14, the form recognition apparatus 40 according to the present example includes a table structure information extraction unit 120, a binarization non-target processing unit 150, and an entry presence / absence determination unit 154. These are electrically connected to each other.
The apparatus 40 includes an original image storage unit 200, a table structure information storage unit 204, a binarized non-target range storage unit 212, a threshold storage unit 214, and a histogram storage unit 216. The table structure information storage unit 204 is in the table structure information extraction unit 120, the binarization non-target range storage unit 212 is in the binarization target non-processing unit 150, the threshold storage unit 214, the histogram storage unit 216, and the original image storage unit. 200 is connected to the entry presence / absence determination unit 154.

FIG. 15 is a flowchart showing processing executed by the device 40.
In the apparatus 40 of this example, the presence or absence of entry of cell information in the table is checked in units of rows, and rows having a certain number or more of unfilled cells are excluded from the binarization target, thereby reducing the binarization processing area.

  In the processing of this example, the table structure information related to the digitized image of the form to be processed is referred to from the table structure information storage unit 204, and from the bottom line of the table included in the form to the top line. , Check for entry.

  First, in S4000, the color image of the form is referred to from the original image storage unit 200, and the presence / absence determination unit 154 checks the number of black pixels to determine the presence / absence of entry for each cell. Add a flag. For example, the entry flag “1” may indicate that there is an entry and the entry flag “0” may indicate that the entry is unclear.

FIG. 16 shows an example of a table cell to which an entry flag is added. Shown below each cell in FIG. 16 is a histogram of the number of black pixels in the cell.
At this time, a cell column in which black pixels having a certain threshold value or more are periodically found in the histogram is output to the histogram storage unit 216 as a histogram of ladder frame cells. The threshold value is stored in advance in the threshold value storage unit 214.

  In addition, the ladder frame cell refers to the table structure information, and the length of the cell in which the heading character is written and the location corresponding to the cell in which the heading character is written (for example, the heading is written). It is also possible to extract by extracting the candidates or candidates by comparing the lengths of the columns corresponding to the cells. In general, the ruled lines separating the digits of the ladder frame cells are often dotted lines or broken lines. Therefore, ladder frame cells may be extracted or candidates may be extracted by extracting ruled lines that are not solid lines such as dotted lines or broken lines on a form.

  In S4005 subsequent to S4000, it is determined whether all the cells belonging to the row are unfilled or unclear. If all the cells belonging to the row are unfilled or ambiguous, set the unfilled flag “1”, which is a flag indicating whether all the cells belonging to the row are unfilled or ambiguous. , The process proceeds to S4010, the line to be processed is moved to one line, and the process returns to S4000.

  In S4005, when a certain number or more of cells having the entry flag “1” and the entry flag “0” are found in the row, there are cells in which characters are entered in the row. At this time, the process proceeds to S4015, and the line where the unfilled flag “1” is set by the binarization non-target processing unit 150 is set in the binarization non-target range storage unit 212 as coordinates that are not binarized. The process of outputting and reducing the range and number of times of binarization is completed.

  In this example, after performing the process of reducing the binarization range and the number of times, the forms can be recognized by performing S1150 to S1165 in FIG.

The effect of the apparatus 40 of this example will be described using the form shown in FIG. 4 as an example.
The number of binarization processes by using the apparatus 40 of this example can reduce the binarization process for the cells in the 13th to 20th rows of the table on the form in FIG. Therefore, binarization processing is not performed on 8 lines out of 22 lines including the line including the top heading of the table and the line including the subtotal column at the bottom of the table. Therefore, the number of binarization processes can be reduced by about 34%.

(Example 4)
In this example, if the cell is a ladder frame, such as a cell related to an amount, a fee, etc., the cells in the ladder frame are grouped together as a ladder frame cell, and the bottom row of the table enters the ladder frame cell. The present invention relates to an apparatus that obtains the maximum number of digits to be entered and reduces the range of partial binarization in ladder frame cells. Here, the cells generated by grouping may be referred to as a plurality of cells.

  This example will be described with reference to FIGS. FIG. 17 is a block diagram of the apparatus 50, and FIG. 18 is a flowchart showing a keyword recognition technique processing method applied to the apparatus 50. FIG. 19 is a diagram for explaining the extraction of a ladder frame in the processing method of the keyword recognition technique applied to the apparatus 50. FIG. 20 is a diagram illustrating a non-binarization target range of an example of a form to be processed by the apparatus 50. FIG. 21 is a diagram for explaining ladder frame grouping in the processing method of the keyword recognition technique applied to the apparatus 50.

  As shown in FIG. 17, the form recognition device 50 according to this example includes a table structure information extraction unit 120, a heading / item extraction unit 140, a simple character recognition processing unit 142, a non-binarization target processing unit 150, a ladder. A frame grouping processing unit 152 and an entry presence / absence determination unit 154 are included.

  The device 50 also includes an original image storage unit 200, a table structure information storage unit 204, a heading dictionary unit 210, a binarized non-target range storage unit 212, a threshold storage unit 214, and a histogram storage unit 216. The table structure information storage unit 204 is connected to the table structure information extraction unit 120, the heading dictionary unit 210 is connected to the heading / item extraction unit 140, and the binarization non-target range storage unit 212 is connected to the binarization target non-processing unit 150. Yes. The original image storage unit 200, the threshold storage unit 214, and the histogram storage unit 216 are connected to the entry presence / absence determination unit 154.

FIG. 18 is a flowchart showing processing executed by the device 50.
In this process, first, based on the table structure information, the subtotal column located at the bottom of the table corresponding to a certain heading is extracted, and the number of digits entered in the subtotal column that is a ladder frame cell is the maximum number of digits. Extract as a number. Then, since the number of digits of the number described in each row of the column of the heading is smaller than the number of digits entered in the subtotal column, the digits exceeding the maximum number of digits in each row are blank. By not performing binarization processing on the blank portion, the number of binarization processing and the binarization processing area can be reduced.

  In S1060, the ladder frame in the table is extracted and output to the table structure information storage unit 204. In the next step S1065, the table structure information in the table structure information storage unit 204 is referred to, and the entry presence / absence determination unit 154 checks the cells in the table (details) using a histogram. For example, as described in the previous example 3 In such a method, ladder frame information is extracted, and a ladder flag “1” is assigned to a cell regarded as a ladder frame. FIG. 19 shows an example in which a ladder flag is assigned to a cell. As shown in FIG. 19, the ladder frame has a feature that, for example, a peak of a histogram by a broken line separating digits in the case of numbers appears spatially and periodically. The threshold value used for specifying the ladder frame is stored in the threshold value storage unit 214 in advance.

  If it is determined in step S1065 that there is a ladder frame, the process advances to step S1070, and the simple character recognition processing unit 142 performs character recognition for the two lines from the bottom line in the form from the information in the table structure information storage unit 204. I do. In next step S1070, the extracted character information is compared with the heading dictionary unit 210, and the heading information regarded as a subtotal column is output to the heading / item information storage unit 208 (not shown). In most cases, since the subtotal column is located at the bottom of the table on the form and is often described together with the total column, it is desirable to perform character recognition for two lines from the bottom of the table.

If it is determined in S1065 that there is no ladder frame, the process of reducing the range for partial binarization in the ladder frame cell is terminated.
In S5000, it is determined whether or not the character information regarded as the subtotal column is in the heading. If it is determined as a result of the determination in S5000, the process proceeds to S1080, and in s1080, the cells existing in the subtotal heading column are checked by a histogram. When the subtotal entry field is a ladder frame, as in the amount field and the fee field in FIG. Calculate the number of digits.

  In step S1155 following step S1080, the binarization target non-binarization processing unit 150 sets the ladder cells other than the calculated number of digits in the amount and fee column ladders in each detail to be binarized and binarized. The ladder flag is deleted from the table structure information of such a cell, which is output to the non-target range storage unit 212.

  In the next S5005, as shown in FIG. 21, the ladder frame grouping processing unit 152 groups adjacent cells among the cells to which the ladder flag of the entire form is added, and the grouping result is displayed in the next S5010. The data is output to the binarized non-target range storage unit 212.

In this example, after carrying out the process of reducing the binarization range and the number of times, the forms can be recognized by performing S1150 to S1165 in FIG.
The effect of the apparatus 50 of this example will be described using the form shown in FIG. 20 as an example.

By using the apparatus 50 of this example, the binarization processing area can be reduced by 4 digits in the “Amount” column and 1 digit in the fee column. This area is about 10% of the entire table.
As described above, in this example, when the subtotal field and the amount field are ladder frames, by specifying the number of digits in the subtotal, the amount of each headline and the maximum number of fees can be specified. It can be narrowed. Therefore, it is possible to speed up the binarization process in the form recognition apparatus.

(Example 5)
In this example, if the cell is not a ladder frame, such as a cell related to the amount, fee, etc., the number of characters in the subtotal entry string (length of the entry string) is referred to, and the blank part in the cell is specified. The present invention relates to a device 60 for reducing the range for binarization.

This example will be described with reference to FIGS.
As shown in FIG. 22, the form recognition device 60 according to this example includes a table structure information extraction unit 120, a heading / item extraction unit 140, a simple character recognition processing unit 142, a non-binarization target processing unit 150, and an entry. A presence / absence determination unit 154 is included.

  The device 60 includes an original image storage unit 200, a table structure information storage unit 204, a heading dictionary unit 210, and a threshold storage unit 214. The table structure information storage unit 204 is connected to the table structure information extraction unit 120, and the heading dictionary unit 210 is connected to the heading / item extraction unit 140. The original image storage unit 200 and the threshold value storage unit 214 are connected to the entry presence / absence determination unit 154.

FIG. 23 is a flowchart showing processing executed by the device 60 of this example.
In the apparatus 60 of this example, in the case of one cell in which the subtotal column that was a ladder frame cell in Example 4 is not a ladder frame, the number of digits of the number entered in the subtotal column is extracted as the maximum character string length. . Then, since the number of digits of the number described in each row of the column of the heading is shorter than the number entered in the subtotal column, the digits exceeding the maximum character string length of each row are blank. By not performing binarization processing on the blank portion, the number of binarization processing and the binarization processing area can be reduced.

  After performing the processing of S1070 and S5000, in S6000, as shown in FIG. 24, a color image is referred to from the original image storage unit 200, and the number of black pixels in the ladder frame is displayed as a histogram by the entry presence / absence determination unit 154. Use this to check and calculate the subtotal string length.

  When the process of S6000 ends, the process proceeds to S1155 for processing. In other words, the binarized non-processing unit 150 sets the area in the cell that exceeds the character string length of the subtotal calculated in S6000 as the binarized non-target range.

  In this example, after performing the process of reducing the binarization range and the number of times, the forms can be recognized by performing S1150 to S1165 in FIG.

The effect of the apparatus 60 of this example will be described using the form shown in FIG. 24 as an example.
By using the apparatus 50 of this example, the binarization processing area can be reduced by 4 digits in the “Amount” column and 1 digit in the fee column. This area is 10% of the entire table.
Note that the processing described in Example 3 may be performed, and as a result, the above processing may not be performed on the cells to which the unfilled flag is added.

(Example 6)
In this example, if there is a kana part and kanji item in the recipient's heading information, the range of partial binarization is reduced by setting the kanji item on the data cell of that item as a non-binarized range It relates to the apparatus 70 which performs.

This example will be described with reference to FIGS.
As shown in FIG. 25, the form recognition apparatus 70 according to the present example includes a table structure information extraction unit 120, a heading / item extraction unit 140, a simple character recognition processing unit 142, a binarization non-processing unit 150, an entry. A presence / absence determination unit 154 is included.
The apparatus 70 also includes an original image storage unit 200, a table structure information storage unit 204, a heading dictionary unit 210, a binarized non-target range storage unit 212, and a threshold storage unit 214. The table structure information storage unit 204 is connected to the table structure information extraction unit 120, and the heading dictionary unit 210 is connected to the heading / item extraction unit 140. The original image storage unit 200 and the threshold value storage unit 214 are connected to the entry presence / absence determination unit 154. The binarization non-target range storage unit 212 is connected to the binarization non-target processing unit 150.

FIG. 26 is a flowchart showing processing executed by the apparatus 70 of this example.
In S7000, the heading information is referenced from the heading / item information storage unit 208 (not shown in FIG. 26), and the coordinate positions of the payee and account number heading on the table structure are specified.

In next step S7005, it is determined whether or not there is a recipient header, and if there is, the flow proceeds to S7010. If there is no recipient headline, the process of reducing the range for partial binarization is terminated.
In S7010, it is determined whether or not there is an account number header, and if there is, the process proceeds to S7015. If there is no account number header, the process of reducing the range for partial binarization is terminated.

  In S7015, the table structure information is referenced from the table structure information storage unit 204, the cell immediately below the account number item is referenced, and the height for one item (one item) is calculated. In next step S7020, the height of one item calculated in step S7015 is referred to specify a cell below the headline recipient item, and as shown in FIG. A histogram is used to check whether there are ruled lines that divide in the top and bottom. The threshold used for the ruled line check is stored in advance in the threshold storage unit 214. Thereafter, the process proceeds to S7025.

  In S7025, it is determined whether there is a horizontal ruled line in the recipient item cell. If there is, the process proceeds to S70330. In step S7030, the color image is referred to from the original image storage unit 200, and the category check is performed by the simple character recognition processing unit 150 on the upper and lower recipient cells to identify the kana item position. If there is no ruled line, the determination process of S7025 is performed over the entire recipient cell.

  In next step S1125, the simple character recognition processing unit 142 checks whether there is a kana item in the recipient item. If it is determined in S1125 that there is a kana item, the process proceeds to S7035.

If it is determined in S1125 that there is no kana item, the process proceeds to S1130.
In S7035, a Kana item flag is assigned to the upper and lower cells of each recipient item string as shown in FIG.

  In S1155 after S7035, the binarization target non-binary range storage unit 212 sets the range where the kana item flag in the recipient item string is not the kana item as the binarization target non-processing range 150. Output to.

  In this example, after performing the process of reducing the binarization range and the number of times, the forms can be recognized by performing S1150 to S1165 in FIG.

The effect of the apparatus 70 of this example will be described using the form shown in FIG. 28 as an example.
The binarization processing area by using the apparatus 50 of this example is the binarization of the “recipient” column, which usually includes kanji, out of the “(phonetic)” and “recipient” columns in FIG. The process is omitted. Therefore, it is possible to reduce the binarization processing area of 7%, which is half of the area 13% in the entire column of “(phonetic)” and “recipient”.

Although several examples of the present invention have been described above, of course, the above examples may be combined.
Regarding the number of binarization processes, for example, by combining the techniques of Example 1 and Example 3, it is possible to reduce the total of 308 times, 66 times in Example 1 and 242 times in Example 3. Regarding the binarized area, for example, by combining the techniques of Example 2, Example 4, and Example 6, the area of 19% + 10% + 7% = 36% can be reduced.

Claims (9)

A form recognition device for recognizing characters described in a form including a table composed of cells that are separated by a ruled line and include or do not include characters or character strings,
In the table, a table structure defining means for defining a table structure that defines the arrangement of the cells;
Headline extracting means for extracting a headline that is a predetermined character or character string from the image;
Heading position specifying means for specifying the position of a heading cell that is a cell including the heading in the table structure;
Of the cells of the table structure defined by the table structure defining means, a filled cell detection means for detecting a filled cell in which some character or character string is entered in the cell;
Binarization target cell selection means for selecting only the filled cells detected by the filled cell detection means as binarization target cells;
Binarization means for performing binarization processing of the binarization target cell in the table and generating a binary image;
Character recognition means for recognizing characters described in a form from the binary image;
A form recognition device comprising:   The binarization target cell selection means, when a cell located at a relative position corresponding to the heading among the cells of the table structure defined by the table structure defining means is not a completed cell, The form recognition apparatus according to claim 1, wherein a cell at a predetermined relative position with respect to a cell that is not a completed cell is excluded from the binarization target cell. A plurality of cells defining means for defining, in the table structure, a plurality of cells located at relative positions corresponding to the headings;
Including
The form recognition apparatus according to claim 1, wherein the binarization target cell selection unit recognizes the plurality of cells as the binarization target cells. Filled multiple cell detection means for detecting a plurality of filled cells in which cells in which characters are written among the plurality of cells are continuous,
Including
The binarization target cell selecting means is a plurality of completed cells among the plurality of cells located at a relative position corresponding to the heading with respect to the position of the cell including the heading specified by the heading position specifying means. 4. The form recognition apparatus according to claim 3, wherein only the plurality of cells detected by the detecting means are selected as binarization target cells. A merged cell defining means for defining in the table structure a merged cell including a plurality of cells located at relative positions corresponding to the headings;
Including
The binarization target cell selecting means selects only a part of the merged cell located at a relative position corresponding to the heading with respect to the position of the cell including the heading specified by the heading position specifying means, The form recognition apparatus according to claim 1, wherein the form recognition device is selected as the binarization target cell. Filled cell detection means for detecting a filled cell in which any character or character string is written in the cell from at least some of the cells of the table structure defined by the table structure defining means;
A maximum cell number defining means for defining a maximum number of the filled cells in the combined cells;
Including
The binarization target cell selecting unit further selects the maximum number of cells from the plurality of cells included in the merged cell located at a relative position corresponding to the heading with respect to the position of the cell including the heading. The form recognition apparatus according to claim 1, wherein the form recognition cell is selected as a binarization target cell depending on the heading. The image of the form is a color image,
The filled-in cell detection means determines whether or not the cell has been filled using a vector in the HSV color space composed of three components of hue, saturation and brightness of the color of the cell. The form recognition apparatus according to any one of claims 1 to 6. A form recognition method for recognizing characters described in a form including a table composed of cells that are separated by a ruled line and include or do not include characters or character strings,
In the table, defining a table structure defining an arrangement of the cells;
Extracting a heading that is a predetermined character or character string from the image;
Identifying in the table structure the position of the header cell that is the cell containing the header;
Detecting a filled cell in which any character or character string is written in the cell among the cells of the table structure defined by the table structure defining means;
Selecting only the completed cells detected by the completed cell detection means as binarization target cells;
Performing a binarization process on the binarization target cell in the table to generate a binary image;
Recognizing characters described in a form from the binary image;
The form recognition method characterized by comprising. It can be used as a form recognition device for recognizing characters written in a form including a table composed of cells that are separated by a ruled line and include or do not include characters or character strings. On the computer,
In the table, a table structure defining function for defining a table structure that defines the arrangement of the cells;
A headline extraction function for extracting a headline that is a predetermined character or character string from the image;
A heading position specifying function for specifying, in the table structure, a position of a heading cell that is a cell including the heading;
A filled cell detection function for detecting a filled cell in which some character or character string is written in the cell among the cells of the table structure defined by the table structure defining means;
A binarization target cell selection function for selecting only the filled cells detected by the filled cell detection means as binarization target cells;
A binarization function for performing binarization processing of the binarization target cell in the table and generating a binary image;
A character recognition function for recognizing characters described in a form from the binary image;
A program characterized by realizing.