JP3573945B2 - Format recognition device and character reading device - Google Patents

Description

【０２５５】
【表２】

【０２５６】
【表３】

【０２５７】
【発明の効果】
上述した説明から明らかなように、この出願の第１の発明のフォーマット認識装置によれば、帳票全体から抽出した特徴を既存の帳票の特徴と照合して帳票のフォーマットを認識でき、また、帳票の一部の特徴を既存の帳票の一部の特徴と照合して帳票のフォーマットを認識できる。なお、第１のフォーマット認識系を、帳票全体の罫線パタンについての特徴を抽出する認識系とするのが良い。かつ、既存の帳票の全体の特徴データとして既存の帳票の罫線パタンによる特徴データを用いる認識系とするのが良い。なぜなら、一般に、帳票は罫線を含む。さらに、罫線は帳票の構造を特徴づける顕著なパラメータである。従って、罫線パタンに着目して第１段階のフォーマット認識を行うと、かなりの精度で帳票のフォーマットを認識できるからである。
【０２５８】
また、この出願の第２の発明のフォーマット認識装置によれば、帳票の一部の特徴を既存の帳票の一部の特徴と照合して帳票のフォーマットを認識する。
【０２５９】
したがって、これら第１および第２の発明のフォーマット認識装置によれば、パタンマッチングのみで帳票のフォーマットを認識できる。そのため、帳票種別番号のない帳票であっても、そのフォーマットを、文字切り出しの処理を含むことなく、認識することができる。
【０２６０】
また、文字読み取り装置の各発明では、上記のフォーマット認識装置と、このフォーマット認識装置の認識結果に応じた読み取り制御データを出力する読み取り制御データメモリと、この読み取り制御データに基づいて文字を認識する文字認識系とを具える。そのため、フォーマット認識装置で認識されたフォーマットに基づいて認識対象の文字を認識することができる。
【図面の簡単な説明】
【図１】第１の発明で用いる帳票の第１の例である。
【図２】第１の発明で用いる帳票の第２の例である。
【図３】第１の発明の第１の実施の形態の文字読み取り装置の説明図である。
【図４】第１の発明の第１の実施の形態の個別フォーマット認識部の一構成例を説明する図である。
【図５】第１の発明の文字読み取り装置の動作説明図である。
【図６】罫線抽出部の動作例を説明する図である。
【図７】罫線パタンの抽出例を説明する図である。
【図８】個別フォーマット認識での特徴抽出の説明図である。
【図９】第１の発明の第２の実施の形態の個別フォーマット認識部の一構成例を説明する図である。
【図１０】第１発明の第２の実施の形態の説明図であり、抽出される特徴の説明図である。
【図１１】第１の発明で用いる帳票の第３の例である。
【図１２】第１の発明の第３の実施の形態の説明図であり、第３の実施の形態の文字読み取り装置の説明図である。
【図１３】第１の発明の第３の実施の形態の文字読み取り装置の特に文字認識部の説明図である。
【図１４】第１の発明の第３の実施の形態の文字読み取り装置の動作説明図である。
【図１５】第１の発明の文字読み取り装置のさらに別の実施の形態の説明図である。
【図１６】従来技術および課題の説明図である。
【符号の説明】
１５，１７，２１，２３，３１，３３：帳票（既存の帳票）
１９，２５，３５：認識対象の帳票
Ｔ：帳票毎の特徴的な一部分（個別フォーマット認識領域）
Ｌ１：罫線（罫線の特徴的な部分）
ａ：紙面
４０、１４０：文字読み取り装置
５０、１５０：フォーマット認識装置
５１：第１の辞書
５３，１５３：第２の辞書
５５：第１のフォーマット認識系
５７，１５７：第２のフォーマット認識系
６０：読み取り制御データメモリ
７０：文字認識系
８０：制御部
９１：表示部
９３：入力部
９５：スキャナ
９７：帳票イメージメモリ[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a format recognition device for recognizing a format type of a form, and a character reading device for recognizing the type of a form format and reading characters according to the recognition result.
[0002]
[Prior art]
There are many types of forms in which the number, shape, size, position, and the like (hereinafter, referred to as a format) of character entry areas are different. When reading characters from a form, it is preferable to recognize the form of the form and read the characters according to the format. This is because characters can be read accurately and at high speed.
[0003]
As an example of a conventional technique for recognizing a format of a form, for example, a technique disclosed in Document 1 (“Overview of Character Recognition”, edited by Shinichiro Hashimoto, Ohmsha, March 20, 1982, pp. 203-206) There is.
[0004]
According to the technique disclosed in this document 1, as shown in FIG. 16, a form type number 13 (0123 in the figure) is printed on a part of the form 11. On the other hand, the character reading device stores in advance read control data necessary for reading characters from a form for each of various types of forms. Here, the read control data is information on the position of each character entry area of the form, information on the character type and character style written on the form, and the like.
[0005]
Then, the character reading device performs character recognition to read the form type number printed on the form. Further, the reading control data corresponding to the form is read based on the recognized number. Then, the character reading device reads characters, that is, performs character recognition from the image data of the form based on the control data.
[0006]
[Problems to be solved by the invention]
However, some forms have no form identification number printed thereon. For example, it is a private form such as a transfer request form, or a general slip on the market. Then, in the case of such a form, there is a problem that the reading control data cannot be used in the conventional character reading device.
[0007]
Therefore, it is desired to realize a format recognition device capable of recognizing the format of a form on which a form type number is not printed.
[0008]
Further, a character reading apparatus capable of recognizing a format of a form on which a form type number is not printed and reading characters from the form using the format recognition result is desired.
[0009]
[Means for Solving the Problems]
(1) Therefore, according to the first aspect of the format recognition apparatus of the present application, the apparatus for recognizing the format of a form includes a first format recognition system and a second format recognition system. However, the first format recognition system is a system that extracts the entire feature of the form to be recognized, compares this feature with the entire feature data of the existing form, and recognizes the format of the form to be recognized. .It is preferable that the first format recognition system is a recognition system that extracts features of ruled line patterns of the entire form to be recognized. In addition, it is preferable to use a recognition system that uses feature data based on ruled line patterns of the existing form as the entire feature data of the existing form. Because, in general, a form includes ruled lines. Furthermore, the ruled lines are salient parameters that characterize the form structure. Therefore, if the first-stage format recognition is performed by paying attention to the ruled line pattern, the format of the form can be recognized with considerable accuracy.In addition, the second format recognition system extracts a part of a feature of a form to be recognized, and compares the feature with feature data of a part of an existing form (hereinafter, also referred to as a “characteristic part”). This is a system for recognizing the format of a form to be recognized.
[0010]
According to the first aspect of the format recognition apparatus, the entire form (or the entire form and a part of the form when the second format recognition system operates) is used for pattern matching. Then, it is determined by a pattern matching method which of the existing forms the format of the form to be recognized corresponds to. Therefore, even if the form is not printed with the form type number, the format of the form can be recognized.
[0011]
Further, in the case of the first invention of this format recognition device, since the second format recognition system is provided, the following unique operation can be obtained.
[0012]
Among existing various forms, for example, there are forms in which only a part is different. For example, like the relationship between the form 15 shown in FIG. 1A and the form 17 shown in FIG. 1B, the arrangement of the ruled lines is the same except that the ruled line L1 exists or not. Further, there is a form having different characteristic figures (in the example of the figure, the form 15 is a pentagonal figure and the form 17 is a star-shaped figure). Also, as in the relationship between the form 21 shown in FIG. 2A and the form 23 shown in FIG. 2B, only the position of the ruled line L1 is different and the arrangement of the other ruled lines is the same. There is a report. When identifying this type of form, for example, when the form (recognition form) is dirty, creased, or the ruled line is faint, the form is extracted from the entire form. This pattern is likely to be, for example, a ruled line pattern between the form 15 and the form 17 or a ruled line pattern between the form 21 and the form 23. Then, when this ruled line pattern is collated using only the first format recognition system, the first recognition result that the format is the form 15 (21) and the second recognition that the format is the form of the form 17 (23). The result (of course, there may be three or more results) may satisfy the criteria set in advance in the first format recognition system, and may be regarded as a recognition result. That is, a plurality of recognition results may be output. Moreover, the result that is not the true recognition result may be the first format recognition result.
[0013]
Then, for example, when reading control data for character reading is read using the format recognition result, there may be a case where the reading control data is not proper reading control data. In such a case, character reading cannot be performed as desired.
[0014]
In such a case, in the first invention of the format recognition apparatus of the present application, the second format recognition can be performed by focusing on a characteristic part of the form using the second format recognition system, so that accurate format recognition can be performed. I can do it.
[0015]
In implementing the first invention of the format recognition apparatus, it is preferable that the second format recognition system be a recognition system that operates when the first format recognition system outputs a predetermined number or more of candidates. . Here, the predetermined number is an arbitrary number of 1 or 2 or more.
[0016]
In this way, it is considered that the result of the first format recognition system can be further complemented. Further, if no candidate is obtained in the first format recognition system, it is better to notify the operator, for example, of the fact without operating the second format recognition system, so that the recognition process can be performed more efficiently. This is because it is considered possible.
[0017]
Alternatively, when the first format recognition system outputs less than a predetermined number of candidates, the second format recognition may include a part of the recognition symmetric form and a part of the existing form for every form. When the first format recognition system outputs a predetermined number or more of candidates, a part of the feature data for each of the output forms is recognized. A recognition system that recognizes the format of a form to be recognized by comparing with a part of the form of the form that is symmetric with the form may be used. In such a case, an effective format recognition by the second format recognition system can be performed when the first format recognition system lists a candidate and when the candidate is not. In this case, a threshold value in the first format recognition system is used as a first format recognition result for each of all existing forms as candidates before performing collation or the like by the second format recognition system. It is preferable to forcibly set a predetermined value equal to or more than (threshold for selecting a candidate). This is because, as will be described in detail later, it is necessary to secure initial settings for rearranging candidate forms in the second format recognition system.
[0018]
In practicing the first invention of the format recognition apparatus, it is preferable to prepare in advance a feature data used in the first and second format recognition systems in a dictionary. This is because, for example, quick processing becomes possible. Therefore, in practicing the present invention, the format recognition device stores a first dictionary holding the characteristic data of the entire form and a second dictionary storing some of the characteristics of the existing form. It is preferable to provide them.
[0019]
In practicing the first invention of the format recognition apparatus, the second format recognition system stores in advance the position information of an area (ie, a characteristic part) where the characteristic data is obtained on the existing form. In addition, it is preferable that the recognition system extracts a feature from a part of the form to be recognized specified by the position information. This makes it possible to easily specify an area of the form to be recognized that corresponds to a characteristic part of the existing form. Of course, when a plurality of existing forms are prepared (typically in most cases), the second format recognition system holds the position information of the area where the characteristic data is obtained for each existing form. Is good. When a plurality of characteristic portions are set on one form, it is preferable to retain the position information of each of the plurality of portions. It is preferable that such position information be stored in a dictionary (for example, the above-mentioned second dictionary).
[0020]
The form of the characteristic part of the form is arbitrary. However, a characteristic part of the form is replaced with one or more characteristic figures (for example, a logo, an arbitrary mark, a character pattern, etc.) existing at a predetermined position (specifically, one or more positions) of the form. (See FIG. 1). For example, a company logo is often written on a form. This is because, if attention is paid to such a characteristic figure, it is considered that the classification of the form is easy.
[0021]
Alternatively, in the case of a form having a ruled line, a characteristic part of the form is one or more places where the characteristic ruled line exists (for example, a place where the ruled line L1 in FIGS. 2A and 2B exists). It is preferred that Forms often include ruled lines. Moreover, each form often includes a characteristic ruled line portion. This is because, if attention is paid to this characteristic ruled line portion, it is considered that the classification of the form is easy.
[0022]
Alternatively, a characteristic part of the form is replaced with one or more characteristic characters (for example, “A” in FIG. 11A and FIG. ○ electricity ”or“ △△ electricity ”). This is because it is considered that the company name or the like described in the form is easy to classify the form.
[0023]
When a characteristic part of the form is a character on the form, the second format recognition system is preferably a recognition system as described below.
[0024]
That is, the character name of the characteristic character is held in advance as characteristic data of the existing form. Of course, the second format recognition system cuts out a character from an area of a form to be recognized that corresponds to an area in which a characteristic character of an existing form was written (of course, if a character exists in this area). Sometimes not.) Then, the extracted character is recognized as a character to obtain a character name. Then, the format is recognized by collating with the character name stored in advance as the characteristic data of the existing form.
[0025]
In some cases, the form type number described in the related art may be used as a characteristic part of the form. However, in this case, the present invention is applied by treating the form type number as a characteristic pattern for pattern matching instead of character recognition.
[0026]
In addition, there may be a case where a characteristic part of a form is set at a plurality of positions, that is, two or more positions on one form. In this case, since the number of collation points increases, the format recognition accuracy improves accordingly.
[0027]
In implementing the first invention of the format recognition apparatus, the feature extracted by the first format recognition system and the feature data used for collation by the first format recognition system are extracted from the entire form in the same manner. If so, there is no particular limitation. But,As mentioned above,The first format recognition system uses the ruled line pattern of the entire form.Features aboutIt is good to use a recognition system that extracts And of the existing formoverallExisting forms as feature dataRuled lineIt is preferable to use a recognition system using feature data based on a line pattern. Because, in general, a form includes ruled lines. Furthermore, the ruled lines are salient parameters that characterize the form structure. Therefore, if the first-stage format recognition is performed by paying attention to the ruled line pattern, the format of the form can be recognized with considerable accuracy.
[0028]
An apparatus for recognizing the format of a form by focusing on the ruled line pattern of the form is disclosed in Japanese Patent Application Laid-Open No. Hei 9-259221 to the present applicant.
[0029]
In implementing the first invention of the format recognition apparatus, a display unit that displays one or both of a candidate form output by the first format recognition system and a candidate form output by the second format recognition system; It is preferable that the apparatus further includes an input unit for inputting an instruction for selecting one or a plurality of forms from the forms displayed on the display unit or an instruction for not applicable.
[0030]
With such a configuration including the display unit and the input unit, the operator can confirm the correctness of the format automatically recognized by the format recognition device. Therefore, the form format can be recognized more accurately.
[0031]
In practicing the first aspect of the format recognition apparatus, it is preferable to use data for collation as a feature matrix. This is because the use of the feature matrix allows the characteristics of the pattern to be expressed well and allows effective matching (pattern matching). Note that there is no particular limitation on how to create the feature matrix. However, the claim16-18The method of creating a feature matrix as claimed in the above is preferable for the purpose of the present invention.
[0032]
Further, in the first aspect of the format recognition apparatus, the first format recognition system may be constituted by two or more format recognition systems different in the manner of extracting the entire feature of the form. Further, there may be a case where the second format recognition system is configured by two or more format recognition systems different in the manner of extracting some features of the form. As examples of the second format recognition system including a plurality of recognition systems, for example, a format recognition system focusing on a characteristic figure existing in a part of a form, a format recognition focusing on a ruled line of a part of the form An example can be given in which at least two recognition systems are selected from among a system and a format recognition system focusing on characteristic characters on a form.
[0033]
(2) Further, the second recognition of the format recognition device of this applicationClearlyAccording to the apparatus for recognizing a format of a form, a part of a form to be recognized is extracted, and the characteristic is compared with characteristic data of a part (a characteristic part) of an existing form, and the recognition is performed. A format recognition system for recognizing a format of a target form is provided.
[0034]
According to the second aspect of the format recognition apparatus, a part of the form is used for pattern matching. Then, it is determined by a pattern matching method which of the existing forms the format of the form to be recognized corresponds to. Therefore, even if the form is not printed with the form type number, the format of the form can be recognized. Of course, since only the characteristic part of the form is focused on, the format recognition processing can be speeded up.
[0035]
In implementing the second invention of the format recognition apparatus, the configuration of the format recognition system is the same as the preferred configuration example described above for the second format recognition system of the first invention of the format recognition apparatus. It is preferable to have a similar configuration.
[0036]
Also in the case of the second invention, the format recognition system may be constituted by two or more format recognition systems which differ from each other in the manner of extracting a part of the form.
[0037]
(3) According to the first aspect of the character reading apparatus of the present application, the format recognition apparatus of the first aspect described above, and the reading control necessary for reading characters from the existing form for each of the existing forms A read control data memory that holds data and outputs one or a plurality of control data according to the recognition result of the format recognition device; and a form to be recognized based on the read control data output from the read control data memory. And a character recognizing device for recognizing characters from.
[0038]
According to this character reading device, character recognition can be performed based on control data appropriate for a form to be recognized. Therefore, character recognition can be performed accurately and efficiently.
[0039]
In practicing the first invention of this character reading device, if the second format recognition system of the format recognition device is a recognition system that performs format recognition by focusing on some characters of a form, It is preferable that the components used for clipping and character recognition are used for both the second format recognition system and the character recognition system.
[0040]
That is, a character cutout unit for cutting out a character pattern from image data of a form to be recognized, a character pattern memory for storing the cutout characters, a character recognition dictionary, and a cutout character pattern as data in the character recognition dictionary. As a character recognition unit for collating and recognizing a character, a character cutout unit, a character pattern memory, a character recognition dictionary, and a character recognition unit which are shared by the second format recognition system and the character recognition system are provided. It is suitable. In this case, compared with the case where the above is not performed, effects such as a reduction in the hardware scale and software scale of the character reading device can be obtained.
[0041]
(4) According to the second aspect of the character reading apparatus of the present application, the format recognition apparatus of the second aspect described above, and reading control necessary for reading characters from the existing form for each of the existing forms A read control data memory that holds data and outputs one or a plurality of control data according to the recognition result of the format recognition device; and a form to be recognized based on the read control data output from the read control data memory. And a character recognizing device for recognizing characters from.
[0042]
According to this character reading device, character recognition can be performed based on control data appropriate for a form to be recognized. Therefore, character recognition can be performed accurately and efficiently.
[0043]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of a format recognition device and a character reading device according to the present invention will be described with reference to the drawings. It should be noted that each drawing used in the description merely schematically shows each component so that the present invention can be understood. Further, in each of the drawings, the same components are denoted by the same reference numerals, and overlapping description thereof may be omitted.
[0044]
1. First invention of format recognition device and character reading device
1-1. Brief description of the first invention
First, the outline of the first invention will be described. There are many types of forms. Some examples of forms will be described with reference to FIGS. 1, 2, and 11. FIG.
[0045]
FIGS. 1A and 1B are diagrams showing existing forms 15 and 17 as a first example of a form group having a slightly different format. FIG. 1C is a diagram showing an example of a form 19 to be recognized whose format is unknown. In each of the forms, the form type number is not printed. This is an example of a correct answer if the format of the form 19 to be recognized is recognized as the format of the form 15.
[0046]
The forms 15 and 17 have the same format except that the ruled line L1 is present or not, and that different characteristic figures are printed in the upper right corner of the form.
[0047]
FIGS. 2A and 2B are diagrams showing existing forms 21 and 23 as a second example of a form group having a slightly different format. FIG. 2C is a diagram showing an example of a form 25 to be recognized whose format is unknown. In each of the forms, the form type number is not printed. This is an example in which if the format of the form 25 to be recognized is recognized as the format of the form 21, the answer is correct.
[0048]
The form 21 and the form 23 have the same format except that the position of the ruled line L1 is different.
[0049]
FIGS. 11A and 11B are diagrams showing existing forms 31, 33 as a third example of a form group having a slightly different format. FIG. 11C is a diagram showing an example of a form 35 to be recognized whose format is unknown. In each of the forms, the form type number is not printed. If the format of the form 35 to be recognized is recognized as the format of the form 31, this is an example of a correct answer.
[0050]
The forms 31 and 33 have the same format except that the ruled line L1 exists or not, and that different characteristic characters are printed in the upper right corner of the form. That is, the form 31 has a ruled line L1 at the upper left corner thereof, and a characteristic character “○ electricity” is printed at the upper right corner of the form. The form 33 does not have the ruled line L1 and has a characteristic character "@Electric" printed on the upper right corner.
[0051]
Each of the forms shown in FIGS. 1A to 2C and FIGS. 11A to 11C is a form on which a ruled line for dividing an area where a predetermined item is described is printed on a sheet a. Then, characters (01, 02 in the figure) representing the attributes of the written characters and predetermined items are transmitted to the used forms (FIG. 1 (C), FIG. 2 (C), FIG. 11 (C)). (In the figure, ABC, that is, DEF, Kakikukeko, Aiokaokikukeko) are described.
[0052]
Here, in the forms 15, 17 and 19 shown in FIGS. 1A to 1C, a predetermined area T including a pentagonal or star-shaped figure is a characteristic part of each form (individual format recognition area). Also called). In the case of the forms 15 and 19, the ruled line L1 can be said to be a characteristic part, but here, a pentagonal figure part is considered as a characteristic part.
[0053]
Also, the forms 21, 23, and 25 shown in FIGS. 2A to 2C are characterized in that the ruled line L1, which is one of the ruled lines, is at the upper right corner or the upper left corner of the form. Therefore, the area T including the ruled line L1 can be said to be a characteristic part (individual format recognition area) of each form.
[0054]
Also, in the forms 31, 33, and 35 shown in FIGS. 11A to 11C, the predetermined area T including ○ electricity or Δ electricity can be said to be a characteristic part (individual format recognition area) for each form. . Note that, in the case of the forms 31 to 35, the ruled line L1 can be said to be a characteristic portion, but here, the portion of ○ electricity or △ electricity is considered as a characteristic portion.
[0055]
Of course, this characteristic figure, characteristic ruled line, or characteristic character is not limited to the above example, but any characteristic figure (for example, a rectangle, a circle, an ellipse, etc.), any characteristic Can be ruled lines or any characteristic character. Further, the region T may be a case where there are a plurality of locations in one form.
[0056]
The form 19 to be recognized shown in FIG. 1C has no form type number printed thereon, but the ruled line pattern of this form 19 is the same as the form 15 shown in FIG. 1A. The form 25 to be recognized shown in FIG. 2C has no form type number printed thereon, but the ruled line pattern of this form 25 is the same as the form 21 shown in FIG. 2A. The form 35 to be recognized shown in FIG. 11C has no form type number printed thereon, but the ruled line pattern of this form 35 is the same as the form 31 shown in FIG. 11A. Therefore, a method of comparing a pattern extracted from the entire form with a pattern prepared in a dictionary in advance, for example, a ruled line extracted from the entire form disclosed in Japanese Patent Application Laid-Open No. 9-259221 to the present applicant. The form 19 can be recognized as a form in the same format as the form 15, the form 25 can be recognized as a form in the same format as the form 21, and the form 35 can be recognized as a form 31 by a method of comparing the pattern with a prepared dictionary. It can be recognized that the form has the same format.
[0057]
However, when the form 19 (the form 25 and the form 35) is dirty, creased, or the ruled line is faint, the pattern extracted from the entire form is, for example, the form 15 A ruled line pattern in the middle of the form 17 (a ruled line pattern in the middle of the form 21 (form 31) and the form 23 (form 33)) tends to be generated. Then, it becomes difficult to discriminate between the two.
[0058]
Thus, in the first invention of this application, a first step of recognizing a format of a form to be recognized by collating a pattern of an entire form with a pattern of each of a plurality of forms in a first dictionary; Extracting a pattern corresponding to each characteristic part from the form to be recognized, collating the pattern with a pattern in the second dictionary, and recognizing the format of the form to be recognized. A format recognition device to be implemented and a character reading device including the recognition device are proposed. This will be described below.
[0059]
1-2. Description of the configuration of the device 40 according to the first embodiment
First, an embodiment (first embodiment) of a character reading apparatus suitable for recognizing the format of the form group described with reference to FIG. 1 and reading characters will be described.
[0060]
FIG. 3 is a functional block diagram schematically showing the configuration of the character reading device 40 according to the first embodiment. The character reading device 40 includes the format recognition device 50 of the first embodiment, a read control data memory 60, a character recognition system 70, a control unit 80, a display unit 91, and an input unit 93. I have.
[0061]
The character reading device 40 includes a scanner 95 as a means for optically reading a form (not shown) and a form image memory 97 for storing image data of the form read by the scanner 95. I have. Hereinafter, each of the constituent components 50 to 93 will be described in detail.
[0062]
The format recognition device 50 includes a first dictionary 51, a second dictionary 53, a first format recognition system 55, a second format recognition system 57, and a control unit 80 (also a control unit of the character reading device 40). There is).
[0063]
The first dictionary 51 (hereinafter, also referred to as a format recognition dictionary 51) previously holds, as characteristic data, pattern data extracted from the entire plurality of existing forms having different formats.
[0064]
A plurality of existing forms having different formats are arbitrary forms prepared in advance. For example, various forms including the form 15 shown in FIG. 1A and the form 17 shown in FIG. 1B can be used.
[0065]
Further, the first dictionary 51 holds ruled line pattern data extracted from the entirety of each existing form as pattern data. Further, the first dictionary 51 also holds information indicating the type of each form.
[0066]
The data of the ruled line pattern held in the first dictionary 51 can be in any suitable form according to the specifications of the reading device 40. In this embodiment, the ruled line pattern is a feature matrix represented by the features of each region obtained by dividing the ruled line pattern into a plurality of regions. What is the feature of each divided region is arbitrary. For example, ruled line patterns are scanned in advance in a plurality of different directions to extract directional patterns, and the ruled line patterns are divided into a plurality of M × N regions, and the number of directional patterns of each divided region is determined by the characteristics of each region. (For example, JP-A-57-23185).
[0067]
The second dictionary 53 (hereinafter, also referred to as an individual format recognition dictionary 53) previously holds, as characteristic data, pattern data extracted from characteristic portions of a plurality of existing forms having different formats. In the example of FIG. 1, the form 15 holds pentagonal pattern data, and the form 17 holds star-shaped pattern data.
[0068]
The data of these patterns can be stored in the second dictionary 53 in any suitable form. In the case of this embodiment, as the data of the pattern to be stored in the second dictionary 53, the number of characteristic parts set on the form (in FIGS. 1A and 1B, 1), information indicating position information, size, and form type of a characteristic part (hereinafter, these are referred to as individual format recognition information), and a characteristic matrix of a pattern.
[0069]
A first format recognition system 55 (hereinafter, also simply referred to as a format recognition system) extracts features of the entire form from the image data of the form to be recognized, and stores the extracted features in feature data in the first dictionary. And the format of the form to be recognized is recognized. More specifically, in this case, the format of the form to be recognized is recognized by comparing the feature matrix of the pattern extracted from the form to be recognized with the feature matrix in the first dictionary 51.
[0070]
The first format recognition system 55 of this embodiment extracts a ruled line pattern as a pattern of the entire form to be recognized. Therefore, the first format recognition system 55 according to the present embodiment includes a ruled line extraction unit 55a that extracts a ruled line pattern of the entire form from a form to be recognized, a ruled line pattern memory 55b that stores the extracted ruled line pattern, The ruled line pattern is compared with the first dictionary 51, and the format recognition unit 55c recognizes the format of the form to be recognized. The details of the first format recognition system 55 will be described later in the description of the operation. Such a first format recognition system 55 can be constituted by, for example, a format recognition system disclosed in JP-A-9-259221 according to the present application.
[0071]
The second format recognition system 57 (hereinafter, also referred to as an individual format recognition system 57) includes a predetermined number of candidate forms (for example, one or more in this example) as recognition results by the first format recognition system 55. , The candidate form is also referred to as a type or a format candidate). Then, some features of the form to be recognized are extracted. In this embodiment, a feature of a pattern corresponding to the characteristic part for each of the one or more candidate forms listed by the first format recognition system is extracted, and the extracted feature is stored in the second dictionary. The format of the form to be recognized is recognized by collating with the characteristic data.
[0072]
The second format recognition system 57 of this embodiment includes a cutout unit 57a (hereinafter, referred to as a partial pattern cutout unit 57a) that cuts out a pattern corresponding to a characteristic part of an existing form from image data of the form to be recognized. The memory 57b for storing the extracted partial pattern (hereinafter also referred to as an individual format recognition pattern memory 57b) and the data of the extracted pattern are compared with the data of the pattern in the second dictionary 53, and A recognition unit 57c (hereinafter, referred to as an individual format recognition unit 57c) for recognizing the format of the form is configured.
[0073]
FIG. 4 is a functional block diagram illustrating a specific configuration example of the second format recognition system 57, particularly, the individual format recognition unit 57c. Of course, this is only an example, and other configurations may be used.
[0074]
The individual format recognition unit 57c in the case of FIG. 4 includes a line width calculation unit 101, a frame detection unit 103, a sub-pattern extraction unit 105, a feature matrix creation unit 107, and a collation unit 109. The details of each of the components 101 to 107 will be described later in the section on the operation, but briefly described below.
[0075]
The line width calculation unit 101 calculates the line width of the pattern extracted by the partial pattern extraction unit 57a from the form to be recognized, that is, the pattern stored in the individual format recognition pattern memory 57b.
[0076]
The frame detection unit 103 detects a frame (figure frame) circumscribing the pattern cut out by the partial pattern cutout unit 57a from the form to be recognized.
[0077]
The sub-pattern extraction unit 105 scans the pattern cut out by the partial pattern cut-out unit 57a from the form to be recognized in a plurality of directions, and extracts a sub-pattern for each scanning direction. In this case, the sub-pattern extraction unit 105 includes an HSP extraction unit 105a that extracts a horizontal sub-pattern (HSP), a VSP extraction unit 105b that extracts a vertical sub-pattern (VSP), and an LSP extraction that extracts a diagonally left sub-pattern (LSP). It comprises a unit 105c, an RSP extraction unit 105d for extracting a diagonally right sub-pattern (RSP), an HSP memory 105e for storing each sub-pattern, a VSP memory 105f, an LSP memory 105g, and an RSP memory 105h.
[0078]
The feature matrix creation unit 107 extracts features from sub-patterns for each of the horizontal, vertical, left-to-left, and right-to-left scanning directions, and creates a feature matrix from the features of each sub-pattern.
[0079]
The collating unit 109 performs collation using the feature matrix created by the feature matrix creating unit 107 and the feature matrix of each standard form stored in the individual format recognition dictionary 53, and based on the result, the form to be recognized. And the certainty and distance from each standard form used for matching.
[0080]
The read control data memory 60 stores, for each of the plurality of forms, read control data necessary for reading characters from the forms. As the read control data, for example, data relating to extraction of a character pattern and data relating to recognition of a character pattern are stored.
[0081]
Examples of the data relating to the character pattern extraction include, for example, the position of a character line existing on the form paper a, the position and number of fields existing on the form paper a, the pitch and number of characters existing on the form paper a. Can be mentioned. Further, as data relating to the recognition of a character pattern, for example, a character type and a character type of a character described on a form paper surface a can be cited. Of course, it is not limited to these.
[0082]
The read control data memory 60 outputs read control data for one or more formats according to the recognition result output from the format recognition device 40. The output read control data is input to the character recognition system 70 via the control unit 80 in this case.
[0083]
The character recognition system 70 recognizes characters from a form to be recognized based on the read control data output from the read control data memory 60. This character recognition system 70 can have any suitable configuration. In this embodiment, the character recognition system 70 includes a character cutout unit 70a, a character pattern memory 70b, a character recognition dictionary 70c, and a character recognition unit 70d.
[0084]
The character extracting unit 70a extracts a character pattern from the form image data. The character pattern memory 70b stores the character pattern extracted by the character extracting unit 70a. The character recognition dictionary 70c holds dictionary data of characters. The character recognizing unit 70d extracts a feature from the extracted character pattern, compares the feature with data in the character recognition dictionary 70c, and recognizes the character pattern based on the matching result.
[0085]
Such a character recognition system can be constituted by, for example, a character recognition system disclosed in Japanese Patent Application Laid-Open No. 9-259221 of the present application (details will be described later in the operation description).
[0086]
The control unit 80 includes the components 55a to 55c of the first format recognition system 55, the components 57a to 57c of the second format recognition system 57, the read control data memory 60, and the components of the character recognition system 70. The components 70a to 70d, the display unit 91, the input unit 93, the scanner 95, and the form image memory 97 are controlled, respectively.
[0087]
The display unit 91 performs display for exchanging information between the character reading device 40 and the operator. The display section 91 can be constituted by any suitable display device such as a CRT. The display unit 91 is also used as a display unit that displays one or both of a candidate form output by the first format recognition system 55 and a candidate form output by the second format recognition system 57. The plurality of candidate forms (format candidates) may be displayed in the form of a list, or may be displayed partly.
[0088]
The input unit 93 inputs various instructions from the operator to the character reading device. The input unit 93 can be composed of any suitable device such as a keyboard and a mouse. When the second format recognition system 57 displays a plurality of format candidates on the display unit 91, the input unit 93 selects one or a plurality of forms from the plurality of forms displayed on the display unit 91. It is also used as an input unit for inputting an instruction or an instruction of not applicable.
[0089]
1-3. Description of the operation of the device 40 according to the first embodiment
(Schematic description of overall operation)
First, the overall operation of the character reading device 40 will be described. FIG. 5 is a diagram showing the overall operation flow of the character reading device 40 of this embodiment.
[0090]
First, the image data of the form 19 to be recognized is obtained by the scanner 95. Therefore, the scanner 95 optically scans the sheet a of the form 19 to generate form image data composed of quantized electric signals, for example, black and white binary electric signals, and stores this data in the form image memory 97. I do. Here, a line drawing element is represented by a black bit, and a background pixel is represented by a white bit. Note that the line drawing element is a pixel representing a line in the sheet a of the form 19, and specifically, in this case, a portion where a predetermined number of black bits are continuous. The background pixel is a pixel representing a ground portion (a background portion of a line) on the sheet a of the form 19. The pressing surface for holding the form 19 set on the scanner 95 has a color, for example, white, which is converted into a background pixel, like the ground on the sheet a of the form 19, and therefore the pressing surface is also represented by a white bit here.
[0091]
The pressing surface for holding down the form may be a color converted into a black pixel, for example, black, like the line in the sheet a of the form 19.
[0092]
When the form image data is stored in the form image memory 97, the control unit 80 activates the ruled line extraction unit 55a of the first format recognition system 55 (start of FIG. 5).
[0093]
The activated ruled line extraction unit 55a scans the form image data stored in the form image memory 97 to extract ruled line patterns (details will be described later), and stores the extracted ruled line patterns in the ruled line pattern memory 55b. (S1 in FIG. 5).
[0094]
When the ruled line extraction unit 55a finishes extracting the ruled line pattern, the control unit 80 activates the format recognition unit 55c.
[0095]
The activated format recognition unit 55c checks the ruled line pattern stored in the ruled line pattern memory 55b against dictionary data of a form format stored in the format recognition dictionary 51 (S2 in FIG. 5). As a result of the comparison, if there is a predetermined number or more (here, for example, one or more) of candidate forms (S3 in FIG. 5), the processing after step S4 (details will be described later) is performed. If there is no candidate form, the fact is output to the control unit 80 (S21). In this case, the control unit 80 performs, for example, a reject process.
[0096]
On the other hand, when one or more candidate forms are output in the recognition by the first format recognition system 55, the control unit 80 activates the second format recognition system 57. Then, as will be described later, the format of the form to be recognized is recognized by performing collation focusing on a part of the form.
[0097]
However, even if one or more candidate forms are indicated in the recognition by the first format recognition system 55, if the individual format recognition information corresponding to each of these candidates is not present in the individual format recognition dictionary 53, the The recognition result obtained by the first format recognition system 55 is output to the control unit 80. If at least the data of the standard form stored in the format recognition dictionary 51 is stored as the data to be stored in the individual format recognition dictionary 53, the individual format recognition information corresponding to the plurality of form types is stored in the individual format recognition dictionary. It is preferable because it can be obtained from the dictionary 53.
[0098]
When the individual format recognition information corresponding to all or a part of each of the candidate forms listed by the first format recognition system 55 is present in the individual format recognition dictionary 53, the control unit 80 performs the processing based on the image data of the form to be recognized. Position information for cutting out a portion corresponding to a characteristic portion of each form is obtained from the individual format recognition dictionary 53 (S5 in FIG. 5). That is, if the form 15 in FIG. 1A and the form 17 in FIG. 1B are the format candidates with respect to the form 19 to be recognized, the characteristic form of each of the forms 15 and 17 is used. The control unit 80 obtains, from the individual format recognition dictionary 53, position information for cutting out a part corresponding to the part T from the image data of the form 19 to be recognized. This is input to the partial pattern cutout unit 57a.
[0099]
The partial pattern cutout unit 57a cuts out a pattern from the form image memory 97 based on the above-mentioned position information and stores it in the individual format recognition pattern memory 57b (S6 in FIG. 5).
[0100]
When the partial pattern cutout unit 57a ends the pattern cutout, the control unit 80 activates the individual format recognition unit 57c. The activated individual format recognition unit 57c extracts the feature from the pattern stored in the individual format recognition pattern memory 57b, and compares the extracted feature with the feature data of the individual format recognition dictionary stored in the individual format recognition dictionary 53. (S7 in FIG. 5). When a plurality of characteristic portions T are defined in one form, the process is repeated from step S5 by the number of times, and a collation result is obtained (S8 in FIG. 5).
[0101]
The above processing (S4 to S8) is repeated for each type of individual format identification, that is, for each of a plurality of format candidates.
[0102]
Next, the control unit 80 changes the collation results in the order of judging the type having the highest similarity based on the collation result of the individual format recognition system (S10 in FIG. 5), and selects one or a plurality of types from the collation results. Then, the selected type is used as the recognition result.
[0103]
When the control unit 80 inputs only one type as the recognition result of the form format, it reads out the read control data corresponding to the recognition result from the read control data memory 60 and sends the read read control data to the character recognition system 70. Output (S11 in FIG. 5).
[0104]
When the control unit 80 inputs a plurality of types as the recognition results of the form format, it is preferable that information for confirming the recognition results be displayed on the display unit 91. The input unit 93 may be configured so that the operator can give an instruction to select a desired form from the plurality of forms or an instruction not to correspond to the form based on this information.
[0105]
For example, the control unit 80 causes the display unit 91 to display candidate forms based on the recognition result of the form format. The operator selects a recognition result that can be determined to be correct by looking at this display, and inputs a signal indicating the selected recognition result to the control unit 80 via the input unit 93.
[0106]
The control unit 80 reads the read control data corresponding to the recognition result selected by the operator from the read control data memory 60, and outputs this to the character recognition system 70 (S11 in FIG. 5).
[0107]
As described above, the read control data in this case is control data for extracting a character pattern and control data for recognizing a character pattern. Therefore, in detail, the control unit 80 outputs control data for extracting a character pattern to the character extracting unit 70a and control data for recognizing the character pattern to the character recognizing unit 70d.
[0108]
Next, the control unit 80 activates the character extracting unit 70a and the character recognizing unit 70d. Then, the character extracting unit 70a extracts a character pattern from the form image data in units of one character, and stores the extracted character pattern in the character pattern memory 70b (S12 in FIG. 5). The character recognizing unit 70d extracts a feature from the extracted character pattern, matches the feature with dictionary data of the character recognition dictionary 70c (S13 in FIG. 5), recognizes a character pattern based on the matching result, The recognition result is output to the control unit 80 (S14 in FIG. 5).
[0109]
The control unit 80 inputs the recognition result of the character pattern for each character, and determines whether or not the recognition has been completed for all the characters on the sheet a of the form 19 to be recognized (S15 in FIG. 5). .
[0110]
If the recognition has not been completed for all the characters on the form 19 to be recognized, the character extracting unit 70a and the character recognizing unit 70d repeat the processing of S11 to S14 according to the operation control of the control unit 80. When the recognition of all the characters on the sheet a of the form 19 is completed, the reading of characters of one form is completed, and the character recognition result obtained for one form is output to the subsequent device (FIG. 5). End).
[0111]
Next, a detailed operation of each unit will be described.
[0112]
(Operation for extracting ruled line patterns)
First, the extraction of a ruled line pattern performed by the ruled line extracting unit 55a will be described. The ruled line extraction unit 55a scans the entire form 19 in the form image memory 97 and extracts a horizontal ruled line pattern and a vertical ruled line pattern as ruled line patterns representing a form format.
[0113]
When extracting a horizontal ruled line pattern, the form image data is horizontally scanned, and a line drawing element of the form image data that is continuous on the horizontal scanning line by a length Lh equal to or longer than the horizontal ruled line threshold TH (Lh) is determined by a line drawing element of the horizontal ruled line pattern. Extract as Line drawing elements (including one line drawing element that is not continuous but continuous on a horizontal scanning line) of the form image data in which the continuous length Lh is less than the horizontal ruled line threshold TH (Lh) are defined by the horizontal ruled line pattern. Extract as background pixel. The background pixels of the form image data are also extracted as the background pixels of the horizontal ruled line pattern. The horizontal ruled line pattern is a pattern composed of line drawing elements and background pixels extracted from the form image data in this manner.
[0114]
Similarly, when extracting the vertical ruled line pattern, the form image data is vertically scanned, and a line drawing element of the form image data that is continuous on the vertical scanning line by a length Lv that is equal to or longer than the vertical ruled line threshold TH (Lv) is extracted. Extract as line drawing element. Line drawing elements (including one line drawing element that is not continuous but continuous on the vertical scanning line) of the form image data in which the continuous length Lv is less than the vertical ruled line threshold TH (Lv) are the vertical ruled line patterns. Extract as background pixel. Background pixels of the form image data are also extracted as background pixels of the vertical ruled line pattern. The vertical ruled line pattern is a pattern composed of line drawing elements and background pixels extracted from the form image data in this manner.
[0115]
When the extracted ruled line pattern is stored in the ruled line pattern memory 55b, a storage area common to the extracted horizontal ruled line pattern and vertical ruled line pattern (hereinafter, ruled line common storage area) is provided in the ruled line pattern memory 55b. Vertical ruled line patterns may be superimposed and stored in the ruled line common storage area. In addition, an individual storage area (hereinafter referred to as a ruled line individual storage area) is provided in the ruled line pattern memory 55b for each of the extracted horizontal ruled line pattern and vertical ruled line pattern, and the horizontal ruled line pattern and the vertical ruled line pattern are stored separately. It may be stored in an area.
[0116]
When storing in the ruled line common storage area, the background pixels are stored in the entire ruled line common storage area (initialization of the ruled line common storage area), and then the line drawing elements of the extracted horizontal ruled line pattern are stored in the common storage area. At the same time, the extracted line drawing elements of the vertical ruled line pattern are stored in the common storage area.
[0117]
In the case of storing in the ruled line individual storage area, background pixels are stored in the entire ruled line individual storage area provided for the horizontal ruled line pattern (initialization of the ruled line individual storage area), and then provided for the horizontal ruled line pattern. The extracted line drawing elements of the horizontal ruled line pattern are stored in the ruled line individual storage area. Also, background pixels are stored in the entire ruled line individual area provided for the vertical ruled line pattern (initialization of the ruled line individual storage area), and then the extracted vertical ruled line pattern is stored in the ruled line individual storage area provided for the vertical ruled line pattern. The line drawing element of is stored.
[0118]
FIG. 6 is a diagram for explaining the operation of the ruled line extraction unit according to this embodiment, and shows a specific example of an operation flow focusing on an operation of extracting a line drawing element of a ruled line pattern. Here, a case where the extracted horizontal ruled line pattern and vertical ruled line pattern are stored in different storage areas will be described.
[0119]
In this case, a corresponding xy coordinate system is set on the form image data storage area of the form image memory 97 and on the ruled line common storage area of the ruled line pattern memory 55b (for example, the horizontal direction is x-y). The axis direction and the vertical direction are defined as the y-axis direction.), Pixel data is read from the storage area of the pixel position represented by this coordinate system, or the pixel data is stored in the storage area of the pixel position represented by this coordinate system. It is assumed that data can be written.
[0120]
The ruled line extracting unit 55a activated by the control unit 80 stores a background pixel, here, a white bit, over the entire ruled line common storage area of the ruled line pattern memory 55b (initialization of the ruled line common storage region).
[0121]
Thereafter, the ruled line extracting unit 55a starts a process of extracting a horizontal ruled line pattern with the scanning direction being the horizontal direction (start of FIG. 6). In this case, the horizontal ruled line threshold value TH (Lh) is used as the threshold value TH, and the length Lh of a continuous line drawing element on the horizontal scanning line is obtained as the continuous length L.
[0122]
Next, the ruled line extraction unit 55a initializes the continuous length L to zero (S1 in FIG. 6), and thereafter, horizontally scans the form image data stored in the form image memory 97 and outputs the form image data. The pixel data is read (S2 in FIG. 6). The pixel data represents data for one pixel.
[0123]
Next, the ruled line extraction unit 55a determines whether or not the pixel data of the read form image data is a line drawing element, here, a black bit (S3 in FIG. 6).
[0124]
If the determination result in S3 is that the pixel data is a line drawing element, the length L of the continuation is determined, and the pixel data of the pixel data is set as a line drawing element candidate position (hereinafter, horizontal ruled line candidate position) for the horizontal ruled line pattern. The position is stored (S4 in FIG. 6), and thereafter, it is determined whether or not scanning of one current scanning line has been completed for the form image data (S5 in FIG. 6). Here, when the determination result that the pixel data is a line drawing element is obtained in S3, the continuous length L is counted up by one, and thus the number of continuous line drawing elements is counted. Find L.
[0125]
If it is determined in S5 that the scanning of one line of the current scanning line has not been completed, the process returns to S2 to read the next pixel data on the current scanning line.
[0126]
When the result of the determination in S5 is that the scanning of one current scanning line has been completed, the continuous length L is compared with the threshold value TH (S6 in FIG. 6). Here, since the horizontal ruled line pattern is extracted, the determined continuous length L is the continuous length Lh on the horizontal scanning line, and the horizontal ruled line threshold TH (Lh) is used as the threshold TH.
[0127]
If it is determined in S6 that the continuous length L is equal to or greater than the threshold value TH, the ruled line extraction unit 55a stores the continuous line drawing elements having the length L in the ruled line pattern memory 55b (S7 in FIG. 6). . Here, the line drawing element is stored in the stored ruled line common storage area at the horizontal ruled line candidate position. Subsequent to S7, the continuous length L is initialized, and the stored horizontal ruled line candidate positions are deleted (S8 in FIG. 6). Then, is the scanning of the form image data over the entire form 19 completed? It is determined whether or not it is (S9 in FIG. 6). If it is determined in S6 that the continuous length L is less than the threshold value TH, S8 to S9 are performed without performing S7.
[0128]
If it is determined in S9 that the scanning of the form image data has not been completed over the entire form 19, the process returns to S2 to read out pixel data on the next scanning line after the current scanning line. If it is determined in S9 that the scanning of the form image data has been completed over the entire form, the process of extracting the horizontal ruled line pattern from the form 19 is completed (end).
[0129]
Further, when the pixel data read out in the determination of S3 is determined to be not a line drawing element, that is, when the pixel data is a black pixel up to the end of one line, the continuous length L is compared with the threshold value TH ( S10 in FIG. 6).
[0130]
When the determination result that the continuous length L is equal to or larger than the threshold value TH is obtained in S10, the ruled line extracting unit 55a stores the continuous line drawing elements of the length L in the ruled line pattern memory 55b (S11 in FIG. 6). . Next, the continuous length L is initialized to zero, and the stored horizontal ruled line candidate position is erased (S12 in FIG. 6). Thereafter, it is determined whether or not the scanning of the current scanning line has been completed. Is determined (S13 in FIG. 6). If the scanning of one line of the current scanning line has not been completed, the process returns to S2 to read the next pixel data on the current scanning line. When scanning of one line of the current scanning line is completed, S9 is performed. If it is determined in S10 that the continuous length L is less than the threshold value TH, S12 to S13 are performed without performing S11.
[0131]
Next, the ruled line extraction unit 55a starts the process of extracting a vertical ruled line pattern by setting the scanning direction to the vertical direction this time (start). In this case, a vertical ruled line threshold TH (Lv) is used as the threshold TH, and a length Lv of a line drawing element that is continuous on a vertical scanning line is obtained as a continuous length L. The other basic operations are the same as the above-described horizontal ruled line pattern extraction operation, and thus description thereof will be omitted.
[0132]
FIG. 7 shows an example in which line drawing elements of horizontal and vertical ruled line patterns are extracted from the form 19 shown in FIG. 1C and line drawing elements of these patterns are stored in separate ruled line individual storage areas as described above. Show.
[0133]
In FIG. 7A, reference numeral 110 denotes a ruled line individual storage area provided for the horizontal ruled line pattern, 111 denotes an area corresponding to the form paper a in the ruled line individual storage area 110, and 112 denotes a horizontal ruled line pattern. In FIG. 7B, reference numeral 113 denotes a ruled line individual storage area provided for the vertical ruled line pattern, 114 denotes an area corresponding to the form paper a in the ruled line individual storage area 113, and 115 denotes a vertical ruled line pattern. Areas inside the ruled line individual storage areas 110 and 113 and outside the paper surface corresponding areas 111 and 114 are pixels of a color representing a pressing surface on which the form 19 is pressed when the form 19 is set on the scanner 95, and are background pixels here. .
[0134]
When the extracted horizontal ruled line pattern and vertical ruled line pattern are stored in a common storage area, the ruled line patterns may be stored in the storage area in a state of being superimposed, and the description thereof will be omitted.
[0135]
(Recognition operation in the first format recognition system)
Next, the operation of the first format recognition system 55 will be described. In recognizing the form format in the first format recognition system 55, the ruled line extraction unit 55a scans form image data of an unknown form format and extracts a ruled line pattern as described above. Then, from the extracted ruled line pattern, the format recognizing unit 55c extracts features by, for example, a method disclosed in Japanese Patent Laid-Open No. 57-23185 as described above, and identifies the extracted features in various forms of the form standard format. Each time, it is checked against the features stored in the format recognition dictionary 51.
[0136]
Then, in this comparison, when a certain condition (hereinafter, an individual recognition condition) is satisfied in this comparison, the format of that type is set as a format candidate of the form to be recognized.
[0137]
This individual recognition condition can be any suitable condition. For example, the format recognizing unit 55c outputs types of the form formats in descending order of similarity, and outputs a distance and a certainty factor for each type. Here, the distance, similarity, and certainty are as follows. The result of comparing the features of the form standard format held in the format recognition dictionary 51 with the features extracted from the image data of the form to be recognized is the distance. The magnitude of the distance is the similarity, and the smaller the distance, the higher the similarity. The certainty is a numerical value obtained by normalizing the distance, and the greater the certainty, the higher the similarity.
[0138]
Then, the present invention is not limited to this. For example, the first condition that the specified number is selected as a format candidate in the order of the above certainty factors, At least the second condition of selecting as a format candidate or the third condition of obtaining a distance difference between adjacent types in descending order of the degree of certainty and selecting a candidate whose distance difference is equal to or less than a certain value as a format candidate A format satisfying one condition is selected as a format candidate. Usually, format candidates are selected by combining these criteria.
[0139]
A specific example of selecting a format candidate by combining the above conditions will be described below with reference to a table described later.
[0140]
Table 1 is a table showing the form standard format, the certainty factor, and the distance output by the format recognition unit 55c. This is a table arranged in order of the degree of certainty. Here, a predetermined number is selected as the individual recognition condition in ascending order of confidence. Here, an example will be considered in which up to the sixth highest degree of certainty are selected as format candidates. Therefore, the form standard formats AF are selected as format candidates. These selected candidates are hereinafter referred to as valid candidates.
[0141]
Next, Table 2 shows an example in which, in addition to the individual recognition conditions of Table 1, a certainty factor of a form standard format type having a certainty factor or more is selected. Here, an example is shown in which a condition that the certainty factor is 90 or more is added. Therefore, the form standard formats A to E are selected.
[0142]
Next, Table 3 shows an example in which, in addition to the individual recognition conditions in Table 2, a distance difference between types of adjacent form standard formats is obtained, and a candidate whose distance difference is equal to or less than a certain value is selected. Here, an example is shown in which a condition that the distance difference is 30 or less is added. Therefore, the form standard formats A to D are selected.
[0143]
(Recognition operation in the second format recognition system)
When the first format recognition system 55 indicates a predetermined number or more (here, for example, one or more) format candidates for the form to be recognized, the format recognition operation by the second format recognition system 57 is as follows. Done.
[0144]
As described above, the individual format recognition dictionary 53, which is the second dictionary, previously stores pattern data of a characteristic part of each standard form.
[0145]
In creating data to be held in the individual format dictionary 53, a plurality of existing forms are prepared, an area T (see FIG. 1 and the like) used for individual format recognition is set for each form, and individual forms are determined from the form image data. A figure as the format recognition area T is cut out, and the cut out figure is scanned to extract features. This feature is combined with information defining the position, size, shape, and the like of the area T, for each type of form standard format, and individually associated with each other, and stored in the individual format recognition dictionary 53 as dictionary data.
[0146]
A plurality of the individual format recognition areas T may be set on the form. In this case, information defining the characteristics and positions, sizes, shapes, and the like of the plurality of areas is stored in the individual format recognition dictionary 53 corresponding to the type of the form standard format.
[0147]
In the individual format recognition, the individual format recognition is performed in order from the last candidate (for example, the candidate shown in Table 3) in the valid candidates output by the format recognition unit 55c of the first format recognition system 55 from the top candidate. Is performed as follows.
[0148]
The partial pattern cutout unit 57a of the individual format recognition system 57 reads out information about the individual format recognition area T stored in advance corresponding to the valid standard form standard format type from the individual format recognition dictionary 53. When a plurality of individual format recognition areas T are set, it is preferable to rank the areas in advance and perform individual format recognition in that order. If there is a type for which the area for individual format recognition is not set, it is left as a valid candidate.
[0149]
The partial pattern cutout unit 57a cuts out the image of the portion corresponding to the individual format recognition area T from the form image memory 97, and stores it in the individual format recognition pattern memory 57b. The individual format recognition unit 57c scans the individual format recognition pattern memory 57b, cuts out the pattern of the characteristic portion from the pattern, and extracts the characteristics of the cut out image.
[0150]
The operation of the individual format recognition unit 57c will be described in detail with reference to FIG.
[0151]
The line width calculation unit 101 of the format recognition unit 57c obtains the line width W for the pattern stored in the individual format recognition pattern memory 57b. Here, the line width calculation unit 101 scans the graphic pattern stored in the individual format recognition pattern memory 57b using a 2 × 2 window, and counts the number of times Q in which all points of the 2 × 2 window become line drawing elements. And the total number A of line drawing elements in the figure pattern are counted. Then, w = A / (AQ) is obtained as the line width w of the graphic pattern.
[0152]
The frame detection unit 103 detects a frame (graphic frame) circumscribing the line portion of the cut graphic pattern. Here, when the positions of the left and right sides in the horizontal direction of the graphic frame are XL and XR, and the positions of the upper and lower sides in the vertical direction are YU and YD, the upper left vertex position (XL, YU) and the lower left vertex position (XL , YD), the upper right vertex position (XR, YU) and the lower right vertex position (XL, YD), and the positions XL, XR, YU, YD are detected as follows.
[0153]
In detecting the left side position XL, vertical scanning lines are sequentially moved from left to right, and for each vertical scanning line, the presence / absence of a line drawing element on the search line is determined. The sub-scanning line position of the obtained vertical scanning line is detected as the left side position XL. Further, in detecting the right side position XR, the vertical scanning lines are sequentially moved from right to left, and the presence or absence of a line drawing element on the search line is determined for each vertical scanning line. The sub-scanning line position of the determined vertical scanning line is detected as the right side position XR. Further, in detecting the upper side position YU, the horizontal scanning lines are sequentially moved from top to bottom, and for each horizontal scanning line, the presence or absence of a line drawing element on the scanning line is determined. The sub-scanning line position of the determined horizontal scanning line is detected as the upper side position YU. In detecting the lower side position YD, the horizontal scanning lines are sequentially moved from bottom to top, and for each horizontal scanning line, the presence or absence of a line drawing element on the scanning line is determined. The sub-scanning line position of the determined horizontal scanning line is detected as the lower side position YD.
[0154]
Next, the HSP extraction unit 105a scans the graphic pattern held in the individual format recognition pattern memory 57b in the horizontal direction, extracts a horizontal sub-pattern from the pattern, and stores the extracted horizontal sub-pattern in the HSP memory 105e. .
[0155]
Note that among the line drawing elements of the graphic pattern, a line drawing element having a continuous length L on the horizontal scanning line that is equal to or larger than the graphic threshold TH1 is extracted as a line drawing element of the horizontal sub-pattern. In addition, among the line drawing elements of the graphic pattern, a line drawing element (including a single line drawing element) whose length L continuous on the horizontal scanning line is less than the graphic threshold value TH1 and a background pixel of the graphic pattern are Extract as a line drawing element of horizontal sub-pattern. The line drawing element of the horizontal sub-pattern corresponds to a line in the horizontal direction.
[0156]
The VSP extraction unit 105b scans the graphic pattern held in the individual format recognition pattern memory 57b in the vertical direction, extracts a vertical sub-pattern from the pattern, and stores the extracted vertical sub-pattern in the VSP memory 105f.
[0157]
Note that, among the line drawing elements of the graphic pattern, line drawing elements having a continuous length L on the vertical scanning line that is equal to or greater than the graphic threshold TH2 are extracted as line drawing elements of the vertical sub-pattern. In addition, among the line drawing elements of the graphic pattern, a line drawing element (including one line drawing element) having a continuous length on the vertical scanning line less than the graphic threshold value TH2 and a background pixel of the graphic pattern are vertically It is extracted as a line drawing element of the sub pattern. The line drawing element of the vertical sub-pattern corresponds to a line in the vertical direction.
[0158]
Further, the LSP extraction unit 105c scans the figure pattern held in the individual format recognition pattern memory 57b in the diagonally left 45 degrees direction, extracts a left diagonal sub-pattern from the pattern, and extracts the extracted diagonally left sub-pattern from the LSP memory. Store in 105g.
[0159]
Note that, among the line drawing elements of the graphic pattern, a line drawing element whose length L that is continuous on the scanning line in the diagonally left direction is equal to or larger than the graphic threshold TH3 is extracted as a line drawing element of the left diagonal sub-pattern. In addition, among the line drawing elements of the graphic pattern, a line drawing element (including a single line drawing element) whose continuous length on the scanning line in the diagonal left direction is less than the graphic threshold value TH3 and a background pixel of the graphic pattern are It is extracted as the line drawing element of the left oblique sub-pattern. The line drawing element of the left diagonal sub-pattern corresponds to a line in the diagonal left direction of 45 degrees.
[0160]
Further, the RSP extracting unit 105d scans the cut-out figure pattern in the diagonal right direction at 45 degrees, extracts a diagonal right sub-pattern from the pattern, and stores the extracted diagonal right sub-pattern in the RSP memo 105h.
[0161]
Note that, among the line drawing elements of the graphic pattern, line drawing elements having a continuous length L on the scanning line in the diagonal right direction that is equal to or greater than the graphic threshold value TH4 are extracted as line drawing elements of the diagonal right sub-pattern. In addition, among the line drawing elements of the figure pattern, a line drawing element (including one line drawing element) having a continuous length on the scanning line in the diagonal right direction that is less than the figure threshold value TH4 and a background pixel of the figure pattern are It is extracted as a line drawing element of the right oblique sub-pattern. The line drawing element of the diagonally right sub-pattern corresponds to a line in the diagonally right direction of 45 degrees.
[0162]
By setting the figure thresholds TH1 to TH4 as appropriate, it is possible to extract horizontal, vertical, diagonally left, and diagonally right sub-patterns from the graphic pattern. Here, figure thresholds TH1 to TH4 are set such that TH1 = N1 · w, TH2 = N2 · w, TH3 = N3 · w, and TH4 = N4 · w based on the line width w. Note that N1 to N4 are arbitrarily set values. Although not limited to this, for example, N1 = N2 = N3 = N4 = 2.
[0163]
FIG. 8 shows an example in which vertical, horizontal, diagonally right and diagonally left sub patterns VSP, HSP, RSP and LSP are extracted from the individual format recognition area T of the form 19 to be recognized shown in FIG. Is shown.
[0164]
Next, the feature matrix creation unit 107 extracts features from the sub-patterns for each scanning direction, and creates a feature matrix of a character pattern extracted from the features of each sub-pattern.
[0165]
The feature matrix creation unit 107 divides a region corresponding to the graphic frame of the horizontal sub-pattern into N × M regions, and assigns a feature amount l to each region._ij, And these N × M feature values l_ijAn N × M-dimensional feature matrix is created.
[0166]
Feature l_ijRepresents the feature amount obtained in the area of the i-th row and the j-th column. For example, the total number B of line drawing elements in the region of the i-th row and the j-th column_ijAnd the line width w obtained for the graphic pattern,_ij= B_ij/ W. Similarly, an N × M-dimensional feature matrix is created for each of the vertical, diagonally left, and diagonally right sub-patterns.
[0167]
Next, the feature matrix creation unit 107 uses the feature matrix created for each of the horizontal, vertical, diagonally left and diagonally right sub-patterns to set the horizontal length of the graphic frame to ΔX, the vertical length to ΔY, and the feature amount l_ijIs obtained by normalizing_ijAsk for. This feature amount L_ijIs L for the horizontal sub-pattern_ij= L_ij/ ΔX, L for vertical sub-pattern_ij= L_ij/ ΔY, L for diagonally left sub-pattern_ij= L_ij/ (ΔX²+ ΔY²)^1/2It becomes.
[0168]
Next, the feature matrix creating unit 107 obtains a feature matrix normalized for each of the horizontal, vertical, left-diagonal, and right-diagonal sub-patterns, and obtains an N × M × 4-dimensional feature matrix composed of these normalized feature matrices. f_AAnd create a feature matrix f_AIs output to the matching unit 109.
[0169]
The matching unit 109 performs matching using the feature matrix of the cut-out graphic pattern and the feature matrix of the individual format dictionary 53, and the degree of certainty with the type of the feature matrix compared with the cut-out graphic based on the matching result. And output the distance.
[0170]
If the output certainty factor or distance satisfies a predetermined condition, this form standard format type is selected as a valid candidate.
[0171]
Here, the predetermined condition can be any suitable condition. For example, there is a method in which the distance is a valid candidate if the distance is equal to or less than a certain value;
[0172]
The feature is extracted for the area stored in the individual format recognition dictionary 53 for all the form standard format types satisfying the individual recognition conditions. Then, the distance and the certainty factor are calculated by comparing with the feature information stored in the dictionary data in advance, and if these satisfy predetermined conditions, this type is selected as a valid candidate, and if the predetermined conditions are not satisfied. Exclude from valid candidates. An example of selecting valid candidates will be described with reference to Table 4.
[0173]
Table 4 shows an example in which the first format recognition system 55 performs the second format recognition on the types A to D that are final candidates.
[0174]
In other words, in Table 4, the individual format recognizing unit 57c acquires the features of the individual format recognition areas corresponding to the candidate types A, B, C, and D from the form to be recognized (details will be described later). 7 is a table showing the result of comparing the extracted feature with a feature previously stored in the dictionary 53 by a distance. Here, if the condition that the distance is 30 or less is set as the predetermined condition, as shown in Table 5, the types A and C are not valid candidates.
[0175]
When a plurality of individual format recognition areas T are set, and as a result of performing individual format recognition of the first area, if there are a plurality of valid candidates, the second and third individual format recognition areas are also described above. The feature extraction described above and the dictionary data collation are performed, and a type that does not satisfy a predetermined condition is regarded as a non-valid candidate. When the number of valid candidates becomes one, the subsequent processing is not performed.
[0176]
After the processing is completed for all the individual format recognition areas, if there are a plurality of valid candidates, the candidates are exchanged under predetermined conditions according to the result of the distance and certainty in each area.
[0177]
Examples of the candidate replacement will be described with reference to Tables 6 to 8 described later. First, a first example will be described with reference to Tables 6 and 7.
[0178]
The first example shows an example of candidate replacement in the case where first and second areas are set as individual format recognition areas T in a candidate form. In the recognition result of the first format recognition system, the type B is the first candidate form and the type D is the second candidate form (see Table 6). However, in the second format recognition system, as a result of comparing the form to be recognized with the first and second individual format recognition areas of the types B and D, only the first individual format recognition area in the type B is obtained. A recognition result equal to or greater than the threshold is obtained. In the type D, a recognition result equal to or larger than the threshold is obtained for each of the first and second individual format recognition areas. That is, type D has a larger number of matchings with the second dictionary than type B. In such a case, in the present invention, replacement of candidate forms is performed as shown in Table 7. That is, the order of the candidate forms is changed such that the type D becomes the first candidate form and the type B becomes the second candidate form.
[0179]
As a second example of candidate replacement, the following may be performed. That is, when a plurality of areas are set as individual format recognition areas in the candidate form, the plurality of areas are prioritized. Then, in the collation of the individual format recognition areas having higher priorities, the type having the smallest distance or certainty is replaced so as to become the first candidate form. For example, in the examples of Tables 6 and 8, ranking is performed such that the matching result of the first individual format recognition area is prioritized. Therefore, in the case of this example, the order of the valid candidates of the type B and the type D is the same (accordingly, the result is the same as in Table 6).
[0180]
After performing the above processing, one or more standard form formats that are valid candidates are output to the control unit 80 as a result of the individual format recognition system 57. Here, when the recognition result of the individual format recognition system 57 is one, the control unit 80 obtains the read control data corresponding to the recognition result from the read control data memory 60 and sends it to the character recognition system 70. When there are a plurality of recognition results of the individual format recognition system 57, the control unit 80 obtains read control data corresponding to each recognition result from the read control data memory 60, and sends the read control data to the character recognition system 70. The form may be sent, or a form corresponding to these recognition results may be displayed on the display unit 91, and an accurate form may be selected from these forms by the operator. In the latter case, the control unit 80 obtains read control data for the form selected by the operator from the read control data memory 60 and sends it to the character recognition system 70.
[0181]
The character recognition system 70 performs a series of processes for character recognition, such as cutting out a character pattern from a form to be recognized, based on the sent character read data.
[0182]
(Character recognition operation)
The character recognition process is performed based on read control data obtained as a result of format recognition by the format recognition device 50.
[0183]
That is, a character pattern is cut out from the image data of a form to be recognized based on data related to cutting out a character pattern in the read control data. In addition, the recognition dictionary is specified by using data (for example, a font) related to character pattern recognition in the read control data.
[0184]
Next, the feature is extracted from the extracted character pattern. The features of the extracted character pattern are collated with the features stored in advance in the character recognition dictionary 70c. For example, the matching is performed using the distance between the feature of the cut-out character pattern and the feature stored in the character recognition dictionary 70c as “reliability” of the recognition. Then, one or more character names are selected from the character names stored in the character recognition dictionary 70c based on the certainty of the above recognition, and the selected character names are output as character pattern recognition results. I do.
[0185]
1-4. Description of the configuration of the device according to the second embodiment
The character reading device 40 of the first embodiment described above is a device particularly suitable when a characteristic part of a form is a characteristic figure. Of course, the character reading device 40 of the first embodiment can cope with a case where a characteristic part of the form is a characteristic part of the ruled line. However, when the characteristic part of the form is the characteristic part of the ruled line, the position of the ruled line L1 is different from the characteristic form of the form, for example, as shown in each form shown in FIGS. In the case of such a part, as shown in FIG. 9, the configuration of the sub-pattern extraction unit 105 of the individual format recognition unit 57c may be simplified as compared with the first embodiment. That is, the LSP extraction unit 105c, the RSP extraction unit 105d, the LSP memory 105g, and the RSP memory 105h provided in the first embodiment are omitted, and the sub-pattern extraction unit 105 is replaced with an HSP extraction unit 105a and a VSP extraction unit 105b. , An HSP memory 105e, and a VSP memory 105f.
[0186]
It is clear that a characteristic part of the ruled line is a horizontal ruled line, a vertical ruled line, or a combination of vertical and horizontal ruled lines, and if a pattern relating to HSP and VSP can be extracted and stored, a sub-pattern can be extracted. .
[0187]
In the character reading device according to the second embodiment, for example, data stored in the second dictionary is changed in accordance with a change in the configuration of the sub-pattern extraction unit. Change the configuration. Other configurations may be the same as those of the character reading device 40 of the first embodiment, for example.
[0188]
1-5. Description of the operation of the device of the second embodiment
The difference between the operation of the apparatus of the second embodiment and the apparatus of the first embodiment is that the operation of the sub-pattern extraction unit in the second format recognition system 57 and the sub-pattern extracted by the sub-pattern extraction unit This is the processing of the feature matrix creation unit and the collation unit accompanying the decrease in the number of types. That is, the sub-pattern extraction unit extracts only VSP and HSP. The feature matrix creation unit and the matching unit create features in the horizontal and vertical scanning directions from the form to be recognized, and match the features with the features stored in the dictionary.
[0189]
FIG. 10 is a diagram showing a characteristic part T of the form 25 to be recognized shown in FIG. 2C, and a vertical sub-pattern (VSP) and a horizontal sub-pattern (HSP) extracted from this part T. .
[0190]
In the character reading device according to the second embodiment, operations other than the above-described operations in the second format recognition system 57 are performed in the same manner as in the first embodiment, and thus description thereof is omitted.
[0191]
1-6. Description of the configuration of the device according to the third embodiment
The character reading device 40 of the first embodiment described above is a device particularly suitable when a characteristic part of a form is a characteristic figure. However, as described above with reference to FIG. 11, a characteristic part of the form may be a character.
[0192]
That is, the forms 31 and 33 shown in FIGS. 11A and 11B have different characteristic characters printed on the upper right corner of the form, as to whether or not the ruled line L1 exists. It has the same format except for the point. That is, the form 31 has a ruled line L1 at the upper left corner thereof, and a characteristic character “○ electricity” is printed at the upper right corner of the form. The form 33 does not have the ruled line L1 and has a characteristic character "@Electric" printed on the upper right corner. On the other hand, in the form 35 to be recognized shown in FIG. 11C, the form type number is not printed, but the ruled line pattern of the form is the same as the form 31 shown in FIG. 11A. Therefore, a method of comparing a pattern extracted from the entire form with a pattern prepared in a dictionary in advance, for example, a ruled line extracted from the entire form disclosed in Japanese Patent Application Laid-Open No. 9-259221 to the present applicant. The form 35 can be recognized as a form in the same format as the form 31 by a method of comparing the pattern with a prepared dictionary.
[0193]
However, when the form 35 is dirty, creased, or the ruled line is faint, the pattern extracted from the entire form is, for example, an intermediate pattern between the form 31 and the form 33. It tends to be a simple ruled line pattern. Then, it becomes difficult for the two to recognize each other.
[0194]
Therefore, in the third embodiment, when the first format recognition system 55 outputs a predetermined number or more (for example, one or more) of candidate forms, the first format recognition system 55 recognizes some characters of the form as second format recognition. Perform format recognition based on the focus. The details will be described below.
[0195]
FIG. 12 is a block diagram showing a configuration of a character reading device 140 suitable for use in the third embodiment.
[0196]
Note that the character reader 140 of this embodiment differs from the character reader 40 of the first embodiment in the configuration of the second dictionary 153 and the configuration of the second format recognition system 157. . Otherwise, the configuration is the same as that of the first embodiment. Thus, in the following description, only the differences will be described.
[0197]
First, the second dictionary 153 (also referred to as an individual format recognition dictionary 153) includes one or a plurality of characteristic features printed on a predetermined part (individual format recognition area T) of each of a plurality of existing forms having different formats. Character names of various characters are stored in advance. However, in the case of this embodiment, in addition to the character names of one or a plurality of characteristic characters described above, the set number of individual format recognition areas T, their positions, and character types of characteristic characters for each form. The second dictionary 153 also stores information such as information (this is also referred to as “individual format recognition information”) in advance.
[0198]
To create such an individual format recognition dictionary 153, a plurality of existing forms are prepared. For example, a plurality of forms including the forms 31 and 33 shown in FIGS. 11A and 11B are prepared. Then, an individual format identification area T is set for each form, and the character names of the one or more characteristic characters, along with the number, position, character type, number of characters, etc. of the set areas, are written in a dictionary. The data is stored in the individual format recognition dictionary 153 in advance as data.
[0199]
The second format recognition system 157 operates when the first format recognition system 55 indicates one or more candidate forms (hereinafter, the candidate form is also referred to as a type or a format candidate). Then, a character is cut out from an area corresponding to the characteristic part T of each of the one or more candidate forms in the form to be recognized, and the cut out character is recognized to obtain a character name. Then, the obtained character names are compared with the character names in the second dictionary to recognize the format of the form to be recognized.
[0200]
The second format recognition system 157 of this embodiment includes a character cutout unit 157a, a character pattern memory 157b, a character recognition dictionary 157c, a character recognition unit 157d, and an individual format recognition unit 157e.
[0201]
The character cutout unit 157a obtains the position data of the individual format recognition area T for each candidate form from the second dictionary 153 by the first format recognition system 55 as a candidate, and based on the data, the form image memory 97. Cut out the pattern from Further, a pattern for each character is cut out by a known method.
[0202]
The character pattern memory 157b stores a pattern for each character cut out by the character cutout unit 157a.
[0203]
The character recognition dictionary 157c holds various character features (standard features) and character names in advance.
[0204]
What kind of data holds the characteristics of the character is arbitrary. However, the character recognition dictionary according to the present embodiment divides a character pattern into a plurality of regions and holds the characteristics of the characters in the form of a characteristic matrix represented by the characteristics of the divided regions.
[0205]
The character recognition unit 157d extracts features from the character pattern, matches the extracted features with a character recognition dictionary to recognize characters, and outputs the character names to the individual format recognition unit 157e.
[0206]
The configuration of the character recognition unit 157d can be arbitrarily suitable. For example, it can be constituted by a character recognition unit disclosed in Japanese Patent Application Laid-Open No. 9-259221. Of course, this is only an example, and the present invention is not limited to this.
[0207]
FIG. 13 is a functional block diagram illustrating a specific configuration example of the character recognition unit 157d.
[0208]
The character recognition unit 157d in the case of FIG. 13 includes a line width calculation unit 201, a frame detection unit 203, a sub-pattern extraction unit 205, a feature matrix creation unit 207, and a collation unit 209. These details will be described later in the section of operation description, but will be simply described as follows.
[0209]
The line width calculation unit 201 calculates the line width of the pattern extracted by the character extraction unit 157a.
[0210]
The frame detection unit 203 detects a frame (character frame) circumscribing the pattern cut out by the character cutout unit 157a.
[0211]
The sub-pattern extraction unit 205 scans the pattern extracted by the character pattern extraction unit 157a in a plurality of directions, and extracts a sub-pattern for each scanning direction. In this case, the sub-pattern extraction unit 205 includes an HSP extraction unit 205a that extracts a horizontal sub-pattern (HSP), a VSP extraction unit 205b that extracts a vertical sub-pattern (VSP), and an LSP extraction that extracts a left diagonal sub-pattern (LSP). It comprises a unit 205c, an RSP extracting unit 205d for extracting a diagonally right sub-pattern (RSP), an HSP memory 205e for storing a corresponding sub-pattern, a VSP memory 205f, an LSP memory 205g, and an RSP memory 205h.
[0212]
The feature matrix creation unit 207 extracts features from the sub-patterns for each of the horizontal, vertical, left-to-left, and right-to-left scanning directions, and creates a feature matrix from the features of each of the sub-patterns.
[0213]
The matching unit 209 performs matching using the feature matrix created by the feature matrix creating unit 207 and the feature matrix of each character stored in the character recognition dictionary 157c, and outputs a character name based on the result.
[0214]
The character cutout unit 157a, character pattern memory 157b, character recognition dictionary 157c, and character recognition unit 157d are shared by the character cutout unit 70a, character pattern memory 70b, character recognition dictionary 70c, and character recognition unit 70d of the character recognition system 70. Is preferred. This has the advantage that the hardware scale and software scale of the character reading device 140 can be reduced.
[0215]
The individual format recognition unit 157e collates the character recognition result of the character recognition unit 157d with data stored in advance in the individual format recognition dictionary 153, and identifies one or more format candidates of the form to be recognized. Output as
[0216]
1-7. Description of the operation of the device 140 according to the third embodiment
Next, the operation of the character reading device 140 will be described. FIG. 14 is a diagram showing the overall operation flow of the character reading device 140.
[0217]
Note that the processing by the first format recognition system 55 and the processing by the character recognition system 70 are the same as those of the device 40 of the first embodiment, and therefore, in the following description, the second format recognition system 157 will be described. The operation will be mainly described.
[0218]
As in the case of the character reading device 40 of the first embodiment, a ruled line pattern is extracted from image data obtained from a form to be recognized (for example, the form 35 shown in FIG. 11C) (S1 in FIG. 14). ), The data of the ruled line pattern is compared with the data of the pattern in the first dictionary 51 (S2 in FIG. 14).
[0219]
Next, it is determined whether or not there is a predetermined number or more (here, for example, one or more examples) of candidate forms (S3 in FIG. 14). If there is no candidate form, the control unit 80 is notified to that effect (S21 in FIG. 14). If there is one or more candidate forms, the control unit 80 activates the second format recognition system 157. However, even when one or more candidate forms are indicated in the recognition by the first format recognition system 55, if the individual format recognition information corresponding to each of these types is not in the individual format recognition dictionary 153, the second format is used. The recognition result obtained by the first format recognition system 55 is output to the control unit 80.
[0220]
When the individual format recognition information corresponding to all or a part of each of the candidate forms output by the first format recognition system 55 is present in the individual format recognition dictionary 153 (S4 in FIG. 14), the control unit 80 determines the recognition target. The position information for cutting out a portion corresponding to a predetermined portion for each type is obtained from the individual format recognition dictionary 153 from the image data of the form (S5 in FIG. 14). That is, for example, when the form 31 in FIG. 11A and the form 33 in FIG. 11B are the format candidates for the form 35 to be recognized, the characteristics of the forms 31 and 33 are respectively used. The control unit 80 obtains, from the individual format recognition dictionary 153, positional information for cutting out a part corresponding to the typical part T from the image data of the form 35 to be recognized. This is input to the character cutout unit 157a.
[0221]
The character cutout unit 157a cuts out a pattern from the form image memory 97 based on the position information and stores it in the character pattern memory 157b (S6 in FIG. 14).
[0222]
When the character cutout unit 157a ends the pattern cutout, the control unit 80 activates the character recognition unit 157d. The activated character recognizing unit 157d extracts a feature from the pattern stored in the character pattern memory 157b, compares the feature with feature data of a standard pattern stored in the character recognition dictionary 157c, and recognizes a recognition result. Is output (S7 in FIG. 5).
[0223]
Each process of feature extraction, collation, and character recognition in the character recognition unit 157d can be performed in the same manner as in the individual format recognition unit 57c described with reference to FIG. 4 in the embodiment of the first invention. Therefore, a brief description will be given below.
[0224]
First, the line width calculation unit 201 calculates w = A / (A−Q) as a line width w from the character pattern stored in the character pattern memory 157b, as in the case of the line width calculation unit 101 in the first invention. Ask.
[0225]
The frame detection unit 203 detects a frame (character frame) circumscribing the line portion of the cut-out character pattern, similarly to the case of the frame detection unit 103 in the first invention.
[0226]
The sub-pattern extraction unit 205 scans the extracted character pattern in each of the horizontal, vertical, diagonally right, and diagonally left directions, and extracts a sub-pattern for each scanning direction. The extraction processing of each sub-pattern is performed in the same manner as the processing in the sub-pattern extraction unit 105 in the first invention.
[0227]
The feature matrix creation unit 207 extracts features from the sub-patterns for each scanning direction, and creates a feature matrix of a character pattern cut out from the features of each sub-pattern, similarly to the feature matrix 107 in the first invention. That is, the feature matrix creation unit 207 divides the area corresponding to the character frame of the horizontal sub-pattern into N × M areas, and assigns a feature amount l to each area._ij, And these N × M feature values l_ijAn N × M-dimensional feature matrix is created.
[0228]
The matching unit 209 performs matching using the feature matrix of the extracted character pattern and the feature matrix of the data in the character recognition dictionary 157c, similarly to the matching unit 107 in the first invention. Then, the character name, certainty factor, and distance of the feature matrix compared with the cut-out character are output.
[0229]
When a plurality of characteristic portions T are defined in one candidate form, the process is repeated from step S5 by the number of the portions, and a recognition result is output (S8 in FIG. 5).
[0230]
The above processing (S4 to S8) is repeated for each type of individual format identification, that is, for each of a plurality of format candidates.
[0231]
When the character recognition unit 157d finishes outputting the recognition result for each format candidate, the control unit 80 activates the individual format recognition unit 157e.
[0232]
The activated individual format recognition unit 157e checks the character name of each format candidate from the character recognition unit 157d against the data in the individual format recognition dictionary 153 to recognize the format of the form to be recognized (see FIG. 14). S10). At this time, the recognition results are exchanged in the order in which the format candidates are determined as having the highest similarity (type), and one or a plurality of format candidates are selected.
[0233]
When the control unit 80 inputs only one type as the recognition result of the form format, the control unit 80 reads the read control data corresponding to the recognition result from the read control data memory 60, and reads the read control data. The data is output to the character recognition system 70 (S11 in FIG. 14).
[0234]
When the control unit 80 inputs a plurality of types as the recognition results of the form format, it is preferable that information for confirming the recognition results be displayed on the display unit 91. The input unit 93 may be configured so that the operator can give an instruction to select a desired form from the plurality of forms or an instruction not to correspond to the form based on this information.
[0235]
For example, the control unit 80 causes the display unit 91 to display candidate forms based on the recognition result of the form format. The operator selects a recognition result that can be determined to be correct by looking at this display, and inputs a signal indicating the selected recognition result to the control unit 80 via the input unit 93.
[0236]
The control unit 80 reads the read control data corresponding to the recognition result selected by the operator from the read control data memory 60, and outputs the read read control data to the character recognition system 70 (S11 in FIG. 14).
[0237]
As described above, the read control data in this case is control data for extracting a character pattern and control data for recognizing a character pattern. The control unit 80 outputs control data for extracting a character pattern to the character extracting unit 70a, and outputs control data for recognizing the character pattern to the character recognizing unit 70d.
[0238]
Thereafter, as described in the first embodiment, the character recognition system 70 recognizes characters on a form to be recognized.
[0239]
In the above description, the second format recognition system operates when the first format recognition system outputs a predetermined number or more of candidates, and the first format recognition system determines that the number of candidates is less than the predetermined number. In the case of only outputting (including 0), the recognition system leaves the process to the control unit (see steps S3 and S21 in FIGS. 5 and 14). However, when the second format recognition system outputs less than a predetermined number of candidates, the first format recognition system outputs a part of the recognition-symmetric form and a part of the existing form for every form. When the first format recognition system outputs a predetermined number or more of candidates, a part of the feature data for each of the output forms is recognized. A recognition system that recognizes the format of a form to be recognized by comparing with a part of the form of the form that is symmetric with the form may be used.
[0240]
In this way, effective format recognition by the second format recognition system can be performed when the first format recognition system names a candidate and when the candidate is not.
[0241]
An operation example of such a configuration will be described with reference to FIG. FIG. 15 is a diagram in which the process of step S21 in FIG. 5 is changed according to the present mode.
[0242]
When the first format recognition system 55 outputs a predetermined number or more of candidates, the processes in steps S4 to S9 are performed in the same manner as described with reference to FIG. 5 (FIG. 15). On the other hand, when the first format recognition system 55 outputs less than a predetermined number of candidates (for example, less than 1), the second format recognition system 57 sets all existing forms in the second dictionary 53 as candidates. (Step S31 in FIG. 15). Then, for the form to be recognized, matching based on a characteristic part of each existing form, that is, processing by the second format recognition system is performed. However, before performing collation or the like by the second format recognition system 57, a threshold value (candidate) in the first format recognition system is used as a first format recognition result of each of all existing forms in the second dictionary 53. The control unit 80 forcibly sets a predetermined value equal to or larger than the threshold value for selecting (step S3 in FIG. 15). The reason for this is to secure the initial setting for rearranging the candidate forms described above. After that, the format recognition is performed by the second format recognition system 57, and one or a plurality of candidate forms as format candidates of the form to be recognized are determined from the existing forms. The candidate form determined in this manner is preferably displayed on the display unit 91, and the operator may use the input unit 93 to determine the final supplementary form.
[0243]
Further, even in the case of focusing on the characteristic characters described with reference to FIG. 14, when the first format recognition system outputs less than a predetermined number of candidates, the second format recognition system outputs the second dictionary. The second format recognition system can be configured so as to recognize the format of a form to be recognized, using all existing forms in the list as candidates. In that case, the configuration of the second format recognition system 157 (FIG. 12) may be changed so that the process of step S31 of FIG. 15 is performed instead of the process of step S21 in FIG.
[0244]
2. Second invention of format recognition device and character reading device
In the above-described first invention, a first step of recognizing a format of a form to be recognized by collating a pattern of the entire form with a pattern of each existing form in the first dictionary; A feature is extracted from an area corresponding to a characteristic part of each candidate form, and the special work is compared with feature data of an existing form in a second dictionary to recognize a format of a form to be recognized. Claimed device to perform the steps and.
[0245]
However, in some cases, the format of the form may be identified by focusing only on the characteristic part of the form existing in a part of the form. This second invention is an example.
[0246]
Therefore, the format recognition apparatus according to the second aspect of the present invention extracts a part of a form of a form to be recognized, compares the characteristic with a part of characteristic data of an existing form, and formats the form of the form to be recognized. It has a format recognition system that recognizes
[0247]
More specifically, data of patterns extracted from characteristic portions of each of a plurality of existing forms having different formats are stored in advance in a dictionary and image data of a form to be recognized. A pattern extraction unit for extracting a pattern corresponding to a characteristic part of each of the patterns, and comparing the data of the pattern extracted by the pattern extraction unit with the data in the dictionary, to format the form of the form to be recognized. It has a format recognition system to recognize.
[0248]
Further, a character reading device according to a second aspect of the present invention includes the format recognition device according to the second aspect of the invention, a read control data memory, and a character recognition system.
[0249]
However, the read control data memory holds read control data necessary for reading characters from the form for each of the forms, and outputs one or a plurality of read control data according to the recognition result of the format recognition device. It is. The character recognition system recognizes a character from a form to be recognized based on read control data output from a read control data memory.
[0250]
The characteristic part of the form is, for example, a figure (including a character pattern) printed on a predetermined part of the form, a characteristic part of the ruled line, or a characteristic part of these figures and the ruled line. Some can be both.
[0251]
Embodiments of the format recognition device and the character reading device according to the second invention include, for example, the configuration of the character reading device 40 described in the embodiment of the first invention, the first dictionary 51 and the first reading device. There is a form in which the format recognition system 55 is removed, and the configuration of the second format recognition system 57 is configured as described below.
[0252]
That is, the second format recognition system 57 is a system that is activated when recognizing the format of a form to be recognized. The partial pattern cutout unit 57a of the second format recognition system 57 cuts out a pattern corresponding to a characteristic part of the existing form from the image data of the form to be recognized. When there are a plurality of existing forms, corresponding portions are cut out from the image data of the form to be recognized as characteristic parts for each of the plurality of existing forms. Position information for extracting a pattern corresponding to a characteristic part of an existing form from image data of the form to be recognized is preferably stored in a dictionary in advance. In addition, the individual format recognition unit 57c extracts a feature from the cut pattern, compares the feature with a feature in the individual format recognition dictionary 53, and selects one or a plurality of formats of the form to be recognized. And
[0253]
With this configuration, a format recognition process that focuses on a characteristic part of the form and a character reading process based on the recognition result can be performed on the form to be recognized.
[0254]
[Table 1]

[0255]
[Table 2]

[0256]
[Table 3]

[0257]
【The invention's effect】
As is clear from the above description, according to the format recognition apparatus of the first invention of the present application, the features extracted from the entire form can be checked against the features of the existing form to recognize the format of the form. Can be compared with some features of existing forms to recognize the format of the form.It is preferable that the first format recognition system is a recognition system that extracts features of ruled line patterns of the entire form. In addition, it is preferable to use a recognition system that uses feature data based on ruled line patterns of the existing form as the entire feature data of the existing form. Because, in general, a form includes ruled lines. Furthermore, the ruled lines are salient parameters that characterize the form structure. Therefore, if the first-stage format recognition is performed by paying attention to the ruled line pattern, the format of the form can be recognized with considerable accuracy.
[0258]
Further, according to the format recognition apparatus of the second invention of this application, a part of the form is compared with a part of the existing form to recognize the form of the form.
[0259]
Therefore, according to the format recognition devices of the first and second inventions, the format of the form can be recognized only by pattern matching. Therefore, even for a form without a form type number, its format can be recognized without including a character cutout process.
[0260]
Further, in each invention of the character reading device, the format recognition device described above, a read control data memory that outputs read control data according to a recognition result of the format recognition device, and recognizes characters based on the read control data. It has a character recognition system. Therefore, the character to be recognized can be recognized based on the format recognized by the format recognition device.
[Brief description of the drawings]
FIG. 1 is a first example of a form used in the first invention.
FIG. 2 is a second example of a form used in the first invention.
FIG. 3 is an explanatory diagram of the character reading device according to the first embodiment of the first invention.
FIG. 4 is a diagram illustrating a configuration example of an individual format recognition unit according to the first embodiment of the first invention.
FIG. 5 is an operation explanatory diagram of the character reading device of the first invention.
FIG. 6 is a diagram illustrating an operation example of a ruled line extraction unit.
FIG. 7 is a diagram illustrating an example of extracting a ruled line pattern.
FIG. 8 is an explanatory diagram of feature extraction in individual format recognition.
FIG. 9 is a diagram illustrating a configuration example of an individual format recognition unit according to the second embodiment of the first invention.
FIG. 10 is an explanatory diagram of the second embodiment of the first invention, and is an explanatory diagram of features to be extracted;
FIG. 11 is a third example of a form used in the first invention.
FIG. 12 is an explanatory diagram of the third embodiment of the first invention, and is an explanatory diagram of the character reading device of the third embodiment.
FIG. 13 is an explanatory diagram particularly illustrating a character recognition unit of the character reading device according to the third embodiment of the first invention.
FIG. 14 is an operation explanatory diagram of the character reading device according to the third embodiment of the first invention.
FIG. 15 is an explanatory diagram of still another embodiment of the character reading device of the first invention.
FIG. 16 is an explanatory diagram of a conventional technique and a problem.
[Explanation of symbols]
15, 17, 21, 23, 31, 33: Form (existing form)
19, 25, 35: Form to be recognized
T: Characteristic part of each form (individual format recognition area)
L1: Ruled line (characteristic part of ruled line)
a: space
40, 140: character reading device
50, 150: Format recognition device
51: First dictionary
53,153: Second dictionary
55: First format recognition system
57, 157: Second format recognition system
60: Read control data memory
70: Character recognition system
80: control unit
91: Display section
93: Input section
95: Scanner
97: Form image memory

Claims (21)

In a device that recognizes the format of a form,
A first format recognition system that extracts the entire feature of the form to be recognized, compares the feature with the entire feature data of the existing form, and recognizes the format of the form to be recognized;
A second format recognition system for extracting a part of the form of the form to be recognized, collating the characteristic with a part of characteristic data of the existing form, and recognizing the format of the form to be recognized. ,
The first format recognition system is a recognition system that extracts a feature of a ruled line pattern from a form to be recognized and uses feature data of a ruled line pattern of the existing form as feature data of the entire existing form. format recognition apparatus according to claim <br/> that. The format recognition device according to claim 1,
A first dictionary holding the entire feature data of the existing form;
A format recognition device, comprising: a second dictionary that holds some features of the existing form. The format recognition device according to claim 1,
The second format recognition system is
A recognition system that extracts features of a predetermined number or more of partial patterns from the form to be recognized, and uses feature data of a predetermined number or more of partial patterns of the form as feature data of a part of the existing form. A format recognition device, comprising: The format recognition device according to claim 1,
The format recognition apparatus according to claim 2, wherein the second format recognition system is a recognition system that operates when the first format recognition system outputs a predetermined number or more of candidates. The format recognition device according to claim 1,
The second format recognition system includes:
When said first format recognition system outputs the candidate less than predetermined number, and some of the features of the form of the recognition Target, matching a part of the feature data of each of all of the form of the existing form To recognize the format of the form to be recognized,
When said first format recognition system has output more candidate predetermined number, and the matching with the some of the features of some of the characteristic data and the recognition subject to form for each form that is the output, A format recognition apparatus, which is a recognition system that recognizes a format of the form to be recognized. The format recognition device according to claim 1,
A format recognition apparatus, wherein a plurality of the forms are set in one form. The format recognition device according to claim 1,
The first format recognition system includes:
The image data of the form to be recognized is scanned vertically and horizontally to extract ruled line patterns, features are extracted from the extracted ruled line patterns, and the extracted features are compared with feature data in a dictionary and formatted. A format recognition apparatus characterized by being a system for recognizing a character. The format recognition device according to claim 1,
A part of the form is a part of the form (including a character pattern) in the form,
The second format recognition system extracts a part of the graphic pattern from the image data of the form to be recognized, extracts a feature of the extracted graphic pattern, and compares the feature with feature data in a dictionary. A format recognition apparatus characterized by being a system for recognizing a format by using a format. The format recognition device according to claim 1,
A part of the form is a part of a ruled line in the form,
The second format recognition system extracts a part of the ruled line pattern from the image data of the form to be recognized, extracts a feature of the extracted ruled line pattern, and compares the feature with feature data in a dictionary. A format recognition apparatus characterized by being a system for recognizing a format by using a format. The format recognition device according to claim 1,
A part of the form is a character in the form,
The second format recognition system cuts out the pattern of the character from the image data of the form to be recognized, performs character recognition to obtain a character name, and determines the character name in advance as characteristic data for each existing form. A format recognizing device, which is a system for recognizing a format by collating with a character name. The format recognition device according to claim 1,
The format recognition apparatus according to claim 2, wherein the second format recognition system is a recognition system that changes the order of recognition candidates based on a result of the comparison. The format recognition device according to claim 1,
The second format recognition system includes:
As a result of the collation, of the existing forms, the smaller the distance to the form to be recognized or the higher the degree of certainty the higher the degree of certainty, the higher the candidate,
In a case where a plurality of places are set as a part of the form in one form and the candidate form is compared with the form, a candidate form in which the distance as a result of the comparison is equal to or less than a threshold value or a certainty degree is greater than or equal to a threshold value is higher in a candidate form. A format recognition apparatus characterized in that the recognition system replaces candidates or assigns priorities to the plurality of places in advance, and replaces a candidate form with a better recognition result for a place with a higher priority with a higher candidate. The format recognition device according to claim 1,
The second format recognition system previously holds position information of an area where the partial feature data is obtained on the existing form, and is specified by the position information of the form to be recognized. A format recognition device, which is a recognition system for extracting a feature from a part. The format recognition device according to claim 13 ,
3. The format recognition device according to claim 2, wherein the position information is held in a second dictionary according to claim 2. The format recognition device according to claim 1,
The first format recognition system includes:
A first condition (N is an arbitrary integer of 1 or more) in which the forms up to the Nth place in the comparison result are set as candidate forms;
A second condition that a form having a certainty factor or more as a collation result or a value having a distance or less is set as a candidate form, and
When the matching result of the M-th form is a candidate form, the difference between the distance or certainty as the matching result of the M-th form and the distance or certainty of the M + 1-th form is smaller than a certain value. and when a third condition that the form of the until the M + 1 of a candidate document (M being any integer not less than 1)
A format recognition apparatus characterized in that the format recognition apparatus is a recognition system that determines a candidate form based on a combination of one or more of the above conditions. The format recognition device according to claim 8 ,
The second format recognition system includes:
The extracted figure pattern is divided into M × N, and line drawing elements having threshold values or more that are determined to be figures in the horizontal, vertical, diagonally left and diagonally right directions of each of the divided areas are classified into a horizontal sub pattern and a vertical sub pattern. , Extracted as a left diagonal direction sub-pattern and a right diagonal direction sub-pattern, and creates a feature matrix for each of these direction sub-patterns.
A format recognition apparatus characterized in that the system recognizes the format by comparing the feature matrix with a feature matrix in a dictionary (where M and N are positive integers, and may be the same or different from each other) .). The format recognition device according to claim 9 ,
The second format recognition system includes:
The extracted ruled line pattern is divided into M × N, and line drawing elements that are determined to be ruled lines in the horizontal and vertical directions of each of the divided regions are extracted as horizontal and vertical sub-patterns, respectively. Create a feature matrix for each direction sub-pattern,
A format recognition apparatus characterized in that the system recognizes the format by comparing the feature matrix with a feature matrix in a dictionary (where M and N are positive integers and may be the same or different from each other). good.). The format recognition device according to claim 10 ,
The second format recognition system includes:
The cut out character pattern is divided into M × N, horizontal, vertical, left diagonal and right diagonal sub-patterns are extracted from each of the divided areas, and a feature matrix is created for each of these sub-patterns.
The feature matrix is compared with a feature matrix in a dictionary to obtain the character name,
A format recognition apparatus characterized in that the character name is compared with a dictionary to recognize a format (however, M and N are positive integers, and may be the same or different). The format recognition device according to claim 1,
A display unit for displaying one or both of a candidate form output by the first format recognition system and a candidate form output by the second format recognition system;
A format recognition apparatus comprising: an input unit for inputting an instruction for selecting one or a plurality of forms from the forms displayed on the display unit, or an instruction for not applicable. A format recognition device according to any one of claims 1 to 19 ,
A read control data memory that holds read control data necessary to read characters from the form for each existing form and outputs one or a plurality of read control data according to the recognition result of the format recognition device;
A character recognition system for recognizing a character from the form to be recognized based on the read control data output from the read control data memory. The character reading device according to claim 20 ,
When the second format recognition system of the format recognition device is a recognition system that performs format recognition by focusing on some characters of a form,
A character extraction unit for extracting a character pattern from image data of a form to be recognized,
A character pattern memory for storing the extracted characters,
Character recognition dictionary, and
A character recognition unit that recognizes a character by comparing the cut-out character pattern with data in the character recognition dictionary,
A character reading device comprising a character cutout unit, a character pattern memory, a character recognition dictionary, and a character recognition unit, which are used in the second format recognition system and the character recognition system.

Images