JP6138038B2 - Form identification device and form identification method - Google Patents

Description

  The present invention relates to a technique for analyzing form image data and identifying a form.

Conventionally, a technique for identifying a form by using a feature point in the form has been disclosed. FIG. 14 is a diagram illustrating a processing example of a conventional form recognition apparatus. FIG. 14A shows an input image 90 input to the form recognition apparatus, and FIG. 14B shows a sample image 91 provided in advance.
First, the form identifying device detects a margin for the binarized input image 90 and extracts a marker region 90a from the detected margin. Using the extracted marker area 90a as a feature point of the input image 90, a corresponding point with the marker area 91a of the sample image 91 shown in FIG. Based on the search result, the difference between the input image 90 and the sample image 91 is calculated, the ID area 90b of the input image 90 is calculated from the ID area 91b of the sample image, and the form ID indicated by the input image 90 is specified.

  As a method for searching for a corresponding marker region 91a from a sample image 91 based on the marker region 90a extracted from the input image 90, for example, Patent Document 1 discloses a marker region on a sample image and a marker region extracted from the input image. A method is disclosed in which all coordinates are compared and a marker region that is more approximate between the sample image and the input image is used as a search result.

JP 2001-220341 A

  However, in the technique disclosed in Patent Document 1 described above, the position of the marker area extracted from the input image needs to match the position of the marker area of the sample image, and the marker area is shifted for each form. In this case, or when it is difficult to extract the marker area due to noise or the like, there is a problem that the form identifier cannot be read and the analysis accuracy is lowered. Furthermore, it is necessary to arrange a plurality of marker areas on the form, and there is a problem that the degree of freedom of form composition is limited.

  The present invention has been made to solve the above-described problem. Even when the marker area is shifted for each form or when it is difficult to extract the marker area from the image data, An object is to recognize a form identifier and identify a form without restricting the configuration.

The form identification device according to the present invention is an area for generating a histogram indicating the frequency of occurrence of black pixels in form image data based on an area set in a sample image stored in advance and an image size of the form image data. A character area extraction unit that generates a histogram from image data of a form within the set area and extracts a character area based on the generated histogram, and a character included in the character area extracted by the character area extraction unit The character recognition unit for recognizing the column, the form recognition information stored in association with the character string and the form identifier in advance, the form identifier corresponding to the character string recognized by the character recognition unit is obtained, and the form is An identifier recognizing unit for identifying.

  According to the present invention, even when the marker area is shifted for each form or when it is difficult to extract the marker area from the image data, the form identifier can be recognized to identify the form. Moreover, it can suppress that the structure of a form is restrict | limited by a marker area | region.

1 is a block diagram illustrating a configuration of a form identification device according to Embodiment 1. FIG. 6 is a diagram illustrating affine transformation by an image correction unit of the form identification device according to Embodiment 1. FIG. FIG. 10 is a diagram illustrating histogram generation by a character region extraction unit of the form identification device according to the first embodiment. It is a figure which shows an example of the ID recognition information memorize | stored in the ID recognition information storage part of the form identification device by Embodiment 1. FIG. 5 is a flowchart showing the operation of the form identification device according to the first embodiment. FIG. 10 is a diagram illustrating calculation of a tilt angle of a ruled line of an image correction unit of the form identification device according to the first embodiment. It is a figure which shows the histogram generation position setting of the character area extraction part of the form identification device by Embodiment 1. FIG. It is a figure which shows extraction of the character area of the character area extraction part of the form identification device by Embodiment 1. FIG. FIG. 10 is a block diagram illustrating a configuration of a form identification device according to a second embodiment. FIG. 10 is a diagram illustrating comparison of histograms by an ID region detection unit of the form identification device according to the second embodiment. It is a figure which shows an example of ID which the ID recognition part of the form identification device by Embodiment 2 reads. 10 is a flowchart illustrating an operation of the form identification device according to the second embodiment. It is a figure which shows an example which detects position shift using a character area in the ID area | region detection part of the form identification device of Embodiment 2. FIG. It is a figure which shows the process example of the conventional form identification device.

Embodiment 1 FIG.
In the first embodiment, a character area included in input image data is extracted as an area for identifying a form (hereinafter referred to as a form identifier area), and the form is identified using the extracted form identifier area. The process of performing will be described.
FIG. 1 is a block diagram illustrating a configuration of a form identification apparatus according to the first embodiment.
The form identification device 10 includes a binarization processing unit 1, an image correction unit 2, a character region extraction unit 3, a character recognition unit 4, an ID recognition information storage unit 5, and an ID recognition unit (identifier recognition unit) 6. .
The binarization processing unit 1 performs binarization processing on the input image data. The image correction unit 2 extracts ruled line information from the binarized image and calculates the inclination of the extracted ruled line. Based on the calculated inclination of the ruled line, the inclination of the image is corrected by affine transformation.

FIG. 2 is a diagram illustrating affine transformation performed by the image correction unit of the form identification device according to the first embodiment. FIG. 2A shows a binarized image before tilt correction, and FIG. 2B shows a binarized image after tilt correction.
The image correction unit 2 extracts ruled lines 11a, 11b, 11c, 11d, 11e, 11f, 11g, and 11h from the binarized image shown in FIG. 2A, and detects the inclination of each extracted ruled line. . Based on the detected inclination of each ruled line, affine transformation for correcting the binarized image in the direction of the arrow A is performed to obtain the binarized image after the inclination correction shown in FIG. Details of inclination detection and correction based on inclination will be described later.

  The character region extraction unit 3 generates a histogram indicating the frequency of occurrence of black pixels from the corrected image corrected by the image correction unit 2, and extracts a character region based on the distribution of the generated histogram. If character region extraction processing is performed on all image data of a form, it is easily affected by ruled lines and noise in the form. Therefore, by setting a target area to be subjected to character area extraction processing in advance and generating a histogram in the set target area, it is possible to improve the analysis accuracy and reduce the amount of calculation. As a method for setting the target area, for example, a method in which the target area is set in advance on a sample image to be compared and enlarged or reduced based on the input image size of the form can be applied.

FIG. 3 is a diagram illustrating histogram generation by the character region extraction unit of the form identification device according to the first embodiment.
A target area 12a is set in the corrected image 12 shown in FIG. 3, and a histogram 12b is generated based on the number of black pixels in the X direction and Y direction of the target area 12a. The generated histogram 12b is normalized with the image size of the corrected image 12.

  The character recognition unit 4 recognizes a character string included in the character region extracted by the character region extraction unit 3. The ID recognition information storage unit 5 is a storage area for storing ID recognition information in which a character string and a form ID are associated with each other. FIG. 4 is a diagram illustrating an example of ID recognition information stored in the ID recognition information storage unit of the form identification device according to the first embodiment. The ID recognition unit 6 refers to the ID recognition information (identifier recognition information) stored in the ID recognition information storage unit 5, acquires the ID (identifier) of the form from the character string recognized by the character recognition unit 4, and is inputted. Identify the form.

Next, the operation of the form identification device 10 will be described.
FIG. 5 is a flowchart showing the operation of the form identification apparatus according to the first embodiment.
The binarization processing unit 1 performs binarization processing of image data (step ST1). The image correction unit 2 extracts ruled lines necessary for image correction from the binary image data that has been binarized in step ST1 (step ST2). As a ruled line extraction method, for example, the method of Reference 1 shown below can be applied.
・ Reference 1
Takashi Hirano, Yasuhiro Okada, Fumio Yoda, "Rule Extraction Method from Document Images", IEICE General Conference, Mar.1998

Next, the image correction unit 2 calculates the slope of the ruled line extracted in step ST2 (step ST3). The process of step ST3 will be described in more detail with reference to FIG.
FIG. 6 is a diagram illustrating calculation of the inclination angle of the ruled line of the image correction unit of the form identification device according to the first embodiment. 6A shows an example of the extracted ruled line, FIG. 6B is an explanatory diagram showing calculation of the tilt angle of the ruled line in the X direction, and FIG. 6C shows the tilt angle of the ruled line in the Y direction. It is explanatory drawing which shows calculation.
6A, the line segment 13a and the line segment 13b in the image 13 are ruled lines in the X direction, and the line segment 13c and the line segment 13d are ruled lines in the Y direction. As shown in FIG. 6B, the angle θ _t formed by the line segment 13a and the line segment 13b and the line segment 13x parallel to the X direction of the image 13 is calculated as the inclination angle θ _t of the ruled lines 13a and 13b. Further, as shown in FIG. 6C, an angle θ _t formed by the line segment 13c and the line segment 13d and the line segment 13y parallel to the Y direction of the image 13 is calculated as the inclination angle θ _t of the ruled line. The ruled line inclination calculation processing is performed on all ruled lines extracted in step ST2.

すなわち、全ての罫線の傾きの平均値を入力された画像の傾きとする。 Next, the image correcting unit 2 calculates cos θ and sin θ represented by the following equation (1) using the inclination angles θ _t of all ruled lines calculated in step ST3 (step ST4).

That is, the average value of the inclinations of all ruled lines is set as the inclination of the input image.

式（２）において、（ｘ´，ｙ´）は補正後の画像位置を、（ｘ，ｙ）は補正前の画像位置を示す。 Further, the image correction unit 2 performs affine transformation represented by the following equation (2) using cos θ and sin θ calculated in step ST4 to perform image correction (step ST5).

In Expression (2), (x ′, y ′) represents the image position after correction, and (x, y) represents the image position before correction.

Next, the character region extraction unit 3 sets an image position for generating a histogram for the corrected image corrected in step ST5 (step ST6). The process of step ST6 will be described in more detail with reference to FIG.
FIG. 7 is a diagram showing a histogram generation position setting of the character region extraction unit of the form identification device according to the first embodiment. FIG. 7A shows a corrected image 14 input from the image correction unit 2, and FIG. 7B shows a sample image 15 in which an image position for generating a histogram is set in advance.
In FIG. 7B, the histogram is defined when the base point O of the histogram generation region 16 set in the sample image 15 is (x, y), the horizontal length is w, and the vertical length is h. Is represented by (hx, hy), the horizontal length of the histogram is hw, and the vertical length of the histogram is hh. The length of the sample image 15 in the horizontal direction is represented by fw, and the length of the sample image 15 in the vertical direction. As shown in FIG. 7A, the horizontal length of the corrected image 14 is represented by iw, and the vertical length of the corrected image 14 is represented by ih.

Using the base point O of the histogram generation region 16 described above, the horizontal and vertical lengths of the sample image 15, and the horizontal and vertical lengths of the corrected image 14, the image position (hx, hy) of the histogram, The horizontal length hw of the histogram and the vertical length hh of the histogram are expressed by the following formula (3).

式（４）において、ｈ（ｘ）はヒストグラムの横方向の黒画素数を示し、ｈ（ｙ）はヒストグラムの縦方向の黒画素数を示す。 Next, the character area extraction unit 3 generates a histogram indicating the occurrence frequency of black pixels in the histogram generation area set in the process of step ST6 (step ST7). The histogram is generated according to the following equation (4).

In Expression (4), h (x) indicates the number of black pixels in the horizontal direction of the histogram, and h (y) indicates the number of black pixels in the vertical direction of the histogram.

式（５）では、ヒストグラムの縦方向の黒画素数ｈ（ｙ）が閾値ＴＨ_histより大きく、ヒストグラムの横方向の幅Ｗが閾値ＴＨＷより大きいことを条件としている。 Further, the character area extraction unit 3 extracts a character area from the histogram generated in step ST7 (step ST8). In the extraction process of step ST8, the character position and the noise or line segment are discriminated based on whether or not the histogram distribution has a certain width as shown in FIG. Therefore, a histogram having a condition represented by the following expression (5) is detected as a character region.

In Expression (5), the number of black pixels h (y) in the vertical direction of the histogram is larger than the threshold value TH _{hist and} the width W in the horizontal direction of the histogram is larger than the threshold value THW.

In the example of FIG. 8, a histogram of noise or line segment 17a is indicated by area 17b, and a histogram of character area 17c is indicated by area 17d. The region 17b is determined to be noise or a line segment because the distribution width Wa is equal to or less than the threshold value THW. On the other hand, the region 17d is detected as a character region because the distribution width Wb is larger than the threshold value THW and has a certain width.
Note that the condition is not limited to the condition of the above-described formula (5), but the condition may be that the number of black pixels h (x) in the horizontal direction of the histogram is larger than the threshold value and the vertical width of the histogram is larger than the threshold value.

The character recognition unit 4 recognizes the character string included in the character area extracted in step ST8, and outputs the recognition result to the ID recognition unit 6 (step ST9). As a character recognition method, for example, the method disclosed in Reference Document 2 below can be applied.
・ Reference 2
Satoshi Mori, Minako Sawaki, Norihiro Suda, Hiroshi Murase, Naoki Takegawa, “Robust Feature Extraction for Image Degradation Using Run-Length Correction”, IEICE Transactions D Vol J86-D2 No.7, pp.1049- 1057, July. 2003.

  The ID recognition unit 6 refers to the ID recognition information stored in the ID recognition information storage unit 5, acquires an ID corresponding to the character string recognized in step ST9, and identifies the form input to the form identification device 10. (Step ST10), the process ends.

  As described above, according to the first embodiment, the character area extracting unit 3 that extracts the character area from the input image data, and the character recognizing unit 4 that recognizes the character string included in the extracted character area. Since the ID recognition unit 6 that acquires the ID corresponding to the character string recognized with reference to the ID recognition information and identifies the form input to the apparatus is provided, it is included in the input image data. The form can be identified using the character area as a form identifier area for recognizing the form. Thereby, even when it is difficult to extract the marker area, the form can be identified. In addition, it is not necessary to arrange a plurality of marker areas in the form, and the degree of freedom of composition of the form can be increased.

  In addition, according to the first embodiment, since the target region for performing the character region extraction process is set in advance, and the histogram is generated in the target region in which the character region extraction unit 3 is set, the character region In the extraction process, the influence of ruled lines and noise in the form can be suppressed, and the analysis accuracy of the form identifying apparatus can be improved and the amount of calculation can be reduced.

Embodiment 2. FIG.
In the second embodiment, in addition to the extraction of the character area shown in the first embodiment, a marker provided in the form in advance for recognizing the form is extracted, and the extracted character area or marker is used for the form. The structure which detects an identifier area | region and identifies a form is shown.
FIG. 9 is a block diagram illustrating a configuration of the form identification apparatus according to the second embodiment.
The form identification device 20 of the second embodiment is provided by adding a marker extraction unit 7 to the form identification device 10 of the first embodiment shown in FIG. 1 and replacing the character recognition unit 4 with an ID region detection unit (identifier region). Detection unit) 8 is provided. In the following description, the same or corresponding parts as the components of the form identification device 10 according to the first embodiment are denoted by the same reference numerals as those used in FIG. 1 and the description thereof is omitted or simplified.

  The marker extraction unit 7 extracts a marker provided in the form in advance from the corrected image corrected by the image correction unit 2 and extracts the position of the marker. The ID region detection unit 8 compares the histogram generated by the character region extraction unit 3 with the histogram in the sample image stored in advance, and calculates the similarity α. Similarly, the marker position extracted by the marker extraction unit 7 is compared with the marker position in the sample image to calculate the similarity β. As the similarity, for example, the sum of the distances between corresponding points may be used.

Next, a histogram comparison method by the ID region detection unit 8 will be described with reference to FIG. FIG. 10 is a diagram illustrating comparison of histograms by the ID region detection unit of the form identification device according to the second embodiment. FIG. 10A shows a histogram generated by the character region extraction unit 3, and FIG. 10B shows a line segment generated by the ID region detection unit 8 based on the histogram.
For each of the histograms 21a, 21b, and 21c shown in FIG. 10A, the ID region detection unit 8 calculates the peak position and the size of the histogram. The calculated peak position and the size of the histogram are represented by line segments 21d, 21e, and 21f as shown in FIG. Histogram similarity α is calculated by comparing the line segments 21d, 21e, and 21f with the histogram line segments in the sample image.

  In general, in an image received by a scanner or FAX, extracted character areas, markers, and the like are rarely present at completely different positions. Therefore, the ID region detection unit 8 calculates the similarity α of the histogram and the similarity β of the marker position based on the difference in the arrangement position of the character region and the marker between the corrected image of the form and the sample image stored in advance. The calculated similarity α and similarity β are compared, and it is determined that a character region or a marker having a higher similarity (having more approximate position information) or a marker can be detected from the input image more accurately. It is used for detection of (identifier area).

The ID recognizing unit (identifier recognizing unit) 6 ′ acquires the ID (identifier) described in the ID area detected by the ID area detecting unit 8 and identifies the input form.
FIG. 11 is a diagram illustrating an example of an ID read by the ID recognition unit of the form identification device according to the second embodiment. FIG. 11A shows a case where the ID is indicated by a number, and FIG. 11B shows a case where the ID is indicated by the number of bars.

Next, the operation of the form identification device 20 will be described.
FIG. 12 is a flowchart illustrating the operation of the form identification apparatus according to the second embodiment. In the following description, the same steps as those in the form identification apparatus 10 according to the first embodiment are denoted by the same reference numerals as those used in FIG. 5, and description thereof is omitted or simplified.
When the image correction unit 2 performs image correction in step ST5, the character region extraction unit 3 performs the processing from step ST6 to step ST8 on the corrected image to generate a histogram and extract the character region. In parallel with the processing from step ST6 to step ST8, the marker extraction unit 7 extracts a marker and a marker position from the corrected image (step ST21).

  The ID region detection unit 8 calculates the similarity α between the histogram generated by the character region extraction unit 3 in steps ST6 to ST8 and the histogram of the sample image stored in advance (step ST22), and marker extraction in step ST21. The similarity β between the marker extracted by the unit 7 and the marker of the sample image is calculated (step ST23). Furthermore, the ID area detection unit 8 compares the similarity α calculated in steps ST22 and ST23 with the similarity β, and determines the character position (ctx, cty) of the input image using a histogram or marker having a higher similarity. (Step ST24).

The comparison process in step ST23 will be described in detail. When the similarity α is high (similarity α> similarity β), it is assumed that the registration has been correctly performed in the histogram, and the input image using the character area of the histogram is used. Character position (ctx, cty) is determined.
On the other hand, when the similarity β is high (similarity α <similarity β), the character position (ctx, cty) of the input image is determined using the marker position on the assumption that the marker has been accurately aligned.

式（６）において、（ｆｘ，ｆｙ）はサンプル画像上で位置合わせに用いる特徴点の位置を示し、（ｃｔｘ，ｃｔｙ）は入力画像の文字位置を示す。 Using the character position (ctx, cty) of the input image determined in step ST23, the positional deviation (cdx, cdy) between the input image and the sample image is detected by the following equation (6) (step ST25).

In Expression (6), (fx, fy) indicates the position of the feature point used for alignment on the sample image, and (ctx, cty) indicates the character position of the input image.

  FIG. 13 is a diagram illustrating an example in which a positional deviation is detected using a character area in the ID area detection unit of the form identification apparatus according to the second embodiment. FIG. 13A shows the input image 22, and FIG. 13B shows the sample image 23. FIG. 13A shows the character area 22a and the ID area 22b of the input image 22, and indicates that the character position of the character area 22a is (ctx, cty). FIG. 13B shows the character area 23a and the ID area 23b of the sample image 23, and shows that the position of the character area 23a is (fx, fy).

When there are a plurality of character areas in the input image, the average value of the character positions of the plurality of character areas is defined as a positional deviation between the input image and the sample image. In that case, the average value of the character positions is calculated based on the following equation (7).

The ID area detection unit 8 determines the upper left coordinates (sx, sy) of the ID area in the input image based on the following equation (8) using the deviation between the input image and the sample image detected in step ST25. (Step ST26).

式（９）において、ｓｗ，ｓｈはサンプル画像中のＩＤ領域の縦横の長さを示し、ＩＤｗ，ＩＤｈは入力画像中のＩＤ領域の縦横の長さを示す。 Further, the size of the ID area in the input image is determined based on the following equation (9) (step ST27).

In Expression (9), sw and sh indicate the vertical and horizontal lengths of the ID area in the sample image, and IDw and IDh indicate the vertical and horizontal lengths of the ID area in the input image.

The ID recognition unit 6 acquires the ID from the ID area in the input image whose size has been determined in step ST27, identifies the form input to the form identification device 20 (step ST28), and ends the process.
In addition, when ID of ID area | region is comprised with characters, such as a number, ID can be recognized by applying the method of the reference document 2 mentioned above, for example.

  In the above-described steps ST26 to ST28, the configuration in which the ID area is determined using the character area based on FIG. 13 is shown. However, when the marker can be accurately aligned, the ID is determined using the marker. Determine the area.

  As described above, according to the second embodiment, the character area extraction unit 3 that generates a histogram for a correction image and extracts a character area, and the marker extraction unit that extracts a marker for the correction image. 7 and an ID region detection unit for detecting an ID region using a feature point having a higher similarity by comparing the similarity between the extracted histogram and the histogram of the sample image and the similarity between the extracted marker and the marker of the sample image 8 and the ID recognizing unit 6 ′ that acquires the ID from the detected ID area and identifies the form, the detection accuracy of the ID area can be improved.

  In the second embodiment described above, the configuration in which the character position and the marker position are used for detection of the ID area by the ID area detection unit 8 is described. However, the marker position and the marker position are not limited to the character position and the marker position. An element whose position does not change in each form such as a ruled line position may be used as a feature point.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Binarization processing part, 2 Image correction part, 3 Character area extraction part, 4 Character recognition part, 5 ID recognition information storage part, 6, 6 'ID recognition part, 7 Marker extraction part, 8 ID area detection part, 10 , 20 Form identification device.

Claims (6)

In the form identification device for identifying the form from the image data of the form,
Based on the area set in the sample image stored in advance and the image size of the image data of the form, an area for generating a histogram indicating the occurrence frequency of black pixels is set in the image data of the form, and the set A character area extraction unit that generates the histogram from the image data of the form within the area, and extracts a character area based on the generated histogram ;
A character recognition unit for recognizing a character string included in the character region extracted by the character region extraction unit;
An identifier recognition unit for referring to the form recognition information stored in association with the character string and the form identifier in advance, acquiring the form identifier corresponding to the character string recognized by the character recognition unit, and identifying the form; A form identification device characterized by comprising: The form identification device according to claim 1 , wherein the character area extraction unit extracts the character area based on the number of black pixels of the generated histogram and the width of the generated histogram . In the form identification device for identifying the form from the image data of the form,
A feature point extracting unit that extracts feature points and position information of the feature points from the image data of the form;
A marker extracting unit that extracts marker and position information of the marker from the image data of the form;
The position information of the feature point extracted by the feature point extraction unit is compared with the position information of the feature point in the sample image stored in advance, and the position information of the marker extracted by the marker extraction unit and in the sample image An identifier area detection unit that detects an identifier area that includes the identifier of the form from the image data of the form using the feature point or the marker that has more approximate position information,
A form identification apparatus comprising: an identifier recognition unit that acquires an identifier of the form from the identifier area detected by the identifier area detection unit and identifies the form.   The form identifying apparatus according to claim 3, wherein the feature point extracting unit extracts a character area or a ruled line included in image data of the form as the feature point. In the form identification method for identifying the form from the image data of the form,
The character area extracting means generates a histogram indicating the occurrence frequency of black pixels in the image data of the form based on the area set in the sample image stored in advance and the image size of the image data of the form. Setting, generating the histogram from the image data of the form within the set area, and extracting a character area based on the generated histogram ;
A step of recognizing a character string included in the extracted character region;
The identifier recognition unit refers to the form recognition information stored in association with the character string and the form identifier in advance, acquires the form identifier corresponding to the recognized character string, and identifies the form; and A form identification method characterized by comprising. In the form identification method for identifying the form from the image data of the form,
A feature point extracting means for extracting a feature point and position information of the feature point from the image data of the form;
Marker extracting means for extracting a marker and position information of the marker from the image data of the form;
The identifier area detecting means compares the position information of the feature point with the position information of the feature point in the sample image stored in advance, and compares the position information of the marker with the position information of the marker in the sample image. Detecting an identifier area including the identifier of the form from the image data of the form using the feature point or the marker having more approximate position information;
A form identifying method comprising: an identifier recognizing unit that obtains an identifier of the form from the identifier area and identifies the form.

Images