JP4008715B2 - Form reading device and form reading processing program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a reading device for a form (for example, a deposit / withdrawal slip used in a bank) including a dropout color portion and a user entry portion related thereto, and a reading processing program for the form.
[0002]
In particular, a plain image or the like in which the red character / number entry frame portion (dropout color portion) is erased from the read image of the form is automatically selected, and R, G, B of the read image is also selected. For example, an AND operation process of each plane binary image and luminance signal binary image is generated to generate a composite image with the contents of the entry frame removed, and based on these selected images and generated images, user-entered letters and numbers And so on.
[0003]
[Prior art]
2. Description of the Related Art Conventionally, an optical character reader that removes a dropout color portion such as a character frame on a form by AND operation of R, G, and B plane binary images of the form is disclosed in, for example, Japanese Patent Laid-Open No. 6-243290. Is disclosed.
[0004]
In the case of this optical character reader, the dropout color portion of the read form is set as the background pixel (white pixel) by performing an AND operation for each bit of the R, G, B plane binary images. A composite image is generated in which the character / number portion is an information pixel (black pixel).
[0005]
[Problems to be solved by the invention]
In the case of such an optical character reading method, since it is assumed that AND operation of each plain binary image is performed, the processing efficiency for a single dropout color (for example, red) form is poor. There was a point.
[0006]
This is because all character frame portions (dropout color portions) are removed from a single dropout color plane image, and there is no particular advantage of AND operation between the image and another color plane image.
[0007]
Therefore, in the present invention, it is determined whether or not there is a single plane in each plane of the form image that can be said to be equivalent to all of the dropout color portions being removed. An object is to improve the efficiency of the character recognition image specifying process in which the dropout color is removed from the form image by selecting and using it for the subsequent character / number recognition processing.
[0008]
In the case where such a single plane does not exist, when a composite image of each plane binary image is generated, this composite is also performed by using the luminance signal binary image (Y signal binary image) of the reading form together. The purpose is to increase the resolution of the image.
[0009]
[Means for Solving the Problems]
The present invention solves this problem as follows.
(1) In a form reading device that optically reads a form including a dropout color part and a user entry part related thereto in pixel units.
Pixel density calculation means for obtaining a pixel density distribution for each plane of the read image of the form;
Based on the pixel density distribution for each plane, a threshold value calculation means for obtaining each threshold value for dropout color discrimination and binarization of the dropout plane for each pixel density distribution;
Plane selection means for selecting a dropout plane of the read image based on each threshold;
Based on the threshold corresponding to the selection, binarization means for binarizing the dropout plane according to the selection;
Is provided.
[0010]
According to the present invention, as described in (1) above, based on the threshold values for dropout color discrimination and dropout plane binarization of each pixel density distribution in each plane of the read image of the target form, that is, plane 2 A dropout plane is selected and binarized without performing a synthesis process such as an AND operation between value images. As a result, the form image generation process in which the dropout color is removed is efficiently executed.
[0011]
As will be described later in this specification, the dropout plane is selected based on the black pixel frequency or the white pixel frequency corresponding to each plane binary image of the read image of the target form as the technology related to the present invention of (1) above. It also shows what to do.
Since the sum of the black pixel frequency and the white pixel frequency in the plain binary image is 100%, any of the frequencies may be used as a condition for specifying the dropout plane. For example, the conditional expression for the black pixel frequency in steps (s12) and (s15) in FIG. 14 is also a conditional expression for the white pixel frequency by substituting “black pixel frequency = 100−white pixel frequency”.
[0012]
Here, by adjusting the threshold for binarization, the threshold for background separation of the local binarization method (FIGS. 17 to 19), the target range, etc., an appropriate black in the blurred portion of the letters and numbers entered by the user By using pixels, the accuracy of specifying the dropout plane is increased, and the recognition accuracy is improved by making the letters and numbers in the final output image thicker.
[0013]
Further, by using the black pixel frequency or the white pixel frequency of the composite binary image (the resolution is higher than that of the plane binary image itself) of the luminance signal binary image and the plain binary image of the read image, the dropout plane Increases specific accuracy.
[0015]
The present invention is directed to a form reading apparatus having such characteristics, and further efficiently executes a form image generation process in which the dropout color is removed, and makes the form image highly accurate. A form reading program is also targeted.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment of the present invention will be described with reference to FIGS.
[0017]
In the following description, for the sake of convenience of explanation, in principle, a red dropout color form is targeted, the size of the form is about A5 size, and the total number of pixels of the read image is 1,557,790 pixels.
[0018]
FIG. 1 is an explanatory diagram showing an outline (part 1) of the dropout plane detection method.
[0019]
In FIG.
11 is a red plane when the form is read with a two-dimensional color CCD,
12 is a histogram of this red plane multi-valued image,
13 is a green plane when the form is read with a two-dimensional color CCD,
14 is a histogram of this green plane multi-valued image,
15 is a blue plane when a form is read with a two-dimensional color CCD,
16 is a histogram of this blue plane multi-valued image,
17 is a red frame on the form (dropout part),
Respectively.
[0020]
The horizontal axes of the histograms 12, 14, and 16 are pixel densities (brightness) indicating 256 gradations from a black level of “0” to a white level of “255”, and the vertical axis corresponds to the number of pixels. ing. Further, the black border frame of each plain image is an image component of a table portion on which the form is placed when the form is read.
[0021]
A two-dimensional color CCD for reading a form is composed of a checkered pattern composed of photoelectric conversion elements for red, green and blue (see FIG. 15).
[0022]
The detection method of FIG. 1 pays attention to the difference in density distribution in the histogram of each plane image of red, green, and blue.
[0023]
The processing target form has, for example, a white background portion and a red frame 17 and the like, and the user enters black characters and numerical values in the red frame.
[0024]
The histogram of each plane 11, 13, 15 of red, green, blue (R, G, B) in this form image is
-In the case of a red plane, the peak distribution is 2 places and the automatic discrimination threshold is “187”.
In both of the green plane and the blue plane, the density distribution has three peak portions and the automatic determination threshold value is “152”.
[0025]
Here, the dropout type of the target form is automatically determined by specifying the plane having the maximum histogram automatic determination threshold. In the case shown in the figure, it is determined as a red dropout color form. Note that the method of obtaining the automatic discrimination threshold of the histogram itself is known (see FIG. 16).
[0026]
The automatic determination threshold (= 187) of the red plane 11 is different from the automatic determination threshold (= 152) of the green and blue planes 13 and 15, that is, the peak conditions of the histograms 12, 14, and 16 of each plane are different. This is because the reflectance of light with respect to a red portion such as the red frame 17 on the form is larger at the red wavelength than at the green wavelength or the blue wavelength.
[0027]
As a matter of course, the read output (density) of the form with respect to the red frame 17 is larger in the red photoelectric conversion element than in the other green and blue photoelectric conversion elements. The read output of the red photoelectric conversion element is substantially equal to that of the background portion (white portion) of the form.
[0028]
Therefore, red pixels (red frame etc.) drop out on the red plane of the read image of the form, and the density distribution of the red pixels in the histogram of the plane is merged with that of the background pixels (white pixels). Become a shape. Such a merge phenomenon does not occur in the green and blue planes in which the red frame remains in the read image.
[0029]
As a result, the automatic determination threshold value, which serves as a guide for the density distribution of each plane, is larger in the red plane than in the other green and blue planes.
[0030]
Of course, in the case of a blue dropout color form such as a withdrawal form, the density distribution of the blue frame is merged with the density distribution of the background pixel (white pixel) in the histogram of the blue plane.
[0031]
In this way, the dropout color type of the form is automatically determined based on the histogram of each plane of the form image. That is, it is determined that the read form in the case of FIG. 1 is a red dropout color. The value used for this automatic determination only needs to correspond to the contents of each plane histogram, and may be, for example, the average value of the pixel density distribution.
[0032]
FIG. 2 is an explanatory diagram showing an overall configuration corresponding to the detection method of FIG.
[0033]
In FIG.
21 is a two-dimensional color CCD having 400,000 pixels and a small capacity for optically reading, for example, an A5 size form.
22 performs pixel shift control for this two-dimensional color CCD (for example, a form image of a total of 3.6 million pixels is obtained based on nine pixel shift processes: see FIG. 15),
23 is a picked-up image synthesizing means for synthesizing, for example, nine images obtained by this pixel shifting process
24 is a shading correction means for equalizing the shade of the composite image;
25, R, G, B plane dividing means for dividing the read image after the shading correction into red, green and blue plane images;
26 to 28 are red, green and blue plane images held in a memory or work area (not shown),
29 is a Y signal image generating means for creating a Y signal image (luminance signal image) from each of the red, green and blue plane images;
30 is a Y signal image held in a memory or work area (not shown);
31 is a local binarization unit that binarizes this Y signal image by a local binarization method (see FIGS. 17 to 19) to create a binary Y image;
32 is a Y signal binary image held in a memory or work area (not shown),
Reference numerals 33 to 35 denote histogram calculation units that perform histogram calculation of red, blue, and green plane images (R image, G image, and B image),
36 to 38 are automatic discrimination threshold method processing units for obtaining a threshold for binarization from the histogram of each plane,
39 determines whether or not a single dropout color type can be specified based on the threshold value of each plane (see step (s6) in FIG. 14). If it can be specified, plane image determination for selecting that plane is performed.・ Selection part,
40 is a selected image binarization unit that binarizes each pixel signal of the selection plane with the automatic discrimination threshold;
41 is a selected plane binary image held in a memory or work area (not shown),
42 is an AND operation combining unit that performs an AND operation on the Y signal binary image 32 and the selected plane binary image 41;
43 is a composite image output unit for outputting the AND composite image,
44 is an integrated block comprising the components 33 to 39,
Respectively.
[0034]
Note that each of the local binarization unit 31 and the selected image binarization unit 40 has a character blur processing function described later (1.3 times the automatic discrimination threshold in FIG. 3, FIGS. 19 and 20, etc.). reference).
[0035]
3-5 is explanatory drawing which shows the outline | summary (the 2) of a dropout plane detection method. 3 shows the case of the R plane, FIG. 4 shows the case of the G plane, and FIG. 5 shows the case of the B plane.
[0036]
In FIGS. 3 to 5 and FIGS. 7 to 9 to be described later, only the substantially upper left 1/4 portion of the entire plane is shown in consideration of the display size of the handwritten character portion of each plane in the drawing.
[0037]
3 to 5,
17 is a red frame similar to FIG.
51 is a Y signal binary image obtained by the local binarization method (see FIGS. 17 to 19) of the luminance signal obtained from the red, blue, and green plain images of the reading form;
52 is a first red plane binary image obtained by binarizing the red plane of the read form with its automatic discrimination threshold (= 123),
53 is a second red plane binary image obtained by binarizing the red plane of the read form with a blur correction threshold (= 159: 1.3 times the automatic discrimination threshold);
54 is an AND composite image of the Y signal binary image 51 and the first red plane binary image 52 (black pixel frequency = 11.823%),
55 is an AND composite image (black pixel frequency = 12.030%) of the Y signal binary image 51 and the second red plane binary image 53;
62 is a first green plane binary image obtained by binarizing the green plane of the read form with its automatic discrimination threshold (= 103);
63 is a second green plane binary image obtained by binarizing the green plane of the read form with the threshold value for blurring correction (= 133: 1.3 times the automatic discrimination threshold value),
64 is an AND composite image of the Y signal binary image 51 and the first green plane binary image 62 (black pixel frequency = 13.653%),
65 is an AND composite image (black pixel frequency = 15.495%) of the Y signal binary image 51 and the second green plane binary image 63;
72 is a first blue plane binary image obtained by binarizing the blue plane of the read form with its automatic discrimination threshold (= 111);
73 is a second blue plane binary image obtained by binarizing the blue plane of the read form with a blur correction threshold (= 144: 1.3 times the automatic discrimination threshold);
74 is an AND composite image of the Y signal binary image 51 and the first blue plane binary image 72 (black pixel frequency = 13.787%),
75 is an AND composite image (black pixel frequency = 16.273%) of the Y signal binary image 51 and the second blue plane binary image 73;
Respectively. The black level is a logical value “1”.
[0038]
The detection method of FIGS. 3 to 5 is based on the difference in the black pixel frequency in each of the AND composite images of the binary image of each plane of the reading form and the Y signal binary image (luminance signal binary image) of each plane. It is the one that paid attention. These AND composite images are also images obtained by removing the noise portion of the binary image of each plane before composition. That is, the resolution is higher than that of the plain image before synthesis.
[0039]
Here, the AND composite images 54 and 55 of the red plane are binary images consisting only of characters and guide information with the dropout color (such as the red frame 17) of the read form almost disappearing.
[0040]
On the other hand, the AND composite images 64 and 65 for the green plane and the AND composite images 74 and 75 for the blue plane are images including a dropout color (such as the red frame 17) of the read form.
[0041]
It should be noted that the frequency of black pixels in each of the red, blue, and green plain binary images is increased by increasing the binarization threshold as shown in the figure. This is because the read density of a pixel corresponding to a thin line part such as a character, a red frame, or a faint part written on the form is small, and the pixel is processed as a white level at the time of binarization with a low threshold, that is, Because it will disappear.
[0042]
The black pixel frequency in the AND composite image 54, 55, 64, 65, 74, 75 of each plane is
(1) In the case of the red plane in FIG.
-11.823% (automatic discrimination threshold: 123)
・ 12.030% (threshold for blurring correction: 159 = 123 × 1.3)
(2) In the case of the green plane in FIG.
・ 13.653% (automatic discrimination threshold: 103)
・ 15.495% (threshold for blurring correction: 133 = 103 × 1.3)
(3) In the case of the blue plane in FIG.
13.787% (automatic discrimination threshold: 111)
16.273% (threshold correction threshold: 144 = 111 × 1.3)
Met. These numerical values are values in the entire image of each plane (not only the ¼ portion image shown in the figure).
[0043]
By determining and comparing the black pixel frequency in the AND composite image of each of the red, green, and blue planes, the plane with the lowest frequency (the red plane in the case of the illustrated form) is identified as the dropout plane. it can.
[0044]
The number of pixels corresponding to the black pixel frequency difference between the AND composite images of these planes is the above-mentioned A5 size and the overall reading form of 1,557,790 pixels,
(1) In case of red plane and green plane,
-28,507 pixels (1.83% of the total: when the automatic discrimination threshold)
-53,978 pixels (3.465% of the total: when the threshold value for blur correction)
(2) In case of red plane and blue plane,
・ 30,595 pixels (1.964% of the total: automatic discrimination threshold)
66,097 pixels (4.243% of the total: at the threshold value for blur correction)
(3) For the green and blue planes,
・ 2,088 pixels (0.134% of the total: when automatic discrimination threshold)
・ 12,119 pixels (0.778% of the total: at the threshold value for blur correction)
It becomes.
[0045]
By using the blur correction threshold value that is larger than the automatic discrimination threshold value in this way, the black pixel frequency difference between the AND composite image of the red plane and the other AND composite image is expanded. Therefore, the dropout plane selection process can be performed more reliably.
[0046]
Note that the dropout plane may be specified based on the black pixel frequency difference of each plane image itself instead of the black pixel frequency difference of the AND composite image.
[0047]
FIG. 6 is an explanatory diagram showing an overall configuration corresponding to the detection method of FIGS.
[0048]
In FIG.
21 to 38 are components having the same contents as the corresponding numbers in FIG.
81-83 are binarization units for binarizing each of the R image 26, G image 27, and B image 28 with the output values of the automatic discrimination threshold method processing units 36-38,
84 to 86 are an R binary image, a G binary image, and a B binary image held in a memory or work area (not shown),
87 is an AND operation combining unit that performs an AND operation on the Y signal binary image 32 and each of the R binary image 84, the G binary image 85, and the B binary image 86;
Reference numerals 88 to 90 denote YR composite binary images, YG composite binary images, and YB composite binary images held in a memory or work area (not shown).
91 obtains the black pixel frequency in the histogram of each composite binary image, and determines whether or not a single dropout color type can be specified based on the black pixel frequency (see step (s12) in FIG. 14). Black pixel frequency calculation / determination unit,
92 is a plane image selection / output unit that selects and outputs the plane image (red plane image in the case of illustration) identified by the black pixel frequency calculation / determination unit 91;
93 is an integrated block composed of the components 81 to 91;
Respectively.
[0049]
7 to 9 are explanatory views showing an outline (part 3) of the dropout plane detection method. 7 shows the case of the R plane, FIG. 8 shows the case of the G plane, and FIG. 9 shows the case of the B plane.
[0050]
7 to 9,
17 and 51 are components having the same contents as the corresponding numbers in FIGS.
52 ′ is a first red plane binary image obtained by binarizing the red plane of the read form by a local binarization method (see FIGS. 17 to 19) using the automatic discrimination threshold (= 123). ,
53 ′ is a second red plane binary image obtained by binarizing the red plane of the reading form by the local binarization method using the blur correction threshold (= 159);
56 is an XOR composite image of the Y signal binary image 51 and the first red plane binary image 52 ′ (black pixel frequency = 5.943%),
57 is an XOR composite image (black pixel frequency = 6.014%) of the Y signal binary image 51 and the second red plane binary image 53 ′;
62 ′ is a first green plane binary image obtained by binarizing the green plane of the read form by the local binarization method using the automatic discrimination threshold (= 103);
63 ′ denotes a second green plane binary image obtained by binarizing the green plane of the read form by the above-described local binarization method using the blur correction threshold (= 133);
66, an XOR composite image (black pixel frequency = 4.424%) of the Y signal binary image 51 and the first green plane binary image 62 ′;
67 is an XOR composite image (black pixel frequency = 3.577%) of the Y signal binary image 51 and the second green plane binary image 63 ′.
72 ′ is a first blue plane binary image obtained by binarizing the blue plane of the read form by the local binarization method using the automatic discrimination threshold (= 111);
73 ′ is a second blue plane binary image obtained by binarizing the blue plane of the read form by the above-described local binarization method using the blur correction threshold (= 144);
76 is an XOR composite image of the Y signal binary image 51 and the first blue plane binary image 72 ′ (black pixel frequency = 4.404%),
77 is an XOR composite image (black pixel frequency = 4.219%) of the Y signal binary image 51 and the second blue plane binary image 73 ′,
Respectively. The black level is a logical value “1”.
[0051]
The detection methods in FIGS. 7 to 9 are different in the black pixel frequency between the XOR composite images of the binary image of each plane of the reading form and the Y signal binary image (luminance signal image) obtained from each plane image. Is focused on.
[0052]
The black pixel frequencies of the XOR composite images 56, 57, 66, 67, 76, and 77 have an inverse relationship to each of the AND composite images in FIGS.
[0053]
That is, in the case of these XOR composite images, the black pixels of the Y signal binary image, and the white pixel portion (the pixel portion disappeared by the binarization process) of the red, blue, and green plain binary images are black pixels. On the other hand, the black pixel portion of the plane binary image (the pixel portion that has not been erased by the binarization process) is inverted to the white pixel.
[0054]
Therefore, the black pixel frequency of the XOR composite image of each plane is maximized in the red plane where the portion that disappears (becomes white) by the binarization processing of the plane image is larger than the other planes.
[0055]
The black pixel frequency in the XOR composite image 56, 57, 66, 67, 76, 77 of each plane is
(1) In the case of the red plane in FIG.
-5.943% (automatic discrimination threshold: 123)
6.014% (threshold for blurring correction: 159 = 123 × 1.3)
(2) In the case of the green plane in FIG.
・ 4.424% (automatic discrimination threshold: 103)
3.577% (threshold correction threshold: 133 = 103 × 1.3)
(2) In the case of the blue plane in FIG.
-4.404% (automatic discrimination threshold: 111)
4.219% (threshold correction threshold: 144 = 111 × 1.3)
Met. These numerical values are values in the entire image of each plane (not only the ¼ portion image shown in the figure).
[0056]
By obtaining and comparing the black pixel frequency in the XOR image of each of the red, blue, and green planes, the plane with the lowest frequency (in the case of the illustrated form, the red plane) can be specified as the dropout plane. .
[0057]
It should be noted that the number of pixels corresponding to the black pixel frequency difference between the XOR composite images of each plane is A5 size and the total number of pixels is 1,557,790 pixels as described above.
(1) In case of red plane and green plane,
23,662 pixels (1.519% of the total: when the automatic discrimination threshold)
37,963 pixels (2.437% of the total: when the threshold value for blur correction)
(2) In case of red plane and blue plane,
23,974 pixels (1.539% of the total: when the automatic discrimination threshold)
27,962 pixels (1.795% of the total: at the threshold value for blur correction)
(3) For the green and blue planes,
・ 311 pixels (0.02% of the total: when automatic discrimination threshold)
・ 10,001 pixels (0.642% of the whole: at the threshold value for blur correction)
It becomes.
[0058]
FIG. 10 is an explanatory diagram showing an overall configuration corresponding to the detection method of FIGS.
[0059]
In FIG.
21 to 38 are components having the same contents as the corresponding numbers in FIG. 2 and FIG.
Reference numerals 101 to 103 denote local binary images that create each plane binary image by binarizing each of the R image 26, the G image 27, and the B image 28 by a local binarization method (see FIGS. 17 to 19). ,
Reference numerals 104 to 106 denote an R binary image, a G binary image, and a B binary image held in a memory or work area (not shown),
107 is an XOR operation combining unit that performs an XOR operation on the Y signal binary image 32 and each of the R binary image 104, the G binary image 105, and the B binary image 106;
108 to 110 are YR composite binary images, YG composite binary images, and YB composite binary images held in a memory or work area (not shown),
111 obtains the black pixel frequency in the histogram of each composite binary image, and determines whether or not a single dropout color type can be specified based on the black pixel frequency (see step (s13) in FIG. 14). Black pixel frequency calculation / determination unit,
112 is a plane image selection / output unit that selects and outputs the plane image (red plane image in the case of illustration) identified by the black pixel frequency calculation / determination unit 111;
Reference numeral 113 denotes an integrated block composed of the components 101 to 111.
Respectively.
[0060]
Note that the local binarization units 101 to 103 have the function of adjusting the local binarization target range in FIG.
[0061]
FIG. 11 is an explanatory diagram showing an example of a form having blue and red dropout colors.
121 is a blue dropout color seal frame,
122 is a red dropout color entry character frame,
Respectively.
[0062]
FIG. 12 is an explanatory diagram showing a composite image output method for the form of FIG.
[0063]
In FIG.
131 is a Y signal binary image generated by binarizing the Y signal image of the read form by a local binarization method;
Reference numeral 132 denotes an R-plane binary image generated by binarizing a red plane image of a read form with an automatic discrimination threshold;
133 is a YR composite binary image generated by AND operation synthesis of the Y signal binary image 131 and the R plane binary image 132;
134 is a B plane binary image generated by binarizing the blue plane image of the read form with an automatic discrimination threshold;
135 is a YRB composite binary image generated by AND operation synthesis of the YR composite binary image 133 and the B plane binary image 134;
Respectively. The black level is a logical value “1”.
[0064]
here,
-From the YR composite binary image 133, the red dropout color (entry text box 122) disappears.
From the YRB composite binary image 135, the blue dropout color (the seal frame 121) disappears in addition to the red dropout color.
[0065]
The YRB composite binary image 135 and the G-plane binary image (not shown) are also AND-combined. However, in the case of the read form shown in the drawing, the G color portion is not actively included. There is no particular change in the image due to this synthesis process.
[0066]
FIG. 13 is an explanatory diagram showing an overall configuration corresponding to the output method of FIG.
[0067]
In FIG.
21 to 32, 39, 44, 91, 93, 111 and 113 are components having the same contents as the corresponding numbers in FIG. 2, FIG. 6 and FIG.
141 is an RGB binary image selection unit that selects, for example, the R binary image 104, the G binary image 105, and the B binary image 106 one by one in order (arbitrary order) in the integrated block 113;
142 is an AND operation combining unit that performs an AND operation on the Y signal binary image 32 and, for example, the R binary image 104 selected by the RGB binary image selection unit 141;
143 is a YR composite binary image stored in a memory or work area (not shown),
144 is an AND operation combining unit that performs an AND operation on the YR combined binary image 143 and, for example, the G binary image 105 selected by the RGB binary image selecting unit 141;
145 is a YRG composite binary image held in a memory or work area (not shown),
Reference numeral 146 denotes an AND operation combining unit that performs an AND operation on the YRG combined binary image 145 and, for example, the B binary image 106 selected by the RGB binary image selecting unit 141.
147 is a YRGB composite binary image held in a memory or work area (not shown), 148 is a composite binary image output unit for outputting the YRGB composite binary image 147,
Respectively.
[0068]
FIG. 14 is an explanatory diagram showing a processing procedure corresponding to the overall configuration of FIG. 13, and the contents thereof are as follows.
(s1) The target form is read with a two-dimensional color CCD, and the process proceeds to the next step.
(s2) The read form is divided into RGB planes, and the process proceeds to the next step.
(s3) A Y signal binary image of the read form is generated based on each plane image, and the process proceeds to the next step.
(s4) The histogram calculation of each of the previous RGB plane images is executed, and the process proceeds to the next step.
(s5) The automatic threshold values (Th1 to Th3) in each plane histogram are calculated (see FIG. 16), and the process proceeds to the next step. Note that the calculated automatic threshold values themselves correspond to Th1 to Th3 in the form of Th1 ≧ Th2 ≧ Th3.
(s6) At each automatic threshold,
・ Th1-Th2 ≧ 30
・ Th2-Th3 ≦ 10
Whether or not both conditions are satisfied is determined. If “satisfy”, the process proceeds to the next step, and if not satisfied, the process proceeds to step (s9). The determination subject at this time is the plain image determination / selection unit 39.
(s7) The binarization of the plane image (red plane in the case of the form in FIG. 1) corresponding to the automatic threshold Th1 is executed, and the process proceeds to the next step.
(s8) An AND operation synthesis of this plain binary image and the previous Y signal binary image is executed, and the flow proceeds to step (s20).
(s9) The RGB plane images are binarized, and the process proceeds to the next step.
(s10) The AND operation synthesis of each plane binary image and the previous Y signal binary image is executed, and the process proceeds to the next step.
(s11) The black pixel frequencies (Bf1 to Bf3) in each composite image are calculated, and the process proceeds to the next step. The calculated black pixel frequencies themselves are associated with Bf1 to Bf3 in the form of Bf1 ≦ Bf2 ≦ Bf3. The black pixel frequency is obtained as a ratio of the number of black pixels to the total number of pixels of the composite image.
(s12) At each black pixel frequency,
・ Bf2-Bf1 ≧ 1.5%
・ Bf3-Bf2 ≦ 0.2%
If both conditions are satisfied, the process proceeds to step (s19). If not, the process proceeds to the next step. The determination subject at this time is the black pixel frequency calculation / determination unit 91.
(s13) The XOR operation synthesis of each plane binary image and the Y signal binary image is executed, and the process proceeds to the next step.
(s14) The black pixel frequency (Bf4 to Bf6) in each composite image is calculated, and the process proceeds to the next step. The calculated black pixel frequencies themselves are associated with Bf4 to Bf6 in the form of Bf4 ≧ Bf5 ≧ Bf6.
(s15) At each black pixel frequency,
・ Bf4-Bf5 ≧ 1.0%
・ Bf5-Bf6 ≦ 0.1%
If both conditions are satisfied, the process proceeds to step (s19). If not, the process proceeds to the next step. The determination subject at this time is the black pixel frequency calculation / determination unit 111.
(s16) The AND operation synthesis of the previous Y signal binary image and the R plane binary image is executed, and the process proceeds to the next step.
(s17) An AND operation synthesis of this YR composite binary image and the previous G plane binary image is executed, and the process proceeds to the next step.
(s18) An AND operation synthesis of this YRG composite binary image and the previous B-plane binary image is executed, and the flow proceeds to step (s20).
(s19) An image corresponding to the black pixel frequency Bf1 or Bf4 is selected, and the process proceeds to the next step. The corresponding image of Bf1 is a composite binary image (YR composite binary image in the case of the form of FIG. 3), and the corresponding image of Bf4 is a plain binary image (red plane binary image in the case of the form of FIG. 7). ).
(s20) The composite image in step (s18) and the selected image in step (s19) are output to a display screen (not shown).
[0069]
In FIG. 13 and FIG. 14, it is not necessary to execute all the block processes of the integrated blocks 44, 93 and 113, and only one or two arbitrary processes of the integrated block may be performed. Further, the execution order of the processes of the integrated blocks 44, 93, 113 is also arbitrary. For example, the order of the integrated blocks 93-113-44 may be used.
[0070]
Needless to say, the components of the integrated block change according to such a change in the processing mode. For example, when the integrated block 44 is omitted, histogram calculation units 33 to 35 and automatic discrimination threshold method processing units 36 to 38 are added as components of the integrated block 93.
[0071]
In the above form image processing,
・ Y signal binary image is generated mainly by local binarization method,
・ Red, blue, and green plain binary images are generated by the automatic discrimination threshold method and the local binarization method.
[0072]
FIGS. 15A and 15B are explanatory views showing an outline of the CCD and the pixel shifting method. FIG. 15A shows the arrangement of the photoelectric conversion elements of the CCD, and FIG. 15B shows how the imaging pixels are shifted.
[0073]
The photoelectric conversion unit for one pixel of the CCD includes one red conversion element R, one blue conversion element B, and two green conversion elements G as shown in the figure.
[0074]
When a form is read by a CCD in which 400,000 photoelectric conversion units (4 elements of RGGB) are arranged at a predetermined interval (two circle portions), first, the pixel signal of the portion (1) is obtained by the first imaging.
[0075]
Thereafter, the CCD or form is shifted eight times in the horizontal direction and the downward direction in the drawing, and the pixel signals of the respective parts (2) to (9) are obtained in order.
[0076]
An image signal of 3.6 million pixels (= 400,000 × 9) can be obtained by synthesizing the individual pixel signals individually obtained by the total of nine imaging operations.
[0077]
FIG. 16 is an explanatory diagram showing a processing procedure for obtaining an automatic discrimination threshold for a multi-value plane image (0 to 255 gradations), and the contents thereof are as follows.
(s31) The normalized histogram value pi at the density level i (0 ≦ i ≦ 255) of each gradation is calculated by “pi = ni / N”. Here, ni is the number of pixels of density level i, and N is the number of read pixels of the entire form.
(s32) The average density level μt of the normalized histogram is calculated by the equation shown in the figure.
(s33) The cumulative value ω (k) of pi and the cumulative value m (k) of “i * pi” from the gradation 0 to the gradation k (0 ≦ k ≦ 255) Calculate for each.
(s34) A variance value σ (k) corresponding to each gradation k (0 ≦ k ≦ 255) is calculated by the equation shown in the drawing using the accumulated values ω (k), m (k) and the average density level μt.
(s35) The gradation k at which the variance value σ (k) is maximum is set as the binarization threshold value of the multi-value plane image.
[0078]
FIG. 17 is an explanatory diagram showing an outline of the local binarization method.
[0079]
In FIG.
151 is a target pixel for local binarization processing,
152 is a 7 × 7 pixel rectangular filter for the target pixel;
153 is a character part stroke pixel in the rectangular filter,
154 is a background pixel in the rectangular filter,
σmin is a threshold for background separation in the rectangular filter (for example, 100),
E is the density average value of each pixel in the rectangular filter,
E2 is the mean square density value of each pixel in the rectangular filter,
g2 is a density dispersion value in the rectangular filter, which is obtained by the equation shown using the density average value E and the density mean square value E2.
S is a threshold value for binarization of the target pixel 151, which is obtained by the illustrated formula using the density average value E and the density variance value g2.
Respectively.
[0080]
Note that the binarization threshold S is used when it is determined that the rectangular filter 152 is not the background portion, that is, when g2> σmin is satisfied.
[0081]
In the local binarization method, a rectangular filter 152 that includes the binarization target pixel is set, and the density dispersion value g2 of the entire pixel in the pixel is obtained, and then the background separation threshold value is obtained. By examining the magnitude relationship with g2, it is determined whether the rectangular filter corresponds to the background portion or the information portion.
[0082]
When the background portion is determined, “white” and “black” of the target pixel 151 are specified based on the magnitude of the density dispersion value g2 (without directly using the density of the target pixel).
[0083]
On the other hand, when it is determined as the information portion, “white” and “black” are specified based on the density level of the target pixel 151.
[0084]
FIG. 18 is an explanatory diagram showing the processing procedure of the local binarization method, and the contents thereof are as follows.
(s41) The average density value E of the rectangular filter 152 is obtained.
(s42) The density mean square value E2 of the rectangular filter 152 is obtained.
(s43) The density dispersion value g2 (= E2-E * E) of the rectangular filter 152 is calculated.
(s44) In the case of “background separation threshold σmin <density variance value g2”, the rectangular filter 152 is determined as the background portion, and the process proceeds to the next step. Proceed to step (s47) by determining that it is a part. Σmin is an initial setting value (value is arbitrary), and is set to “100” here.
(s45) Threshold for binarization S (= E + (g2) ^1/2 * 0.1) is calculated. By reducing this multiplication coefficient, noise around the pixels around the character can be suppressed.
(s46) If “binarization threshold S <density of target pixel 151”, the process proceeds to step (s48), and if the magnitude relationship is not established, the process proceeds to step (s49).
(s47) If “reference value <density average value E”, the process proceeds to the next step, and if the magnitude relationship is not established, the process proceeds to step (s49). Note that what is used as the reference value is arbitrary. For example, an automatic determination threshold value of the target image may be used as the reference value.
(s48) The target pixel 151 is determined to be “white”.
(s49) The target pixel 151 is determined to be “black”.
[0085]
FIG. 19 is an explanatory diagram illustrating adjustment of a local binarization target range (a range estimated as a blurred character portion),
ot is an automatic discrimination threshold in the target image histogram,
a and b are coefficients for adjusting the local binarization target range,
b-ave is the density average of the black region (density 0 to ot),
w-ave is the density average of the white region (density ot + 1 to 255),
bt is the lower limit concentration of the local binarization target range,
wt is the upper limit concentration of the local binarization target range,
Respectively.
[0086]
Here, the local binarization target range (density bt to wt) in the target image histogram is calculated by the two equations shown in the figure. The local binarization target range can be arbitrarily set by adjusting the coefficients a and b in the formula. The adjustment of the coefficients a and b is performed by hardware operation or software execution in the local binarization unit.
[0087]
By executing the local binarization method only for pixels in such an estimated range (density bt to wt), a significantly high-speed process is possible.
[0088]
Each pixel outside the estimation range is uniformly determined to be “black (density 0 to bt)” or “white (density wt to 255)”. For blurring of letters and numbers in this white area, this pixel portion can be determined to be “black” by increasing wt. On the other hand, the processing time required for binarization of the entire image becomes longer as the estimated range is expanded.
[0089]
In this example, “a = 0.35, b = 0.62” is set, and when local binarization is performed on 5 to 8% of pixels in the entire image, all pixels are locally binarized. A binarized image of the same level as that obtained was obtained.
[0090]
FIG. 20 is an explanatory diagram illustrating an example of a change in a binary image due to blurring correction during local binarization. FIG. 20A illustrates a case where the background separation threshold in FIG. 17 is changed (σmin = 100 → 50). (B) shows a case where the local binarization target range of FIG. 19 is changed (a = 0.35, b = 0.62 → a = 0.35, b = 0.90).
[0091]
As a recording medium for storing the processing program for the above-described dropout plane detection and composite image output,
・ Program provider's database (DASD etc.)
・ Portable recording media in various formats
-RAM or hard disk on the computer body side
Any of these may be used. The program is loaded into the computer main body and executed on the main memory.
[0092]
(Supplementary Note 1) In a form reading device that optically reads a form including a dropout color part and a user-filled part related thereto in pixel units.
A density distribution calculating means for obtaining a pixel density distribution for each plane of the read image;
A plane selection unit that obtains a characteristic value of the pixel density distribution and identifies a dropout plane based on the characteristic value.
A form reader characterized by that.
(Supplementary Note 2) The density distribution calculating means uses an automatic determination threshold value of the pixel density distribution as the characteristic value.
The form reading device according to Supplementary Note 1, wherein
(Supplementary Note 3) In a form reading apparatus that optically reads a form including a dropout color part and a user entry part related to the dropout color part in pixel units.
Binarization means for binarizing each plane of the read image;
A plane selection unit that obtains a black pixel frequency or a white pixel frequency corresponding to the plane binary image and specifies a dropout plane based on the pixel frequency;
A form reader characterized by that.
(Additional remark 4) The said binarization means has a binarization process correction | amendment function for respond | corresponding to the blurred part of the character and number of a user entry,
The form reading apparatus according to Supplementary Note 3, wherein
(Additional remark 5) The said plane selection means is the pixel frequency of the synthetic | combination binary image produced | generated by the calculation process of the luminance signal binary image of the said read image, and the said plane binary image as said black pixel frequency or said white pixel frequency. Use
The form reading apparatus according to Supplementary Note 3, wherein
(Supplementary Note 6) In a form reading device that optically reads a form including a dropout color part and a user entry part related to the dropout color part in pixel units.
Image generating means for generating each plane binary image and luminance signal binary image of the read image;
A first combined binary image is generated by arithmetic processing of the luminance signal binary image and one of the plane binary images, and then the first combined binary image and the remaining of the plane binary image A second composite binary image is generated by arithmetic processing with one of the second composite binary image, and a third composite binary image is obtained by arithmetic processing of the second composite binary image and the remaining one of the plane binary images. Image synthesizing means for generating an image,
A form reader characterized by that.
(Supplementary note 7) The image generation means has a binarization processing correction function for dealing with a blurred portion of letters and numbers entered by a user.
The form reading apparatus according to Supplementary Note 6, wherein
(Supplementary note 8) In a program used in a reading process of a form including a dropout color part and a user entry part related thereto,
The program is
Obtain the characteristic value of the pixel density distribution of each plane of this form read image,
It is for causing a computer to realize a function of identifying a dropout plane based on the characteristic value.
A form reading processing program characterized by the above.
(Supplementary note 9) In a program used in the reading process of a form including a dropout color part and a user entry part related thereto,
The program is
Find the black pixel frequency or white pixel frequency corresponding to each plain binarized image of this form read image,
In order for the computer to realize the function of specifying the dropout plane based on the pixel frequency,
A form reading processing program characterized by the above.
(Supplementary Note 10) In a program used in a form reading process including a dropout color portion and a user entry portion related thereto,
The program is
A first composite binary image is generated by calculating the luminance signal binary image and one of the plane binary images of the form read image, and then the first composite binary image and the planes. A second composite binary image is generated by an arithmetic process with the remaining one of the binary images, and a second composite binary image is generated by an arithmetic process with the second composite binary image and the remaining one of the plane binary images. 3 for causing a computer to realize a function of generating a composite binary image 3.
A form reading processing program characterized by the above.
[0093]
【The invention's effect】
As described above, the present invention is based on the characteristic value of the pixel density distribution in each plane of the read image of the target form (for example, automatic discrimination threshold) and the black pixel frequency or white pixel frequency corresponding to each plane binary image. That is, since the dropout plane is specified without performing a synthesis process such as an AND operation between the plane binary images, it is possible to efficiently execute a form image generation process in which the dropout color is removed.
[0094]
Further, since the black pixel frequency or the white pixel frequency of the synthesized binary image (the resolution is higher than that of the plane binary image itself) of the luminance signal binary image and the plain binary image of the read image is used, the dropout plane is used. Can improve the accuracy of identification.
[0095]
In addition, when the combined binary image of each plane is generated when the dropout plane cannot be specified based on the above-described characteristic value and black pixel frequency, the calculation synthesis with the luminance signal binary image of the form is also executed. Therefore, the resolution of the synthesized binary image itself can be increased.
[0096]
In addition, by adjusting the threshold for binarization, the threshold for background separation of the local binarization method, the target range, etc., we are aiming at appropriate black pixelization of the blurred part of the letters and numbers entered by the user, In addition to improving the accuracy of specifying the dropout plane, it is possible to increase the recognition accuracy by making the letters and numbers in the final output image thicker.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing an outline (part 1) of a dropout plane detection method;
FIG. 2 is an explanatory diagram showing an overall configuration corresponding to the detection method of FIG. 1;
FIG. 3 is an explanatory diagram showing an outline (part 2: R plane) of a dropout plane detection method;
FIG. 4 is an explanatory diagram showing an outline (part 2: G plane) of a dropout plane detection method;
FIG. 5 is an explanatory diagram showing an outline (part 2: B plane) of a dropout plane detection method;
FIG. 6 is an explanatory diagram showing an overall configuration corresponding to the detection method of FIGS.
FIG. 7 is an explanatory diagram showing an outline (part 3: R plane) of a dropout plane detection method;
FIG. 8 is an explanatory diagram showing an outline (part 3: G plane) of a dropout plane detection method;
FIG. 9 is an explanatory diagram showing an outline (part 3: B plane) of a dropout plane detection method;
FIG. 10 is an explanatory diagram showing an overall configuration corresponding to the detection method of FIGS.
FIG. 11 is an explanatory diagram showing an example of a form having blue and red dropout colors.
12 is an explanatory diagram showing an outline of a composite image output method for the form in FIG. 11; FIG.
13 is an explanatory diagram showing an overall configuration corresponding to the output method of FIG. 12. FIG.
14 is an explanatory diagram showing a processing procedure corresponding to the overall configuration of FIG. 13;
FIG. 15 is an explanatory diagram showing an outline of a CCD and a pixel shifting method;
FIG. 16 is an explanatory diagram illustrating a processing procedure of an automatic discrimination threshold method for a plain multi-valued image.
FIG. 17 is an explanatory diagram showing an outline of a local binarization method (filter).
FIG. 18 is an explanatory diagram showing a processing procedure of a local binarization method.
FIG. 19 is an explanatory diagram illustrating adjustment of a local binarization target range.
FIG. 20 is an explanatory diagram illustrating a change example of a binary image by blur correction at the time of local binarization.
[Explanation of symbols]
11: Red plane
12: Histogram of red plane multilevel image
13: Green plane
14: Histogram of green plane multilevel image
15: Blue plane
16: Histogram of blue plane multi-valued image
17: Red frame on the form (dropout part)
21: Two-dimensional color CCD
22: Pixel shifting means
23: Captured image composition means
24: Shading correction means
25: R, G, B plane dividing means
26-28
: Red, green, and blue plain images
29: Y signal image generating means,
30: Y signal image
31: Local binarization unit
32: Y signal binary image
33-35
: Histogram calculator
36-38
: Automatic discrimination threshold method processor
39: Plain image determination / selection unit
40: Selected image binarization unit
41: Selected plane binary image
42: AND operation synthesis unit
43: Composite image output unit
44: Integrated block
51: Y signal binary image
52, 52 '
: First red plane binary image
53, 53 '
: Second red plane binary image
54: AND composite image (black pixel frequency = 11.823%)
55: AND composite image (black pixel frequency = 12.030%)
56: XOR composite image (black pixel frequency = 5.943%)
57: XOR composite image (black pixel frequency = 6.014%)
62, 62 '
: First green plane binary image
63, 63 '
: Second green plane binary image
64: AND composite image (black pixel frequency = 13.653%)
65: AND composite image (black pixel frequency = 15.495%)
66: XOR composite image (black pixel frequency = 4.424%)
67: XOR composite image (black pixel frequency = 3.577%)
72, 72 '
: First blue plane binary image
73, 73 '
: Second blue plane binary image
74: AND composite image (black pixel frequency = 13.787%)
75: AND composite image (black pixel frequency = 16.273%)
76: XOR composite image (black pixel frequency = 4.404%)
77: XOR composite image (black pixel frequency = 4.219%)
81-83
: Binarization part
84: R binary image
85: G binary image
86: B binary image
87: AND operation synthesis unit
88: YR composite binary image
89: YG composite binary image
90: YB composite binary image
91: Black pixel frequency calculation / determination unit
92: Plain image selection / output unit
93: Integrated block
101-103
: Local binarization unit
104: R binary image
105: G binary image
106: B binary image
107: XOR operation synthesis unit
108: YR composite binary image
109: YG composite binary image
110: YB composite binary image
111: Black pixel frequency calculation / determination unit
112: Plain image selection / output unit
113: Integrated block
121: Seal frame (blue dropout color)
122: Text box (red dropout color)
131: Y signal binary image
132: R-plane binary image
133: YR composite binary image
134: B-plane binary image
135: YRB composite binary image
141: RGB binary image selection unit
142: AND operation synthesis unit
143: YR composite binary image
144: AND operation synthesis unit
145: YRG composite binary image
146: AND operation synthesis unit
147: YRGB composite binary image
148: Composite binary image output unit
151: Target pixel for local binarization processing
152: 7 × 7 pixel rectangular filter for the target pixel
153: Character part stroke pixel
154: Background pixel
σmin: threshold for background separation (for example, 100)
E: Average density value of each pixel in the rectangular filter
E2: density mean square value of each pixel in the rectangular filter
g2: Density dispersion value in the rectangular filter
S: threshold for binarization of the target pixel
ot: Automatic discrimination threshold
a, b: coefficients for adjusting the local binarization target range
b-ave: density average of black region (density 0 to ot)
w-ave: density average of white area (density ot + 1 to 255)
bt: the lower limit concentration of the local binarization target range
wt: Upper limit concentration of the local binarization target range

Claims (2)

In a form reading device that optically reads a form including a dropout color part and a user entry part related thereto in pixel units,
Pixel density calculation means for obtaining a pixel density distribution for each plane of the read image of the form ;
Based on the pixel density distribution of each of the planes, each the pixel density distribution, a threshold value calculating means for determining the threshold values for the drop-out color discrimination and dropout plane binarization,
Plane selection means for selecting a dropout plane of the read image based on each threshold;
Based on the threshold corresponding to the selection, binarization means for binarizing the dropout plane according to the selection;
A form reading apparatus comprising: In the program used in the reading process of the form including the dropout color part and the user entry part related to it,
The program is
Perform a calculation to obtain the pixel density distribution for each plane of the read image of the form,
Based on the pixel density distribution of each of the planes, each the pixel density distribution, obtains the threshold values for the drop-out color discrimination and dropout plane binarization,
Based on each of the threshold values, a dropout plane of the read image is selected,
Based on the threshold corresponding to the selection, binarize the dropout plane for the selection,
A form reading processing program characterized in that the function is implemented in a computer.

Images