JP4117648B2 - Form, form processing method, form processing program, recording medium recording form processing program, and form processing apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a form, a form processing method, a form processing program, a recording medium on which a form processing program is recorded, and a form processing apparatus, and more particularly, can be used for reading a form used in a window or an office, and is processed on the form side. The present invention relates to a form in which a method is described, a form processing method for processing the form, a form processing program, a recording medium storing the form processing program, and a form processing apparatus.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, as a bridge between paper and computer processing, an optical form reading device, a device that dynamically reads handwriting information from writing on paper, an e-pen, an anoto pen, and the like are known. However, the latter requires a special device for writing.
Further, in order to extract handwriting from the entry frame of the form, a dropout color may be used for the entry frame. However, the use of color increases the cost, and there is a problem that monochrome copies and monochrome faxes that are currently widely used cannot be used. Due to reasons such as printing costs, reading device costs, and the use of monochrome faxes, forms using dropouts are decreasing and monochrome forms are becoming mainstream. In addition, with the high accuracy of laser beam printers and the generalization of page description languages such as PostScript, it is possible to print forms consisting of fine dot textures with an inexpensive printer. Furthermore, anyone can easily design a form by creating a form creation editor. Since handwritten forms are easy to input and remain as evidence, digitization has not progressed, and there is an increasing need for easy digitization.
[0003]
A processing method for a form having an entry frame composed of dots has been proposed (see, for example, Patent Document 1 and Non-Patent Document 3). In addition, a method for recognizing and correcting characters entered in a form has been proposed (see, for example, Patent Documents 2 and 3). An active form system in which processing is described on the form side has been proposed (see, for example, Patent Document 1, Non-Patent Documents 1 and 2).
In addition, a method for recognizing a form structure from detection of a line or an intersection has been proposed (for example, see Non-Patent Documents 4 and 5).
[0004]
[Patent Document 1]
International Publication No. 01/93188 Pamphlet
[Patent Document 2]
JP 2001-052110 A
[Patent Document 3]
JP 2001-052111 A
[Non-Patent Document 1]
Taro Shimamura, Ran Toki, Atsushi Masuda, Motoki Onuma, Takeshi Hamada, Atsushi Kuronuma, Masaki Nakagawa, “Design and Prototype of Active Form System”, IEICE Technical Report, December 2002, PRMU2002-132, Vol.102, No. 531, pp.19-24.
[Non-Patent Document 2]
Zhu Qinlan, Shimamura Taro, Nakagawa Masaki, `` Prototype of Active Form Processing Technology '', IEICE, IEICE Technical Report, December 2002, PRMU2002, Vol.102, No.531, pp.25-30.
[Non-Patent Document 3]
Yoji Maeda, Masaki Nakagawa, “Document Editing Method Using Paper Interface”, Journal of Information Processing Society of Japan, 2000, Vol. 41, No. 5, pp. 1308-1316.
[Non-Patent Document 4]
T. Watanabe, Q. Luo, and N. Sugie, `` Layout recognition of multi-kinds of table form documents '', IEEE Trans. On Pattern Analysis and Machine Intelligence, 1995, Vol. 17, No. 4, pp.432- 334.
[Non-Patent Document 5]
LY Tseng and RC Chen, `` Recognition and data extraction of form documents based on three types of line segments '', Pattern Recognition, 1998, Vol. 31, No. 10, pp.1525-1540.
[0005]
[Problems to be solved by the invention]
However, in monochrome forms, it may be difficult to separate the form entry frame from the handwriting. For example, when a handwritten character overlaps with an entry frame, or when there is dirt on a form and reading noise, it may not be accurately separated. In addition, although the entered characters can be recognized, misreading may still occur. In particular, it is assumed that context processing cannot be used for recognizing an e-mail address, and that the subsequent processing is greatly affected.
In view of the above, the present invention can be used to read a form used in a window or an office, and can easily separate a form entry frame and a written character, a form processing method, a form processing program, and a program recording the program. It is an object of the present invention to provide a recording medium and a form processing apparatus. Another object of the present invention is to increase the recognition rate of written characters. Furthermore, the present invention is intended to describe the processing method of the form itself on the form side instead of the apparatus side, to use a general-purpose machine as a reading apparatus, and to process various forms on a single machine. .
[0006]
Another object of the present invention is to provide a form having a dot shape and a form processing method for increasing the amount of information to be superimposed. An object of the present invention is to provide a coding method and a decoding method for stably reading superimposition information from overlapping and fouling with handwriting. It is another object of the present invention to provide an auxiliary entry method for alphanumeric symbols that are particularly easy to misread. Another object of the present invention is to provide a form that can be easily created by an editor and printed by a printer. Another object of the present invention is to shorten the processing time by omitting the trouble of scanning the image between the lines of the entry frame.
[0007]
[Means for Solving the Problems]
The present invention facilitates separation from handwriting even in a monochrome environment by using a dot texture composed of fine dots in an entry frame, and superimposes handwritten entry attributes and instruction information after reading. In the active form, in which the form side leads the processing completely, the following is provided.
As a code shape for superimposing information in an entry frame, a dot shape is proposed so that a large amount of information can be superimposed stably by a dot texture.
[0008]
In addition, as a method for recognizing a code figure, a recognition method using a histogram in four directions and a recognition method using partial scanning are proposed. In the recognition method using the four-way histogram, the code shape feature is judged by recognizing the code by taking a four-way histogram for the code image. In the recognition method by partial scanning, by partially scanning the code image, scanning is performed sequentially for each position where the line segment should be in the code figure, and if there is a line segment, the corresponding information bit is set to 1, 0 if there is no line segment.
[0009]
In addition, as a stable detection method for the entry frame, in order to facilitate the detection of the entry frame using the dot texture, the entry frame can be stably displayed by taking a histogram in the vertical and horizontal directions after expanding each dot. To detect.
Since there is a space between the entry boxes in the form, the processing time is increased by omitting the blank processing time. Even if you do not separate the entry frame from the handwriting, you can detect the line of the entry frame while including the handwriting (handwriting is in the way to detect the position in the line, but it is not in the way to detect the line). If line detection is performed before separation and only one line-by-line image is labeled, labeling processing between lines can be omitted.
[0010]
We also propose a method to prevent the influence of noise when detecting the border of the entry frame. When detecting the position of the entry frame by the histogram in the vertical or horizontal direction, in order to avoid interference with the noise, the value below a certain threshold is regarded as 0 for the result of the histogram and is ignored as a blank even in a noisy place. To do. By doing so, the position of the entry frame can be detected without being affected by noise. However, since the value below the threshold value is regarded as 0, the information below the threshold value is also ignored at the boundary of the entry frame, which has an adverse effect on reading the information superimposed in the entry frame. In order to prevent such information loss when detecting the position of the entry frame, the position of the entry frame is detected at a position below the threshold in the histogram, and the histogram is further moved from this position toward the outside of the entry frame. If the histogram value does not become 0 even when the value of the histogram has changed from a value greater than 0 to 0, or the length traced from the start position f exceeds a certain threshold, the process starts. The position shifted from the position f to the outside by the threshold value is set as the start or end position of the entry frame.
[0011]
Also, as a method of extracting code figures even when a dot texture document is scanned at a slight tilt, the number of dot lines in the dot texture is smaller than the number of dot columns, and the histogram overlap in the vertical direction is small. Then, a vertical histogram is taken for the dot texture image, and the information bar is cut out from the left to the right of the information bar. Next, a horizontal histogram is taken for each column image, and a figure for each code is cut out.
[0012]
As a method of detecting and skipping information records that have been erased by overlapping with handwriting, the code figure for each column is cut out from the left to the right in the information bar image, and the interval between the previous column and the next column is assumed. When the code figure and handwriting overlap each other when the width is larger than the predetermined width, it is determined that the code figure is missing due to the handwriting separation process, and the subsequent code figure is ignored until a delimiter is found. Otherwise, the column code is cut out and recognized for processing.
This prevents errors such as attaching minute noise to each code figure and connecting them, or processing minute noise as a code figure. For example, if the size of the cut out figure is smaller than the expected code figure, it is ignored as noise. This prevents errors such as sticking and linking or processing minute noise as code figures.
[0013]
Also, we propose a method for dealing with information bar position detection errors. The boundary line between the information bar of the entry box and the other part may be detected with an error from the boundary line of the correct information bar. If the code figure is cut out and decoded according to the detected information bar boundary line, a part of the code figure that protrudes from the information bar is excluded and recognized, which may cause a recognition error. In order to avoid this, the following processes (1) to (4) are executed. (1) According to the detected information bar boundary line, the position of each code column is detected from left to right by a histogram in the vertical direction with respect to the information bar image. (2) A horizontal histogram is scanned with respect to the code image for each column, and each code is cut out. The cutout direction of the upper information bar of the entry frame is from top to bottom, and the cutout direction of the lower information bar is from bottom to top. From this stage onward, the information on the boundary line b is not used. (3) After one code is cut out, the number of codes cut out in this column and the scanned length L are examined. Let N be the number of code lines in the information bar. If the number of codes in this column has already reached N, or if the scanned length L has reached a length corresponding to N, code extraction is terminated. Otherwise, the histogram is subsequently scanned and the code is cut out. Here, the purpose of detecting the end with the scanned length is to prevent the code figure from going over the area of the information bar when the code figure overlaps with the handwriting and is lost. (4) The number of codes is checked for each column when the code segmentation is completed. If N is not reached, it is regarded as an error in the number of codes, and is ignored until a delimiter is found for the subsequent code figure.
[0014]
We also propose a method to increase the recognition rate of email addresses. Characters used in e-mail addresses are alphanumeric symbols with a small amount of information and have a low character recognition rate, and context processing cannot be used due to meaningless character strings. In e-mail address misrecognition, the main cause is misrecognition of similar characters. If this misrecognition can be avoided, the recognition rate can be increased. Therefore, in order to limit the character type of the email address, another entry frame is provided near the email address entry frame in the active form, and the corresponding character type of the email address is entered with a simple symbol. Considering the ease of recognition and the ease of entry by people, allow horizontal lines to be entered by overlapping horizontal lines above, below, or both within the frame.
A form is a form for character input in which a character input frame is formed with a dot texture in which a plurality of minute dots (black areas) or line segments are two-dimensionally arranged. Consists of a heading character drawn with a dot texture with a wide line including a plurality of dots, and a collection of dots or line segments arranged two-dimensionally. And an entry frame defined by a dot texture with wide lines to include.
[0015]
Since the processing unit is set from the form in which characters are input, each dot is set smaller than the handwritten black region, each dot constituting the dot texture disappears before the character image by the contraction processing, or The form for detecting the character image of the written character is targeted by removing the entry frame image based on the dot texture by either removing the connected component smaller than the predetermined threshold.
In this method, by using an entry frame composed of a dot texture composed of minute dots, even in a monochrome environment, handwriting can be easily separated even if it overlaps the entry frame. In addition, by embedding handwritten attributes and character types in the entry frame, the recognition rate can be increased for cut out handwritten characters. If the contents of handwritten characters are associated with attribute and headline information, settings for each form can be eliminated. Furthermore, by embedding a processing method for handwriting after reading, it is possible to realize a system for instructing processing on the form side according to the form only by reading the form. A form becomes active by embedding a program that has been required on the processing system side in the form side. This is the reason for the “active form”.
[0016]
According to the first solution of the present invention,
A form input step for inputting form image data including entry frame data composed of a code in which information is expressed by an inclination of a line segment or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
Cutting out each code of the detected information bar;
A step of obtaining a histogram of four directions of the cut code, the vertical direction, the horizontal direction, the right diagonal direction, and the left diagonal direction;
A code recognition step for recognizing the code by determining the shape feature of the code based on the obtained histogram in the four directions;
, A form processing program for causing a computer to execute these processes, and a computer-readable recording medium recording the program.
[0017]
According to the second solution of the present invention,
A form input step for inputting form image data including code data in which information is expressed by a dot width, a slope of a line segment, or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
A superimposition information recognition step of recognizing each code of the information bar detected in the entry frame detection step and extracting superimposition information;
A character recognition step of performing character recognition according to the attribute and / or character type of the handwritten data indicated by the extracted superimposition information for the handwritten data separated in the entry frame detecting step;
A command creation step for creating command data for causing other application systems to perform predetermined processing based on the character information recognized in the character recognition step and the superimposition information extracted in the superimposition information recognition step;
Outputting the instruction document data created in the instruction document creating step and / or storing the form image data inputted in the form input step and the instruction document data in association with each other;
, A form processing program for causing a computer to execute these processes, and a computer-readable recording medium recording the program.
[0018]
According to the third solution of the present invention,
A form input unit for inputting form image data including input frame data composed of a code in which information is expressed by a slope of a line segment or a combination of a plurality of line segments, and written handwritten data;
A processing unit that detects an entry frame of input form image data and recognizes a code in the entry frame;
With
The processor is
Means for inputting form image data from the form input unit;
Based on the form image data input from the form input unit, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, An entry frame detecting means for detecting
Means for cutting out each code of the detected information bar;
Means for obtaining a histogram of four directions of the cut cord in the vertical direction, the horizontal direction, the right diagonal direction, and the left diagonal direction;
A code recognition means for recognizing the code by judging the shape characteristic of the code based on the obtained histogram in the four directions;
A form processing apparatus is provided.
[0019]
According to the fourth solution of the present invention,
A form input unit for inputting form image data including a dot width or a slope of a line segment or a code in which information is expressed by a combination of a plurality of line segments, and written handwritten data;
A storage unit for storing form image data and / or command data created based on the form image data;
A processing unit that recognizes the handwritten information entered in the input form image data and the information superimposed on the entry frame and executes a predetermined process;
With
The processor is
Means for inputting form image data from the form input unit;
An entry frame that separates entry frame data and handwritten data based on the input form image data, and detects each entry frame data and an information bar in the entry frame data in which predetermined information is superimposed by a code. Detection means;
Superimposition information recognition means for recognizing each code of the information bar detected by the entry frame detection means and extracting superposition information;
Character recognition means for performing character recognition on the handwritten data separated by the entry frame detection means according to the attribute and / or character type of the handwritten data indicated by the extracted superposition information;
Instruction book creation means for creating instruction book data for causing other application systems to perform predetermined processing based on the character information recognized by the character recognition means and the superposition information extracted by the superposition information recognition means;
Means for outputting the instruction book data created by the command book creating means and / or storing the form image data inputted by the inputting means in correspondence with the instruction book data in the storage unit;
A form processing apparatus is provided.
[0020]
Also, in one of the solution means of the present invention, the entry frame detection step and the entry frame detection means are each of the entry frame data based on horizontal and vertical histograms or line segment detection and / or intersection detection with respect to the form image data. And an information bar in the entry frame data on which predetermined information is superimposed by a code.
[0021]
Furthermore, according to the fifth solution of the present invention,
An information bar in which information records on which predetermined information is superimposed are arranged over a plurality of lines by a code in which information is expressed by a dot width, a slope of a line segment, or a combination of a plurality of line segments. An entry box comprising:
A horizontal bar composed of the code and including information on the form or the information bar;
A form is provided.
[0022]
DETAILED DESCRIPTION OF THE INVENTION
1. Overview of active forms and their processing systems
1.1 Active forms
FIG. 1 is a configuration diagram of an active form.
By using a dot texture composed of fine dots in the entry frame, the active form enables separation of the entry frame and handwritten characters even in a monochrome environment, and further distributes the dots in a non-uniform manner. It is a form based on a technology that allows embedding of the character attributes and instruction information after reading into the entry frame.
The form includes a horizontal bar 10 and an entry frame 20 having an information bar 30. A horizontal line printed under the form title is called a horizontal bar 10. The entry frame 20 and the horizontal bar 10 are printed with a dot texture. The dot texture of the entry frame includes handwritten attributes (for example, names, addresses, email addresses, free writing, etc.), character types (for example, numbers, kanji, English), headings, and processing methods (for example, handwriting). Superimpose information (superimposition information) on the upper and lower sides of the entry frame instructing characters (recognizing characters, sending description contents by e-mail, saving in a database, etc.). A portion where this information is superimposed is referred to as an information bar 30. Further, the information on the entire form is superimposed on the horizontal bar 10. In addition, a division for dividing and entering characters one by one in the entry frame is called a cell 40.
[0023]
In an entry frame with a dot texture, dots are arranged in a horizontal (row) direction and a vertical (column) direction to form a texture. If the number of dot lines in the entry frame is the same, including the horizontal bar 10, and if the number of dot lines in the horizontal bar 10 is obtained, the number of dot lines in all the entry frames can be easily known. Further, in the present embodiment, it is possible to superimpose superimposition information on a code figure in which information is represented by a slope of a line segment or a combination of line segments other than dots. You may comprise this line segment using a dot. In the following description, the code figure may be simply referred to as a dot.
[0024]
1.2 Active form processing system
FIG. 2 is a hardware configuration diagram of the active form processing system.
The active form processing system includes, for example, a processing unit 1, a form input unit 2, an output unit 3, and a storage unit 5. The active form processing system may include an appropriate input unit and display unit 4. The active form processing system, for example, extracts description content and superimposition information from the read image data of the active form and outputs it to the instruction sheet. Details of the processing of the processing unit will be described later. The storage unit 5 stores form image data and command data created based on the form image data. In addition, for example, a table in which the value indicated by the code corresponds to the information is stored in the storage unit 5 in advance. Note that the storage unit 5 can include, for example, a table in which feature values of standard patterns correspond to information indicated by the standard patterns.
[0025]
2. Active form details
2.1 Code shape for information superimposition
In the active form, information indicating the handwritten attribute, character type, heading, and processing method to be entered in each entry frame is superimposed on the dot texture. One method for embedding information in the dot texture is to change the shape of the dots. Therefore, some code shapes representing information will be described.
[0026]
FIG. 3 is a block diagram of a code using two types of large and small figures (two types of large and small codes). Two types of dots of different sizes represent binary bits 1 and 0, and information is superimposed on this. The height of the two cords is the same and is h. Let L be the width of a large dot. Considering the gap between codes, an area for arranging codes evenly in a form is defined as a code arrangement area. The size of each code is the same, and the code is placed at the center. If the arrangement area of each code is arranged without leaving a gap vertically and horizontally like a grid, the code has a dot texture with a gap. Similarly, other types of codes to be described later are arranged at the center of the arrangement area to generate a dot texture for the entry frame.
[0027]
FIG. 4 is a configuration diagram of a code (8-direction line segment code) using 8-direction line segments. Eight types of codes shown in FIG. 4 from 0 to 7 can be represented by eight line segments. In FIG. 4, numbers such as “000” and “001” shown below each code are binary numbers corresponding to these codes. Each code can be associated with an appropriate number other than those shown in the figure. In addition to the eight directions, multi-directions using line segments inclined at a predetermined angle may be used. A square area of the same size for drawing a code is called a drawing area.
[0028]
FIG. 5 shows the relationship between the drawing area and the arrangement area. The drawing area is smaller than the arrangement area and is located at the center of the arrangement area. Similar to the above, the arrangement area is arranged without gaps in the vertical and horizontal directions like a grid, so that each cord is arranged with a gap.
FIG. ¹² Code that uses different types of figures (2 ¹² FIG. 2 ¹² As shown in FIG. 6, the type code is composed of a combination of 12 types of line segments. Based on the drawing area of size a × a, 2 depending on the presence or absence of these 12 types (12) of line segments. ¹² Types, that is, 4096 types of codes are created.
[0029]
FIG. ⁸ Code that uses different types of figures (2 ⁸ FIG. 2 ⁸ As shown in FIG. 7, the type code is composed of a combination of eight types of line segments. Based on the drawing area of size a × a, 2 depending on the presence / absence of these 8 types (8 lines). ⁸ Types, that is, 256 types of codes are created. 2 ⁸ Type code, 2 ¹² In addition to the type code, 2 depending on the appropriate line segment combination ⁿ Type codes can be created.
[0030]
2.2 Coding information
In order to guarantee the minute dot shape of the dot texture, a form is created in the PostScript language, the form image data is sent to the PostScript printer, and the form is printed. The coding of each of the two types of large and small codes and the 8-direction line segment code will be described. Other codes can basically use the latter coding as well.
[0031]
2.2.1 Coding in two types of large and small codes
FIG. 8 is a diagram showing information records to be superimposed. The information shown in FIG. 8 is superimposed for each dot texture entry frame. This is called an information record. For example, an information record is represented by a fixed length of 8 + 8 + 8 + 16 × 4 + 8 = 96 bits. The information record can be superimposed on the upper side of the entry frame, for example.
FIG. 9 is a table showing correspondence between attributes and values indicated by codes. The attribute is a handwritten attribute entered in the entry frame, and the heading characters in the entry frame are classified into categories. There are various types of attributes depending on the type and usage of the form. FIG. 9 is an example of attributes obtained by extracting heading characters from 36 types of forms that are actually used. As shown in FIG. 9, for example, it can be classified into 14 types of attributes and coded. However, appropriate attributes can be used, and are not limited to these.
[0032]
FIG. 10 is a table showing the correspondence between character types and instructions and values indicated by codes. The character type is the character type of the handwritten character entered in the entry box. The command is a processing method after reading for the handwritten text entered in the box. Character types and instructions can be represented by 8-bit flags. For example, as shown in FIG. 10, a bit number corresponding to each character type and command is set (for example, 1). It should be noted that other appropriate character types and commands can be used. For example, the command “recognize” may be “recognize with frame”, “recognize without frame”, or the like. In the heading character, for example, a Shift-JIS code (64 bits in total) of the first four characters of the entry frame heading is embedded. The terminal symbol is embedded with 8-bit zeros. In the storage unit 5 of the active form processing system, for example, tables as shown in FIGS. 9 and 10 are stored in advance.
[0033]
FIG. 11 is a diagram illustrating a method of coding information records into dot rows.
Two kinds of large and small codes are arranged horizontally (line), and the above information is superimposed on the upper side of the entry frame, for example. Since there is a risk that the superimposition information is lost due to the overlap of the entry frame and the handwritten character, the same superimposition information is repeated as much as possible within the dot arrangement of one line, and, for example, dots indicating 0 are inserted in the remaining part. In the second and subsequent lines, for example, a dot indicating 0 is put at the head and shifted by 2 dots, and the above dot line is repeated. In other words, 96-bit information records are connected as long as possible, and a dot indicating 0 is inserted in the remaining part. Losing information can be restored by taking a majority decision of these superposition information. It should be noted that the number of dots shifted in each row can be shifted by an appropriate number of dots other than two dots. Further, the coding method shown in FIG. 11 can be applied to other codes such as an 8-way line segment code in addition to the two types of large and small codes.
[0034]
2.2.2 Coding in 8 direction line segments
FIG. 12 is a diagram showing information records to be superimposed. The 8-way line segment code can be developed by superimposing information on two types of large and small codes, and more information can be superimposed, and an efficient superposition method such as error detection can be used. The information record shown in FIG. 12 is superimposed for each entry frame.
The attribute can be the same as the superposition of two kinds of large and small codes. Since the information amount of one code of the 8-way line segment code is 3 bits, for the convenience of expression, for example, 0 is added to the end (or the top) of the attribute of the two types of large and small codes shown in FIG. A bit. The character type and instruction can be represented by a 9-bit flag. The expression method can be the same as in FIG. Further, as the character type of the bit number 9, in order to increase the recognition rate of the mail address, information indicating the character type such as alphanumeric characters, “@”, “.”, Etc. used therefor can be used. The heading character is represented by a Shift-JIS code from 1 to 10 double-byte characters. The improvement of the mail address recognition rate will be described later. The information record of FIG. 12 (undefined length of 45 to 207 bits) is divided every 3 bits and converted into a code representing it. The information record including each code is superimposed on the upper and lower information bars of the entry frame.
[0035]
FIG. 13 shows an example of superposition of the 8-direction line segment code. As shown in FIG. 13, for example, considering a four-row dot texture, four codes are arranged from top to bottom for each column, and are coded from left to right. In the example shown in FIG. 13, line segments are arranged in the directions 1, 2, 3, 4, 5, and 6. A delimiter is added at the beginning and end of the information record, and a parity code is placed before the end, and the second and third information records are repeated. This is repeated as much as possible. The second and subsequent information records are the same as the first information record, and first check whether there is an error in the parity, and then take a majority vote of a plurality of information records to restore the lost information. Can do.
[0036]
FIG. 14 is an explanatory diagram of a parity code setting method. A parity code is added so that the value of the corresponding bit between the delimited figures becomes 0 by exclusive OR. For example, as shown in FIG. 14, for data indicating “000”, “001”, and “010”, the parity code is set so that the exclusive OR of the corresponding bit values of each data becomes zero. A code indicating “011” is added.
FIG. 15 shows a figure representing a delimiter. The delimiter has a function of indicating the boundary of each information record. For example, the delimiter is assumed to be 8 by taking over the number of the 8-direction line segment code shown in FIG. In addition to the shape shown in FIG. 15, the delimiter may have an appropriate shape that can be distinguished from that.
[0037]
2.3 Code figure recognition method
To recognize the code, any of the following methods can be used.
2.3.1 Recognition method using histograms in four directions
FIG. 16 is an explanatory diagram of a recognition method using a histogram in four directions. By taking histograms in four directions (for example, vertical, horizontal, right diagonal, left diagonal) with respect to the code image (data), the code shape feature can be determined to recognize the code. As shown in FIG. 16, the histograms in the four directions are clearly different depending on the code.
[0038]
2.3.2 Recognition method by matching
A large number of image samples are created for all types of codes, and feature values of each sample are extracted. An average feature value can be obtained for each code type, and a standard pattern for each code can be created. The feature value of the input code is extracted by the same method as the feature value extraction of the standard pattern sample, is matched with the standard pattern of each code, the standard pattern having the closest feature is obtained, and this standard pattern is used as the recognition result.
FIG. 17 is an explanatory diagram of extraction of standard pattern samples and feature values of input codes. As a feature value extraction method, a method generally used in character symbol recognition can be used. First, the line density is calculated, nonlinear normalization is performed so that the line density is uniform, and smoothing is performed to maintain line connectivity for eliminating jagged edges of the character image ((1) in FIG. 17). . Next, components in four directions of horizontal, vertical and diagonal are extracted from the normalized pattern ((2) in FIG. 17). Each is divided into 8 × 8 sections, and the amount of direction components included in each section is used as a feature, and 64 pieces thereof are arranged, and then four direction components are arranged to obtain a 64 × 4 = 256-dimensional feature vector ( FIG. 17 (3)).
[0039]
FIG. 18 shows an explanatory diagram of code recognition by matching with a standard pattern. For example, the characteristic value of the standard pattern as shown on the right side of FIG. 18 and the information indicated by the standard pattern are stored in the storage unit in advance, and the characteristic value of the read code shown on the left side of FIG. The code can be recognized by comparing with the value.
2.3.3 Recognition method by partial scanning
FIG. 19 is an explanatory diagram of a recognition method based on partial scanning. A code figure is recognized by partially scanning the code image. Scan in order for each position where the line segment should be in the code figure. If there is a line segment, the corresponding information bit is set to 1, and if there is no line segment, it is set to 0. The diagram shown in FIG. ⁸ It is an example of a kind code. The white part is the part without the line segment, and the black part is the part with the line segment. In FIG. 19, the background is gray to make it easy to see a portion without a line segment, but in an actual form, the background and a portion without a line segment can be the same color. Each line constituting the code corresponds to a 2-bit digit. These line segments are scanned in sequence. If there is a line segment, the corresponding digit is set to 1, and if there is no line segment, it is set to 0. If the drawing area or the arrangement area of the code can be cut out, the scanning portion can scan a predetermined position in these areas. By sequentially scanning, the 2-bit number “10110110” is obtained in the example of FIG. When converted to decimal, it is “182”. This value is the value of this code.
[0040]
2.4 Examination of each cord shape
The size of each code described above and the result of recognition experiment for each code will be described.
[0041]
2.4.1 Experimental environmental conditions
The processing device used for creating and recognizing each code is a model equipped with a 2.26 GHz CPU of Pentium (registered trademark) 4 and a memory of 1.00 GB, and mounted with Windows (registered trademark) XP as an OS.
As the printing accuracy, 1200 dpi, which can be printed even with an inexpensive laser beam printer, was adopted. As described above, the active form can be printed by creating a form code in the PostScript language and sending the form code to the PostScript printer. The higher the printing accuracy, the better. However, if a very high accuracy is assumed, the apparatus cost increases. As a result of the experiment, the minimum diameter of a round dot that can be printed with an accuracy of 1200 dpi is 0.1 mm.
[0042]
The scanning accuracy was 600 dpi. The higher the scanning accuracy for reading a form, the better. However, if the accuracy is high, the size of the read form increases, and the time for form processing increases. For example, when scanning is performed with an accuracy of 600 dpi, the size of the A4 form is 4924 × 6883, and the processing time of the computer is less than 2 seconds for a normal form of two types of large and small codes at this size.
In addition, in order to prevent dots having a small dot texture from being connected by scanning, it is necessary to provide an appropriate interval between the dots. It is desirable that there is a minimum interval between dots of 0.2 mm in a 600 dpi scan, including printing errors during printing and errors due to scans in a tilted form or binarization. This size can be printed with a printing accuracy of 1200 dpi. This interval does not impair the appearance of the form.
[0043]
2.4.2 Examination of each code
FIG. 20 is a dot texture entry frame created by two types of large and small codes and an 8-direction line segment code. FIG. 20A shows a dot texture entry frame created by two types of large and small codes.
Since the minimum size dot that can be printed with an accuracy of 1200 dpi is 0.1 mm, the height h of the cord shown in FIG. 3 can be 0.1 mm. In order to distinguish large dots from small dots, the length L of the large dots shown in FIG. 3 can be set to 0.25 mm in consideration of errors such as scanning and binarization when the form is inclined. As described in the scanning accuracy, in order to set the interval between dots to 0.2 mm, the size of the rectangular arrangement area shown in FIG. 3 can be 0.45 width × 0.3 mm height. FIG. 20A shows an example of a dot texture entry frame created with a code of this size. In this example, information is superimposed on the upper side of the entry frame.
Since there are only two types of codes, they can be easily identified by taking a vertical histogram and determining the width of the code. As a result of recognizing one form of the two types of codes having the above-mentioned sizes, the processing time until the correction interface is called is 1782 ms. There were no identification errors.
[0044]
FIG. 20B shows a dot texture entry frame created by the 8-direction line segment code. The line of the 8-direction line segment code has a thickness of 0.1 mm, which is the minimum size that can be printed at 1200 dpi. The number of pixels with a size of 0.1 mm in a 600 dpi scan is three. In order to distinguish the code indicating “100” and the code indicating “101” shown in FIG. 4, 4 pixels are required for the width (horizontal direction) of the code “101”. When converted to the size before scanning, it is 0.168 mm. In consideration of errors and the like, the width of the “101” code can be 0.2 mm. In this case, the size of the drawing area square is 0.4 mm. If the interval between dots is guaranteed to be 0.2 mm, the size of the arrangement area is 0.6 × 0.6 mm. FIG. 20B is an example of an entry frame created with a code of this size. In this example, as in FIG. 20A, the code is shifted by 2 codes in each row in the horizontal direction, and information is superimposed on the upper side of the entry frame.
[0045]
By taking a histogram in four directions, an eight-direction line segment code can be easily identified. In order to compare, the code recognition process was tried about the recognition method by matching and the recognition method by a histogram of four directions. The standard pattern dictionary for the matching method was created by making a total of 160 images in each of the 8 types of code in the PostScript language, using the images obtained by 1200 dpi printing and 600 dpi scanning as samples, and taking the characteristics. .
FIG. 21 shows a result of a trial of a recognition method using a four-way histogram and matching. The recognition rate is the recognition rate for all sample images used for creating a dictionary, and both reach 100%. However, the processing time is faster in the recognition method using the histogram. The processing time for one form indicates the processing time until a correction interface for correcting a recognized character is called.
[0046]
In FIG. ¹² This indicates the dot texture entry frame created by the type code.
This code shape is considerably complicated, and if the code image is small, the combination lines come into contact with each other, and even a person may not be able to identify them. The size of the combination line was made so that it could be identified by a person without contact, and was identified by a decoding method based on matching, and a size that could guarantee a recognition rate of 98% or more was found. The drawing area of this size is 0.9 × 0.9 mm. Considering the minimum code interval of 0.2 mm, the arrangement area is 1.1 × 1.1 mm. This size can be further reduced if the printing accuracy and the scanner accuracy are increased. FIG. 22 is an example of an entry frame created with a code of this size. In this example, information is superimposed on the upper side of the entry frame.
[0047]
When the code shape is small and a large number of lines are gathered, the feature cannot be identified by the recognition method using the histogram. In order to identify by the method using the histogram, it is necessary to use a code having a size larger than the above-described size. If the code size is increased, the appearance of the form is affected. Therefore, the method using the histogram is stopped, and in this example, the recognition method using matching and the recognition method using partial scanning are tried. A standard pattern dictionary for matching is created by creating 69632 images in PostScript language, 17 for each of 4,096 types of code, using 1200 dpi prints and 600 dpi scans as samples, and taking the features. Created.
[0048]
FIG. 23 shows the result of code recognition. The recognition rate is a recognition rate for all samples used for creating a dictionary. The processing time for one form is the processing time until the correction interface is called. A recognition method based on partial scanning is better for both processing time and recognition rate.
[0049]
In FIG. ⁸ This indicates the dot texture entry frame created by the type code. 2 ¹² It is the same as the type code. If this code is too small, the combination lines come into contact with each other, and even humans cannot identify them. A size was created so that a person could be identified without touching the combination line, and the size was identified by a recognition method based on matching. The drawing area of this size is 0.78 × 0.78 mm. Considering the minimum code interval of 0.2 mm, the arrangement area is 0.98 × 0.98 mm. This size can be further reduced if the printing accuracy and the scanner accuracy are increased. FIG. 24 shows an example of an entry frame created by a code figure of this size. In this example, information is superimposed on the upper side of the entry frame.
[0050]
This code also needs to be larger than the above-mentioned size in order to identify features by a recognition method using a histogram. Therefore, a recognition method by matching and a recognition method by partial scanning were tried. A dictionary of matching standard patterns was created by making a total of 1792 images for each of 256 types of codes in the PostScript language, using images obtained by 1200 dpi printing and 600 dpi scanning as samples, and taking the characteristics.
FIG. 25 shows the recognition experiment results. The processing time for one form is the processing time until the correction interface is called. The recognition rate was the recognition rate for all samples used to create the dictionary, and both reached 100%. However, the processing time is faster with the recognition method based on partial scanning.
[0051]
2.4.3 Comparison of code shapes
FIG. 26 is a comparison diagram of recognition results of various code shapes. The decoding time per bit is embedded in the information bar from the time when the information bar of the entry frame is passed to the decoder engine based on the fastest recognition method for the various code shapes described above until the decoding result is obtained. It is the time divided by the amount of information. 2 ¹² Type and 2 ⁸ Kinds of codes are more difficult to cut and recognize in a realistic size than other codes. In addition, when the two types of code shapes are large, the difference from other codes is that a person can identify the shape and does not look like a code. Furthermore, if a form created with these two methods is processed by the labeling method, which is the fastest of the separation methods for handwriting and handwriting described in the following section, it cannot be separated from small characters (such as punctuation marks). Will adversely affect the recognition of characters. Although it is possible to separate the entry frame and the handwriting by other methods, the processing time for the separation becomes longer. On the other hand, the 8-direction line segment code has better information amount and decoding speed than the two types of large and small codes. Also, the two types of large and small codes are simple and easy to make a system.
[0052]
3. Main flowchart of active form processing
FIG. 27 shows a flowchart of the active form processing system. Below, each process of FIG. 27 is demonstrated. When the active form processing system starts processing, it first reads an active form image (data) (S101). Next, the active form processing system separates the entry frame and the handwriting, and detects the entry frame and the entry frame structure such as the position of the information bar (S103). The active form processing system reads the information superimposed for each entry frame according to the position of the information bar (S105). Details of the processes in steps S103 and S105 will be described later. The active form processing system stores data such as the detected position of the information bar and the entry frame structure in the storage unit at an appropriate timing.
[0053]
The active form processing system performs off-line character recognition on the handwritten character image in the entry frame according to the handwritten character attribute and the character type when the handwritten character written according to the superimposition information attribute is a character (S107). The active form processing system corrects the offline recognition result through the correction interface (S109). The correction interface executes processing for correcting the recognized character. The active form processing system outputs the description content and superimposition information extracted from the active form, for example, in the form of a CSV file (S111). This is called an instruction book. Further, the form image is stored in the storage unit in association with the instruction document. The active form processing system ends the active form processing. Thereafter, other application systems perform processing on the information extracted from the active form according to the instruction document.
[0054]
4). Processing of separation of entry frame and handwriting and detection of entry frame and entry frame structure
4.1 Process flow
FIG. 28 shows a flowchart for separating the entry frame and the handwriting, and detecting the entry frame and entry frame structure. The flowchart shown in FIG. 28 is a detailed process of step S103 in the flowchart shown in FIG. Even if you do not separate the entry frame from the handwriting, you can detect the line of the entry frame while including the handwriting (handwriting is in the way to detect the position in the line, but it is not in the way to detect the line). If line detection is performed before separation and only one line-by-line image is labeled (algorithm 2), the labeling process between lines can be omitted. The processing procedure is as follows.
[0055]
First, the active form processing system takes a histogram in the horizontal direction for all form images (data) and detects an image for each line (S201). Next, the active form processing system applies a labeling process to the line-by-line image to separate the entry frame and the handwriting (S203). In addition to the labeling process, morphological and median filter methods can be used to separate the entry frame from the handwriting. Labeling and the like will be described later.
Further, the active form processing system detects the position of the entry frame and the square position with a histogram in the vertical direction for the image of only the entry frame for each line (S205). The active form processing system detects the position of the information bar of the entry frame with a histogram in the horizontal direction with respect to the image for each entry frame (S207). Note that the active form processing system may consider a value equal to or less than a predetermined threshold as 0 when detecting a line and an entry frame, and ignore it as a blank even in a noisy place. In this case, an information loss prevention process at the time of detecting the entry frame, which will be described later, may be executed.
[0056]
4.2 Separation method of entry frame and handwriting
4.2.1 Labeling method
Labeling refers to attaching the same label to the connected components of black pixels connected to a binary image. There are two labeling methods: a method of calling a recursive function and a method of reciprocating scanning.
[0057]
4.2.2 Morphological methods
There are the following two methods for shrinking and expanding a form image, respectively.
As a contraction method, (a) a method in which an image is scanned and a white pixel is detected and a pixel in the vicinity of 8 or 4 is also whitened. (B) a pixel is scanned and a black pixel is detected. When the neighborhood or 4 neighborhood is examined, and at least one of the 8 neighborhood or 4 neighborhood pixels is white, any method of making this black pixel white can be used.
As a dilation method, (a) if the image is scanned at a black pixel, the pixel in the vicinity of 8 or 4 is also blackened. (B) if the image is scanned at a white pixel, the pixel 8 If the neighborhood or 4 neighborhood is examined and at least one of the 8 neighborhood or 4 neighborhood pixels is black, any of the methods of making the white pixel black can be used.
[0058]
4.2.3 Medinian filter method
The Medinan filter is a technique often used for image smoothing. The Medinian filter is a median value of density in a region, that is, if it is a 3 × 3 region, nine density values are arranged in order of low (or high), and the fifth (center) density value is It is a filter that uses a new density value at the center.
FIG. 29 is an explanatory diagram of the median filter. The original density value of the target pixel at the center is 22. Use 3x3 median filter. If the nine density values in the filter are arranged in the low order, the order is 22, 22, 23, 24, 24, 24, 25, 26, 27. Since the median is the fifth value 24, the new density value of the target pixel is 24. In the case of a binary image, for a connected component of black pixels having an area (number of black pixels) smaller than half the size of the median filter, the median value is the same regardless of which black pixel of the connected component is applied with the median filter. It is always a white pixel value. Therefore, this connected component is deleted.
[0059]
4.2.4 Separation of entry frame and handwriting
As the separation method of the entry frame and the handwriting, a separation method by labeling, a separation method by contraction and expansion, a separation method combining these, and a separation method by a median filter can be considered.
I tried to separate the entry frame and handwriting for the two-size code and the 8-direction line code. These sample forms are filled in with a pen of normal thickness in order to compare the handwriting separation method with several entry frames. As a result of the separation, a large amount of handwritten marks remained in the image of the entry frame by the separation method by contraction and expansion. It is necessary to fill in with a thicker pen to be able to separate the entry frame and handwriting cleanly. However, this increases the separation time due to shrinkage and expansion. Moreover, a writing instrument will be restrict | limited.
According to the result of the separation, the separation method by labeling that calls the recursive function is the fastest and there is no restriction on the handwritten thickness and the size of the entry frame dot. Therefore, a labeling method for calling a recursive function is effective as a method for separating an entry frame and a handwriting.
FIG. 30 shows separated entry frames and handwritten characters when the entry frame and handwriting overlap. It can be confirmed that the overlapping of the entry frame and the handwriting does not affect the handwriting separation.
[0060]
4.3 Detection of entry frame and entry frame structure
In order to detect the entry frame, there are a method using line segment detection and intersection detection (described later), a method of taking histograms in the vertical and horizontal directions as described below, and the like. The active form method is effective without depending on the entry frame detection method. However, in the following, since further technical development was performed by the method using the histogram, the detection of the entry frame using the histogram will be described.
[0061]
4.3.1 Detection by histogram
FIG. 31 is an explanatory diagram of detection of an entry frame and an entry frame structure using a histogram. By executing each process shown in FIG. 28, a histogram as shown in FIG. 31 is obtained.
First, as shown in FIG. 31A, a histogram in the horizontal direction is taken over the entire form image, and an entry frame is detected line by line according to the obtained histogram (corresponding to S201). Next, as shown in FIG. 31B, a histogram in the vertical direction is taken with respect to the detected row, and the position of the entry frame and the square of the entry frame are detected according to the obtained histogram (corresponding to S205). Next, as shown in FIG. 31C, a horizontal histogram is taken with respect to the detected entry frame, and the information bar position of the entry frame is detected according to the obtained histogram (corresponding to S207).
[0062]
4.3.2 Examination of detection method of entry frame structure
32 to 34 are explanatory diagrams of a method for detecting the entry frame structure.
As shown in FIG. 32, when the height and width of the entry frame are sufficiently large, the structure of the entry frame can be clearly distinguished by the histogram in the vertical direction and the horizontal direction. An intermediate value between the maximum value and the minimum value of the histogram is obtained, and this intermediate value is used as a threshold value to detect the square position of the entry frame and the position of the information bar. However, as shown in FIG. 33, when the width or height of the entry frame is small, the intermediate value cannot be obtained stably. 32 and 33, the size of the entry frame is enlarged and reduced for ease of viewing, so it is difficult to see the difference in the size of the entry frame at a glance, but from the number of codes arranged. You can see the difference in the size of the entry frame. Furthermore, an entry frame erased by overlapping with handwriting as shown in FIG. 34 may result in an incorrect result if it is processed as it is. Therefore, when identifying the structure of the entry frame, first, the dots of the entry frame are expanded as shown in FIG. Since it takes time to expand the dots in the entry frame until they completely contact each other, it may be expanded to some extent according to the dot size. An intermediate value of the histogram is obtained for the expanded entry frame, and the structure of the entry box can be easily identified. The entry frame with the two types of large and small codes can recognize the structure without performing the expansion process because the dot size is small. In the entry frame by the 8-direction line segment code, the dot texture is sparse depending on the size and shape of the dots. In order to recognize the entry frame structure from the histogram, it is preferable to perform an expansion process.
[0063]
4.3.3 Reduction of processing time considering blanks
FIG. 35 shows a method (algorithm 1) for processing six types of large / small two-type codes and eight-direction line segment codes without considering blanks in the form (algorithm 2). ). Since there is a space between the entry frames in the form, in the processing shown in the above-described flowchart, the processing time for the form is shortened by omitting the blank processing time for labeling.
The process of algorithm 2 is the same as the process according to the present embodiment shown in FIG. The processing procedure of Algorithm 1 is as follows. First, the entire form image is labeled to separate the entry frame and handwriting. Next, a line is detected with a histogram in the horizontal direction with respect to an image having only an entry frame. Further, the position and the mass position of the entry frame are detected by a histogram in the vertical direction with respect to the entry frame image for each line. The position of the information bar of the entry frame is detected by a histogram in the horizontal direction with respect to the entry frame image for each line. In Algorithm 1, all forms are labeled, whereas in Algorithm 2, only the detected lines are labeled. Regardless of the density of the entry frame, Algorithm 2, that is, a method that considers the space between lines is faster. Note that the accuracy of the processing results by the two algorithms is the same.
[0064]
4.4 Information loss prevention processing when detecting entry boxes
FIG. 36 is an explanatory diagram of information loss prevention processing when an entry frame is detected.
When detecting the position of the entry frame by the histogram in the horizontal or vertical direction, in order to avoid the hindrance of noise, a value equal to or lower than a predetermined threshold is regarded as 0 with respect to the result of the histogram, and even in a noisy place Ignore it as blank. In FIG. 36, there are many noises on the upper and outer sides of the entry frame. However, by performing threshold processing, the position f of the entry frame can be detected without being affected by the noise. However, since the value below the threshold value is regarded as 0, the information below the threshold value is also ignored at the boundary of the entry frame, which has an adverse effect on reading the information superimposed in the entry frame. Therefore, when detecting the position of the entry frame, in order to prevent such information loss, the position f of the entry frame is detected at a position below the threshold value in the histogram, and further, from this position f to the outside of the entry frame. The threshold value is a position where the histogram value is traced in the direction (for example, upward in the example of FIG. 36) and the histogram value has changed from a value greater than 0 to 0, or the length traced from the first position f. If the value of the histogram does not become 0 even if the value exceeds, the position shifted by the threshold value from the initial position f is set as the start or end position of the entry frame. This makes it possible to correctly detect the start or end position of the entry frame and avoid noise outside the entry frame. FIG. 36 shows an example of row detection (scanning up and down), but the same applies to column detection (scanning left and right).
[0065]
5. Decoding
5.1 Process flow
Next, the process (decoding) in step S105 in FIG. 27 will be described.
FIG. 37 shows a flowchart for decoding large and small two types of codes. The flowchart shown in FIG. 37 is decoding for a form in which information records are arranged in the horizontal direction of the information bar.
First, the active form processing system takes a histogram in the horizontal and vertical directions with respect to the information bar of the entry frame detected in step S103, and cuts out dots for each line (S301). Next, the active form processing system measures the length of the dot according to the longitudinal histogram of each cut out dot, and determines whether the information indicated by the dot is 1 or 0. For example, the active form processing system sets 1 for a long dot and 0 for a short dot. Then, the active form processing system returns the shift of the line by a predetermined bit, for example, 2 bits (S305). The active form processing system obtains a large amount of information records from the information bar of the entry frame (S307). For example, the active form processing system can obtain as many information records as possible from the information bar of the entry frame. The active form processing system may obtain a predetermined number of information bars. The active form processing system obtains the superposition information record by taking the majority of the values of the information record (S309). The error can be recovered by taking a majority vote. In the above description, the two types of large and small codes have been described, but the same can be applied to a form having other codes arranged in the horizontal direction.
[0066]
Next, decoding of the 8-direction line segment code will be described. In cutting out the 8-way line segment code, error prevention and detection are performed, and an efficient decoding method is performed. The size of the 8-direction line segment code is described as the above-described size. In addition, it is possible to perform decoding even when the drawing area has a size of 0.35 × 0.35 mm, 0.45 × 0.45 mm, 0.5 × 0.5 mm, and 0.6 × 0.6 mm by experiment. It has been confirmed. The larger the code size, the higher the recognition rate, but the smaller the code, the better the appearance of the form. When decoding 8-way line segment code, the required space between dots is 0.2 mm, and forms can be created with the desired code size. The code size is automatically detected and adjusted to this size. Decoding. Further, since the 8-direction line segment code uses an indefinite length information record by a delimiter and a parity code for each information record, an information record length error and an information record content error can be detected. Details of these processes will be described below.
[0067]
FIG. 38 shows a flowchart of decoding of the 8-way line segment code. The flowchart shown in FIG. 38 is for decoding a form in which a code, a parity code, and a delimiter are arranged in the vertical direction of the information bar.
First, the active form processing system sets an information record (37-01). For example, the information bar detected in step S103 or a part thereof is read from the storage unit. Next, the active form processing system obtains a delimiter at the beginning of the information record. Specifically, the following steps 37-02 to 37-06 are executed. First, the active form processing system determines whether it is the end of the image (37-02). For example, the active form processing system determines whether there is an image (data) to be cut out next. In the case of the end of the image (37-02), the active form processing system proceeds to the process of step 37-17. On the other hand, when the active form processing system is not the end of the image (37-02), the active form processing system cuts out the figure of the code (37-03). For example, the active form processing system determines the division of the histogram in the vertical direction and the horizontal direction, and cuts out the figure of the code based on the width. The active form processing system determines whether it is a clipping error (37-04). As the cut-out error, for example, it is conceivable that the code cannot be cut out for the reason that the code is deleted due to the overlap of handwriting, or that the cut-out code is not in a predetermined position and other codes are deleted. In the case of a cut-out error (37-04), the active form processing system proceeds to the process of step 37-14. On the other hand, if it is not a cut-out error (37-04), the figure of the cut-out code is recognized (37-05). The active form processing system determines whether the recognized code is the head delimiter (37-06). The active form processing system proceeds to the process of step 37-07 if the recognized code is the head delimiter, and considers it as an error when it is not the head delimiter, and proceeds to the process of step 37-14.
[0068]
In the process of steps 37-14 to 37-16, the next terminal delimiter is found. First, the active form processing system cuts out the figure of the next code (37-14). The active form processing system recognizes the cut out code (37-15). The active form processing system determines whether the recognized code is a terminal delimiter (37-16). Note that the same symbol can be used for the head delimiter and the end delimiter. If the recognized code is not the terminal delimiter, the active form processing system returns to Step 37-14. The active form processing system repeats steps 37-14 to 37-16 until it finds a delimiter. On the other hand, in the case of a terminal delimiter, the process proceeds to step 37-01. On the other hand, the active form processing system obtains the data of the information record when the code recognized in step 37-06 is the head delimiter. Specifically, the following steps 37-07 to 37-13 are executed.
[0069]
First, the active form processing system determines whether it is the end of the image (37-07). In the case of the end of the image (37-07), the active form processing system proceeds to the process of step S37-17. On the other hand, if it is not the end of the image (37-07), the active form processing system cuts out the graphic of the next code (37-08). The active form processing system determines whether it is a clipping error (37-09). In the case of a clipping error (37-09), the active form processing system proceeds to the process of step 37-14. On the other hand, if it is not a cut-out error (37-09), the figure of the cut-out code is recognized (37-10). The active form processing system determines whether the recognized code is a terminal delimiter (37-11). If the recognized code is not a terminal delimiter (37-11), the active form processing system stores the recognition result in the storage unit in association with the set information record (37-12), and the process of step 37-07 Return to. The active form processing system repeats the processes from 37-07 onward, recognizes the code until the end delimiter is found, and obtains data of the information record. On the other hand, when the recognized code is a terminal delimiter (37-11), the active form processing system ends the processing of one information record (37-13), returns to step 37-01, and further receives the next information. Perform processing on the record.
[0070]
In step 37-17, the active form processing system ends the process of obtaining multiple information superimposed on the information record, and proceeds to the process of step 37-18. The active form processing system checks the length of the information record and deletes the information record that is not within the possible range (for example, 45 to 207 bits in the example shown in FIG. 12) (37-18). The active form processing system obtains the length of the information record by taking the majority of the length of the information record (37-19). The active form processing system checks the length of the information record and deletes the incorrect information record (37-20). The active form processing system checks the parity code of the information record and deletes the incorrect information record (37-21). The active form processing system obtains the superimposition information record of the entry frame by taking the majority of the remaining information record values (37-22). Further, the active form processing system stores the obtained information record in the storage unit as appropriate, and ends the process.
[0071]
5.2 Detection of code size and number of code lines in information bar
FIG. 39 is an explanatory diagram for calculating the size of the drawing area. As an example, four lines of code are arranged in the horizontal bar shown in FIG. In order to cut out the code, the horizontal bar is used to detect the code size and the number of code lines in the information bar.
First, a vertical histogram is taken with respect to the horizontal bar, and the center position of each code graphic string is detected as shown in FIG. Next, the distance between the centers of each code figure sequence is obtained, and an average is obtained for these distances. This result is the side length of the square of the arrangement area of the code figure. If the size of the dot interval of 0.2 mm (4 pixels by 600 dpi scan) is subtracted from the side length of the arrangement area, the side length of the drawing area square can be obtained.
[0072]
The form in the present embodiment is set so that the number of code lines in the horizontal bar is the same as the number of code lines in the information bar of the entry frame. The number of code lines in the information bar of the entry box can be obtained. If the form is inclined, it is difficult to detect the number of code lines for the entire horizontal bar. Therefore, according to the vertical histogram taken for the horizontal bar when the code size is detected, the horizontal line histogram is taken for only several columns of code figures, so that the code line of the information bar of the entry frame The number can be detected. In addition to obtaining the number of code lines of the information bar by detecting the number of code lines of the horizontal bar, the information indicating the number of code lines is superimposed on the code constituting the horizontal bar, and the code is recognized. You may make it ask.
[0073]
5.3 Extraction of code figure
Next, extraction of code figures (for example, steps 37-03, 37-08, and 37-14 in FIG. 38) will be described.
[0074]
5.3.1 Necessity of a method for cutting out code figures by histogram
FIG. 40 shows an example of code cutout based on the code size.
If the size of the code can be detected, a method of cutting out the code with the code size as a fixed length can be considered. However, since there is some kind of error (for example, accumulation of small errors), there are cases where the code figure cannot be cut out correctly with the code size being a fixed length. As shown in FIG. 40, when a code for each column is cut out with the code size BC being a fixed length and the result of the cutting is simulated by drawing a vertical line, the first several columns can be cut out correctly. In the case of the back row, the error becomes large, so that it may not be cut out correctly. In the active form processing system according to the present embodiment, the code figure is cut out by using a histogram instead of cutting the code size as a fixed length, thereby realizing accurate code cutting. .
[0075]
5.3.2 Extraction direction of code figure
First, a horizontal histogram is taken with respect to the image of the information bar, and each line is cut out. Next, a vertical histogram is taken for the image for each row, and a figure for each code is cut out. In the case of an information bar having no inclination, a code figure can be easily cut out in this cutting direction. However, as shown in FIG. 41, in the case of an information bar scanned at an angle, a code figure cannot be easily cut out in this cutting direction. However, even in such a case, pay attention to the fact that the number of rows is small compared to the number of columns and the overlapping of the histograms in the vertical direction is small, and using this property, even if the information bar is slightly inclined, the figure for each code is correctly displayed. Can be cut out. For example, the active form processing system first takes a histogram in the vertical direction with respect to the image of the information bar, and cuts out each column from the left to the right of the information bar. Next, a horizontal histogram is taken for each column image, and a figure for each code is cut out.
[0076]
5.3.3 Detection and processing of information records with overlapping handwriting
Next, clipping errors (corresponding to 37-03, 37-08, and 37-14 in FIG. 38) will be described. Due to the overlap between the handwriting and the entry frame, the part overlapping the handwriting is erased in the separated entry frame. This is detected efficiently, and a method is adopted in which processing is not performed for information records with errors. In the extraction of the code figures 37-03 and 37-08 in FIG. 38, such an error is detected for the extracted code, and 37-04 and 37-09 in FIG. 38 to 37-14 in FIG. The process proceeds to 37-15 and 37-16, and the code is skipped until the next delimiter is found. In other words, no processing is performed for information records with errors. In the information bar image, the code figure for each column is cut out from the left to the right, and when the distance between the previous column and the next column is larger than the expected width, the code figure and the handwriting overlap, Judge that the code figure is missing.
[0077]
FIG. 42 is an explanatory diagram of detection and processing of information records in which handwriting overlaps. FIG. 42 is the leftmost part of the information bar of the entry, and the code drawing area is represented by a virtually small square. The dot texture of the information bar is formed by arranging the drawing areas uniformly with a minimum interval (4 pixels) of 0.2 mm. The side length BB of the drawing area square is obtained from the horizontal bar by the method described above. The position of the column center position S + the drawing area side length BB / 2 is the maximum right position R2 of the column, and the actual right position of the column vertical histogram is R1.
[0078]
When the code figure of each column is cut out, the distance d between the right position R1 of this column and the reference a is obtained, and whether or not there is a handwriting overlap error is determined according to the distance d. First, how to obtain the reference position a will be described. In the image of the information bar, the code figure for each column is cut out from the left to the right. Initially, the initial position of the reference position a is the leftmost position of the information bar. Each time the code figure for each column is cut out, the reference position a is updated. The new position of the reference position “a” is the center position S of the cut out column + the drawing area side length BB / 2. That is, the reference position a always points to the maximum right position R2 of the code string except for the initial value. Since the reference position a changes based on the center position of the column, the initial position may not be correct due to noise or the like. A specific process for detecting a handwritten overlap error according to the reference position a will be described below.
[0079]
First, every time a new code string is cut out, the active form processing system obtains a distance d between the right position R1 of this string and the reference a (for example, corresponding to step 37-03). Next, the active form processing system, when the distance d is larger than the drawing area side length BB by 6 pixels (drawing area interval 4 pixels + margin 2 pixels) or more (for example, corresponding to step 37-04), It is regarded as an overlap error (cutout error) and is ignored until a delimiter is found for the subsequent code figure (corresponding to steps 37-14 to 37-16). On the other hand, when the distance d is not 6 pixels or more larger than the drawing area side length BB, the active form processing system performs a process of cutting out and recognizing a column code (for example, corresponding to steps 37-05 and 37-10).
[0080]
5.3.4 Errors in code figures
FIG. 43 is a diagram illustrating an example of an error in a code figure.
As shown in FIG. 43, the code figure may come into contact with noise, and the shape and size may become abnormal. For example, there are a code figure horizontal size error due to horizontal connection, a code figure vertical size error due to vertical connection, a code figure vertical size error due to noise, and a horizontal code figure horizontal size error due to information bar position error. For these abnormalities, if the size of the cut out figure is smaller than the expected code figure, it is ignored as noise. This prevents errors such as attaching minute noises to figures and connecting them, or processing minute noises as code figures.
[0081]
When cutting out the code figure for each column, the width of the column is checked. If the width of the column is larger than the drawing area width BB + 4 (pixel), it is determined that the code figure has a horizontal size error, and the width from the reference position a to BB + 4 is determined. This column is divided and cut out. The 8-direction line segment code is composed of line segments. As described above, the line thickness is 0.1 mm, and the size in 600 dpi scan is about 3 pixels. Considering the error, the code line thickness threshold is set to 5 pixels, for example. Since the code figure can be recognized, the width BB of the drawing area is always larger than the thickness of the line. Therefore, both the width and height of the code cannot be smaller than 5 pixels of line thickness. When cutting out a figure for each code, check the width and length of the code. If both the width and length of the code are 5 pixels or less, it is judged as an error in the size and width of the code figure, and this figure is ignored. . If the length of the code figure is larger than the drawing area width BB + 4, it is determined that the code figure has a vertical size error, and the code figure is divided and cut out from the top of the figure by the width of BB + 4.
[0082]
By doing so, it is possible to prevent errors such as attaching minute noise to each code graphic and connecting them, or processing minute noise as a code graphic.
5.3.5 Correspondence to code figure damage due to position detection error of information bar
44 to 46 are explanatory views (1) to (3) of the processing for dealing with the code figure damage due to the position detection error of the information bar.
When detecting the position of the information bar in the entry frame, as shown in FIG. 44, for example, the information bar detected in step S207 described above and the boundary b of the other part are not in error with the correct information bar boundary. There may be. If the code figure is cut out and decoded in accordance with the information bar boundary line b, a part of the code figure protruding from the information bar is excluded and recognized, which may cause erroneous recognition. In order to avoid this, the active form processing system executes the code cutout process (corresponding to steps 37-03, 37-08, and 37-14 in FIG. 38) and the cutout error process as follows. Can do.
[0083]
In accordance with the detected information bar boundary line b, the active form processing system performs code from left to right on the information bar image, for example, for data from the top code to the boundary line b using a vertical histogram. Detect the position of each column. Next, as shown in FIG. 45, the active form processing system scans a horizontal histogram on the code image for each column and cuts out each code. As shown in FIG. 45, the cutting direction of the upper information bar of the entry frame is scanned from top to bottom. On the other hand, the cutting direction of the lower information bar scans from the bottom to the top. From this stage onward, the information on the boundary line b is not used.
[0084]
The active form processing system cuts out one code for each row, and then checks the number of cut out codes and the scanned length L in this row. If the number of code lines of the information bar obtained from the horizontal bar is N, if the number of codes in this column has already reached N and / or the scanned length L has reached a length corresponding to N , To finish cutting the code. Otherwise, the histogram is subsequently scanned and the code is cut out. Here, the purpose of detecting the end with the scanned length is to prevent the code figure from going over the area of the information bar when the code figure overlaps with the handwriting and is lost.
The active form processing system checks the number of codes for each column when the code segmentation is completed. If N has not been reached, it is regarded as an error in the number of codes (cutout error shown in FIG. 38), and is ignored until a delimiter is found for the subsequent code figure.
[0085]
By not using the boundary line b in the cutout stage of the code figure, the code figure protruding from the detected information bar boundary line can be correctly recognized. The result of cutting out the code figure by this method is shown in FIG. In FIG. 46, a code frame cut out for convenience is attached and displayed. The information bar at the top of the entry frame shows that the code is correctly cut out without being affected by the error of the boundary line b.
[0086]
5.4 Recognition of 8-way line segment code
FIG. 47 is an explanatory diagram for the recognition of the 8-direction line segment code.
Next, code figure recognition (corresponding to 37-05, 37-10, and 37-15 in FIG. 38) will be described.
If the code size can be detected, a decoding method adapted to the code size can be performed. The side length of the code drawing area square is BB. A histogram in four directions is taken for the code figure. As shown in FIG. 47, the widths of the histograms in the four directions are separately set to l, x, s, and h. In consideration of errors such as errors, the code line thickness threshold is set to 5 pixels.
[0087]
FIG. 48 shows an algorithm for code recognition. FIG. 49 shows an example of a program for recognizing codes.
First, the active form processing system determines whether h ≦ 5 (S501). The active form processing system sets code = 0 when yes, and when no, determines whether h ≧ BB-2 (S502). The active form processing system moves to step S503 when yes and moves to step S508 when no. The active form processing system determines whether x ≧ BB-2 (S503). The active form processing system moves to step S504 when yes and moves to step S506 when no. The active form processing system determines whether s ≦ 5 (S504). The active form processing system sets the code to 2 when yes and determines whether l ≦ 5 when no (S505). The active form processing system sets the code = 6 for yes and the code = 8 for no.
[0088]
In step 506, the active form processing system determines whether x> 5 (S506). The active form processing system sets the code to 4 when no, and determines whether s <BB-2 when yes (S507). The active form processing system sets the code = 3 for yes and the code = 5 for no.
In step 508, the active form processing system determines whether s <BB-2 (S508). In the active form processing system, the code is 1 when yes and the code is 7 when no.
In addition to the above determination, an appropriate determination can be made according to the size of the code and the thickness of the line. In addition to using the four-way histogram, the code can also be recognized by the partial scanning and matching described above.
[0089]
5.5 Detection and processing of information record length errors
Next, detection and processing of information record length errors (corresponding to 37-18, 37-19, and 37-20 in FIG. 38) will be described.
The 8-direction line segment code can detect the length error of the information record by the delimiter. As described in the above coding method, the length of the information record of the 8-way line segment code is indefinite, but the number of code figures including the parity symbol is 16 to 70 (the information record is 45 ~ 207 bits). When recognizing a code figure, if the delimiter is recognized incorrectly, the length of the information record may not be between 16 and 70. In that case, the information up to the next delimiter is skipped and these information records are deleted (step 37-18).
[0090]
When all the codes in the information bar are processed and recognized, a plurality of records are obtained. If the lengths of these records are not all the same, a majority decision of each length is taken, and the maximum number of records is set to the correct length (corresponding to steps 37-19). Also, the length of the information record is checked, and the information record whose length is different from the length obtained by the majority vote is deleted. The next parity check is performed on the remaining information records.
[0091]
5.6 Checking parity code
The parity code is checked for the information record (corresponding to 37-21 in FIG. 38). For example, for each information record, it is checked whether or not the number of 1s with the same number of bits is an even number. If it is, the information record is correct. If it is not correct, the information record is determined to be incorrect and the information record is deleted.
[0092]
5.7 Acquisition of entry frame overlay information record
A majority vote is taken for each bit for the information record obtained by the method described above (corresponding to 37-22 in FIG. 38). By doing so, it is possible to obtain data with strong certainty for each entry frame.
[0093]
6). How to increase email address recognition rate
FIG. 50 shows an entry frame for increasing the recognition rate of mail addresses.
The mail address notation differs depending on the mail server, and the mail address character type is also different, but the commonly used mail address format is the domain address format, and the character type is "@", alphanumeric characters, "." (Period) , “−” (Hyphen), “_” (underscore). In the active form processing, the recognition rate can be increased by limiting to these character types when the handwritten content is an email address according to the information superimposed on the entry frame.
[0094]
However, these character types generally have a low recognition rate in off-line character recognition, and the recognition rate of mail addresses depends on this recognition rate. Even if the character types are limited, English uppercase letters, lowercase English letters and numbers affect each other and tend to be misrecognized. Also, since the e-mail address is a character string that does not make sense, context processing that increases the recognition rate cannot be performed. This is also the reason why the e-mail address recognition rate cannot be increased. However, it is very important for the form processing system because there is a risk of making a big mistake if the mail address is recognized incorrectly. In e-mail address misrecognition, the main cause is misrecognition of similar characters. If this misrecognition can be avoided, the recognition rate can be increased.
[0095]
As shown in FIG. 50, in order to limit the character type of the mail address, for example, another entry frame is provided under the mail address entry frame in the active form. In this box, enter the corresponding character type for the email address above. Considering the ease of recognition and the ease of entry by people, for example, the following symbols are entered.
(1) ￣: English capital letters (allows you to fill in a frame)
(2) _: English lower case letters (can be overlaid with a frame)
(3) =: Number (allows you to fill in a frame)
(4) Blank: Other
[0096]
These codes can be easily and reliably recognized by taking a histogram in the vertical direction and the horizontal direction, or matching three kinds of symbols in the frame. By doing so, it is possible to avoid the mutual influence between the character types of the mail address, and the recognition rate is increased. As shown in FIG. 50, these symbols can be entered across successive squares. As a result, it is possible to write in character string units instead of character units, so that instructions are simplified. You only need to look inside the square for recognition. The information record of the entry box for limiting the character type can include appropriate data indicating that the above-described code for enhancing the recognition of the mail address is entered as the character type.
[0097]
7). Prototype of active form processing system
An active form processing system was prototyped using the technology described above. The outline will be described below.
[0098]
7.1 Start and form reading
When the active form system is started up, menus such as [imageScan], [file], [imageSize], and [Setting], an execution button, and the like are displayed. For example, the active form processing system reads a form through a scanner by operating an [imageScan] menu. The active form processing system can also open an existing form image from the [file] menu. The active form processing system can display the read form image. Further, the display size of the image can be changed through the [imageSize] menu. Set the application to be called after the form processing system is finished from the [Setting] menu. The progress of the form processing is displayed through the progress bar. When the image is read and displayed and the execution button is pressed, the active form processing system starts processing.
[0099]
7.2 Separation of entry frame and handwriting ・ Detection of entry frame position and entry frame structure
FIG. 51 is an example of the processing flow of the active form processing system.
After reading the form, the active form processing system detects the entry frame and the entry frame square position, and separates the entry frame and the handwriting (52-1 in FIG. 51). The active form processing system obtains a frame-only image and a hand-written image for each entry frame, and classifies the entry frame into an empty entry frame, a horizontal bar, a handwritten entry frame, and noise (52-2 in FIG. 51). For example, if the number of black pixels of an image with only handwriting in the entry frame is less than a certain value, it is determined that the entry frame is empty. A vertical histogram is taken for an image with only an entry frame, and if the fluctuation in the vertical histogram is small and not more than a certain value, it is determined as a horizontal bar. If the width and height of the entry frame are small and less than a certain value, it is determined that an error has occurred. The other entry frames shall be handwritten entry frames. Next, the active form processing system takes a horizontal histogram for the frame image of each entry frame, and positions the upper and lower frames where information such as handwritten character attributes, character types, instructions, and headings are embedded. Is detected (52-3 in FIG. 51).
[0100]
7.3 Superimposition information reading
For each entry frame, the active form processing system passes the position information of the upper and lower frames in which information is embedded to a decoding engine that recognizes the overlay information, and decodes the overlay information in the entry frame (52 in FIG. 51). -4). The handwritten attributes embedded in one entry frame include, for example, types such as an address, an e-mail address, an organization name, a person name, a job title, an amount, a number, a date and time, a mark, a free handwriting, a check, and a figure.
[0101]
7.4 Character separation and off-line character recognition
The active form processing system classifies the entry frame into check, figure / mark, mail address, and other types of handwritten characters according to the handwritten attribute included in the decoded superposition information (52-5 in FIG. 51). The active form processing system separates each other handwritten character into characters and performs character recognition according to the type of handwriting for each entry frame included in the superimposed information (52-9, 52-10 in FIG. 51, 52-12, 52-13). Thereby, the character recognition rate can be increased. In the case of other handwritten characters, the active form processing system determines whether to apply the context processing according to the attributes of the handwritten characters. When the attribute is an organization name, job title, free writing, or address, context processing is performed (52-11 in FIG. 51). Context processing is not performed for a person name, amount, number, date and time of day. The active form processing system detects the type of each character separately for each e-mail address, and performs character recognition according to this type to increase the character recognition rate (52-6 in FIG. 51). 52-7, 52-8).
[0102]
7.5 Correction of handwritten character offline recognition results
Next, the active form processing system corrects the offline recognition result through the correction interface.
[0103]
7.6 Completing the order and saving the form image
When the active form processing system finishes correcting the result of offline recognition, it outputs the content extracted from the active form in the form of, for example, a CSV file. The output file is called a command (command data). An application that is called after the active form processing system is finished performs processing on the form in accordance with the instruction. The form image is saved in the same folder as the instruction document.
FIG. 52 shows the format of the output file. The instruction sheet represents information records for each entry frame on a line, separated by commas. The first line shows the contents of each column. [Name] is a heading, [property] is an attribute, [contents] is handwritten content, [action] is a command, and [character] is a character type. The second and subsequent lines represent information records for each entry frame. When handwriting is not written in the entry frame, nothing is output like “,,” in the column corresponding to [contents].
[0104]
8). Evaluation experiment
8.1 Experimental data and experimental environment
Print 2 sheets of A4 size large and small 2 types of code and 2 forms of 8 direction line code at 1200 dpi and ask 10 people to write them one by one. A sample and a sample of 20 8-way line segment codes were collected. These forms were scanned at 600 dpi and 8-bit gray level, and a preliminary evaluation of the active form processing system was performed. FIG. 53 is a form based on the 8-direction line segment code used in the evaluation experiment. The same applies to forms of two types of large and small codes.
[0105]
As an experimental environment, a PC having a 2.26 GHz CPU of Pentium (registered trademark) 4 was used. In the case of collecting handwritten character data without an off-line frame, because it is experimental data, the method of collecting data for on-line handwritten character experiment was used, and the following conditions were set when writing data. (1) Character size and character spacing are free. (2) It is impossible to continue between characters.
In the case of two types of large and small code forms, only the character recognition with a frame is possible in order to finish the system easily, and further, the character type limitation for each character is not provided to increase the recognition rate of the mail address. In the case of an 8-direction line segment code form, character recognition without a frame is also possible, and the character type for each character is limited to the mail address.
[0106]
8.2 Evaluation items for off-line handwritten character recognition
Frameless recognition is a process of recognizing a handwritten character string pattern as a “character string”. However, when evaluating the recognition rate of “character strings”, it is necessary to clarify the scale of evaluation. For example, as a result of recognizing a character string of about 20 characters per line, when a case of “only one character was misrecognized” and a case of “most misrecognized most characters” occurred, both were simply regarded as “recognition failure” on one line. It is not fair to evaluate. For this reason, an evaluation item for online handwritten character recognition without an online frame is used, and the following evaluation items are provided.
[0107]
8.2.1 Character recognition rate
The “character recognition rate” expressed by the following formula is adopted as an evaluation item.
Character recognition rate = number of correctly recognized characters / number of correct characters
FIG. 54 is an explanatory diagram of a method for calculating the recognition rate in character units. For example, if a result of “Sunday is clear” is returned for the input pattern “Tomorrow is sunny”, in this case, the character that has been correctly recognized is only one character “Sunny”. Since the correct number of characters is 5, the recognition rate in character units is calculated as 1/5 = 0.20 = 20%.
[0108]
8.2.2 Normal division rate
Furthermore, an evaluation item related to the accuracy of character division (referred to as “normal division ratio”) is also added. The normal division rate is defined by the following equation.
Correct division rate = 1- (number of character division failures / number of correct characters)
In the example of FIG. 54, there are the following two character division failure locations. (1) The place where “Ming” is originally divided into “Month” and “Day”. (2) Originally, the two characters of “day” and “ha” are attached to one character. Therefore, the normal division rate is calculated as 1-2 / 5 = 60%.
[0109]
8.3 Experimental results
8.3.1 Two types of codes
FIG. 55 shows the experimental results of two types of large and small code forms. The average processing time is the average processing time until the form is recognized and the correction interface is called. In the form sample, wrongly written characters (3 characters in 20 form samples) and characters not included in 4443 character types in the offline recognition dictionary (1 character in 20 form samples) are evaluated. I can't put it inside.
The recognition rate was improved by 33.98% from 59.74% to 93.72% by limiting the character type of the recognition category according to the type of handwritten character read from the entry frame for the cut handwritten character. Since the recognition rate of handwritten characters includes the recognition rate of email addresses, it depends on the recognition rate of email addresses. In addition, of the 20 forms, two forms are messy and there are characters that cannot be identified by humans, which causes a reduction in the character recognition rate.
[0110]
The processing time from reading a form to passing it to the correction interface is less than 4 seconds for a complex form and less than 2 seconds for a simple form in this experimental environment. Within the range of this experiment, the detection rate of the entry frame position reached 100%. As the success rate of decoding the superimposition information record, a large and small two-type code with a small amount of information and a simple code superposition method were adopted, and each data code error check was not provided, so that a recognition rate of 100% could not be obtained.
[0111]
8.3.2 8-way line segment code
FIG. 56 shows the experimental results of the 8-direction line segment code. In the case of an 8-way line segment code form, there is no frame-based handwritten character recognition and character type limitation for each character of the mail address. Therefore, in handwriting character recognition rate item evaluation, there are handwritten characters with a frame other than the mail address, mail address, And it evaluates by three classifications, a handwritten character without a frame. The average processing time is the average processing time until the form is recognized and the correction interface is called.
In the form sample, wrongly written characters (2 characters in 20 form samples) are not included in the evaluation. Furthermore, an error was attached to the form image by scanning in one form, so that it was recognized as a frame with characters even though it was a blank entry frame. However, since the correction was made by the correction interface, this is not included in the evaluation of the handwritten character recognition rate.
[0112]
By limiting the character type of the recognition category according to the type of handwritten character read from the entry frame, the recognition rate of the handwritten character with a frame other than the mail address is increased from 86.78% to 98.85% 12.07. % Improved. In the mail address, the recognition rate improved by 16.96% from 97.15% to 80.19% using the attribute by limiting the character type for each character. The 10th-ranked cumulative recognition rate of the mail address is almost the same regardless of whether or not the characters are limited, but the first-rank determination rate is drastically lowered when the character type is not limited. This seems to be due to the mutual influence of similar characters as the cause of the decline in the e-mail address recognition rate.
[0113]
The processing time from reading the form to passing it to the correction interface is about 4 seconds for the complicated form A and about 3 seconds for the form B in this experimental environment. Within the range of this experiment, the detection rate of the entry frame position reached 100%. As for the success rate of decoding the superimposed record, an 8-way line segment code with a large amount of information and an efficient code superimposing method were adopted and each data error check was provided, so that a recognition rate of 100% was obtained. In the case of frameless handwritten characters, a recognition rate of 97.71% in character units and a 98.47% normal division rate were obtained.
[0114]
9. Line segment detection, intersection detection
Further, methods for recognizing a form structure from detection of a line or an intersection without using a histogram include methods described in Non-Patent Documents 4 and 5, for example. The outline of line segment detection and intersection point detection in the present embodiment will be described below. The following processes are executed by the active form processing system (processing unit).
[0115]
9.1 Form representation model
Since most forms include three types of line segments (horizontal, vertical, and diagonal), the form structure is expressed by these three types of line segments. One line segment is represented by the coordinates of the start point and end point. In the form according to the present embodiment, the entry frame, horizontal bar, etc. are composed of dots or code figures, and these dots are expanded and connected to form a line segment. Horizontal, vertical, diagonal lines Find a segment.
Also, the horizontal line segment is sorted in the direction from left to right from top to bottom according to the coordinates of the leftmost starting point. That is, first, sorting is performed in the direction from top to bottom according to the Y coordinate of these start points. Then, the horizontal line segments having the same Y coordinate are sorted in the direction from left to right according to the X coordinate of the starting point. Similarly, for vertical line segments, first sort from left to right according to the X coordinate of the top starting point, and then from top to bottom according to the Y coordinate for the vertical segment of the same X coordinate starting point. Sort. The diagonal line segment is the same as the horizontal line segment, but sorts using the top starting point instead of the leftmost starting point. Note that the sorting direction can be any appropriate direction.
[0116]
The form expression model is composed of, for example, the following five parts.
(1) (NH, NV, NS): NH, NV, NS represent the number of horizontal, vertical and diagonal line segments, respectively. (2) (SLH, SLV, SLS): SLH, SLV, SLS represent a list of horizontal, vertical and diagonal line segments, respectively. (3) [(X _min , Y _min ); (X _max , Y _max )]: The top left and bottom right coordinates of the square area including all line segments. (4) [(XD (i) _min , YD (i) _min ); (XD (i) _max , YD (i) _max )]: The upper left and lower right coordinates of the square area of the i-th entry frame. (5) [(XN (i) _min , YN (i) _min ); (XN (i) _max , YN (i) _max )]: The upper left and lower right coordinates of the square area of the i-th name frame.
[0117]
9.2 Line segment extraction and adjustment
Here, a process until the line segment is extracted from the form image (data) will be described.
First, a form image (data) is input and tilt correction is performed. This rotates the image so that the longest possible line segment near horizontal is exactly horizontal. A vertical line may be used. Further, attention may be paid to only one line, or a plurality of lines may be focused, and they may be rotated so as to be horizontal or vertical on average.
An entry frame and handwriting are separated from the form image of the active form, and an image of only the entry frame is obtained. For the separation of the entry frame and the handwritten frame, the same method as described above such as labeling can be used. Expansion processing is performed until a small dot is connected to an image having only an entry frame. Then, three types of line segments (horizontal, vertical, and diagonal) are extracted.
For the extracted line segment, the horizontal line segment is the same when the Y coordinates of the two end points are different within the predetermined range, and the vertical line segment is the case where the X coordinates of the two end points are different within the predetermined range. Can be corrected by taking the average of each coordinate so that they are the same. Thus, for example, when there is a slight shift in coordinates, it can be handled as a horizontal or vertical line segment without being treated as an oblique line segment. Next, sorting is performed as described above.
[0118]
For example, the following correction can be performed on the sorted line segments.
(1) Concatenation of broken line segments: After extracting and sorting line segments, the line segments are divided into three groups: horizontal, vertical, and diagonal. Concatenation processing is performed on line segments cut by scanning or the like. In a group of horizontal line segments, if two line segments having substantially the same Y coordinate are closer than 0.5 cm, for example, the two line segments are connected. The same applies to vertical line segments and diagonal line segments.
(2) Delete small line segment: If the length of the horizontal line segment is shorter than 85% of the minimum distance between the vertical line segments, for example, the horizontal line segment is regarded as a stroke line and deleted. For example, if the length of the vertical line segment is shorter than 85% of the minimum distance between the horizontal line segments, the vertical line segment is regarded as a stroke line and is deleted.
(3) Correction of line segment: If the distance from the end point of the vertical line segment to the horizontal line segment is shorter than 0.5 cm, the vertical line segment is corrected so as to touch the horizontal line segment. The same applies to horizontal line segments and diagonal line segments.
In addition, the numerical value used for the above-mentioned judgment can be made into an appropriate numerical value.
[0119]
9.3 Detection of name and entry frames
Next, processing for detecting a name frame and an entry frame based on the extracted line segment will be described.
An ordinary form contains an entry frame, a name frame, and a mixed frame. The entry frame is a frame for receiving data. The name frame is a frame including text such as explanation and guidance. The mixed frame serves as both an entry frame and a name frame. In the frame detection algorithm described below, the input data is the sorted horizontal and vertical line segment lists SLH and SLV, and the output data is the square area data (entry frame data) of the entry frame and the name frame / mixed frame. It is.
(Step 1) Detect all contact points, intersections and end points for horizontal and vertical line segments. For each horizontal line, a vertical line that touches or intersects the horizontal line is detected, and the coordinates of these contact points and intersections are calculated. If there is no vertical line intersecting the contact point at the end point of the horizontal line, the end point of this horizontal line is set as the end point. Note that if the frame is composed of a closed figure like a rectangle, the end points will not appear basically, but the line segments that make up the frame may be cut off due to, for example, image errors or noise. Terminal points may appear. In addition, there are cases in which a form or the like is entered on the upper side or lower side of a line segment instead of a frame.
[0120]
Separation of contact points or intersections is meaningless, and the respective contact points, intersections, and end points are classified into four types of types 1 to 4 as shown in FIG. 57, for example, based on the positional relationship between horizontal lines and vertical lines. The three contact points and intersections shown in Type 1 indicate, for example, that the point is on the upper side of the frame. The point shown in Type 2 indicates that it is on the upper side or the lower side of the frame, for example. The point shown in Type 3 indicates that it is on the lower side of the frame, for example. Type 4 indicates an end point. An entry box is detected according to these types. For example, the frame can be detected by appropriately finding two points of type 1 (indicating the upper two vertices of the frame) and two points of type 3 (indicating the lower two vertices of the frame).
All points are sorted in order from top to bottom and from left to right. This is because the line segments are sorted as described above. First, the pointer used in the following process is set to the first sorted point.
[0121]
(Step 2) The point A pointed to by the pointer and the point B next to the point A are examined, and the following processing is executed to detect and register (store) the frame. (1) If the Y coordinates of A and B are different, jump to step 4. (2) If both A and B belong to type 4, scan from AB line segment upward until a horizontal line segment or character string of black pixels is found. If no black pixel is found, the top of the image is reached. In either case, a square area with the line AB as the bottom is obtained. If the height of this square area is smaller than 0.4 cm, for example, it is ignored and jumps to step 4. Otherwise, it is registered (stored) as one frame.
(3) If both A and B belong to Type 1 or Type 2, the first to form a square (or rectangular, the same applies hereinafter) ABCD belonging to Types 2 and 3 from B and later in the sorted list Find points C and D. If the points C and D are found, the square ABCD is registered as a frame. If not, jump to Step 4. (4) If A belongs to type 4 and B belongs to type 1 or type 2, points C and D are found similarly. C is type 4, D is type 2 or type 3, and ABCD is square. If the points C and D are found, the square ABCD is registered as a frame. If not, jump to Step 4.
(5) If A belongs to type 1 or type 2 and B belongs to type 4, points C and D are found similarly. C is a type 2 or type 3, D is a type 4 and ABCD is square. If the points C and D are found, the square ABCD is registered as a frame. If not, jump to Step 4. (6) If all the above conditions are not satisfied, jump to Step 4.
[0122]
(Step 3) The square area of the frame obtained in Step 2 is scanned. If the number of black pixels in the square area is equal to or greater than a predetermined threshold, the square area is labeled as a name area / mixed area square area. Otherwise, label it as an entry box.
(Step 4) Add 1 to the pointer and point to the next point. If the pointer points to the last point, the process ends. Otherwise jump to step 2.
[0123]
【The invention's effect】
According to the present invention, it is possible to read a form used in a window or an office, and a form and form processing method, a form processing program, a form processing program, a recording medium on which the program is recorded, and a form that can be easily separated from a form entry frame and entry characters. A processing device can be provided. In addition, according to the present invention, the recognition rate of written characters can be increased. Furthermore, according to the present invention, the processing method of the form itself can be described not on the apparatus side but on the form side, a general-purpose machine can be used as a reading apparatus, and various forms can be processed on a single machine.
Further, according to the present invention, it is possible to provide a form having a dot shape and a form processing method for increasing the amount of information to be superimposed. According to the present invention, it is possible to provide a coding method and a decoding method for stably reading superimposition information from handwriting overlap and contamination. Furthermore, according to the present invention, it is possible to provide an auxiliary entry method such as alphanumeric symbols that are particularly easily misread. Further, according to the present invention, it is possible to provide a form that can be easily created by an editor and printed by a printer. Further, according to the present invention, the trouble of scanning the image between the lines of the entry frame can be omitted, and the processing time can be shortened.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an active form.
FIG. 2 is a hardware configuration diagram of an active form processing system.
FIG. 3 is a configuration diagram of a code using two types of large and small graphics (two types of large and small codes).
FIG. 4 is a configuration diagram of a code (8-direction line segment code) using 8-direction line segments.
FIG. 5 shows a relationship between a drawing area and an arrangement area.
FIG. 6 ¹² Code that uses different types of figures (2 ¹² Type code) configuration diagram.
FIG. 7 ⁸ Code that uses different types of figures (2 ⁸ Type code) configuration diagram.
FIG. 8 is a diagram showing information records to be superimposed.
FIG. 9 is a table showing correspondence between attributes and values indicated by codes.
FIG. 10 is a table showing correspondence between character types and instructions and values indicated by codes.
FIG. 11 is a diagram illustrating a method for coding an information record in a dot row.
FIG. 12 is a diagram showing information records to be superimposed.
FIG. 13 shows an example of superposition of an 8-direction line segment code.
FIG. 14 is an explanatory diagram of a parity code setting method.
FIG. 15 is a diagram showing a delimiter.
FIG. 16 is an explanatory diagram of a recognition method using a histogram in four directions.
FIG. 17 is an explanatory diagram of extraction of standard pattern samples and feature values of input codes.
FIG. 18 is an explanatory diagram of code recognition by matching with a standard pattern.
FIG. 19 is an explanatory diagram of a recognition method by partial scanning.
FIG. 20 is a dot texture entry frame made up of two types of large and small codes and an 8-direction line segment code.
FIG. 21 shows a result of a recognition method based on a four-way histogram and matching.
FIG. 22 ¹² An entry frame for the dot texture created by the type code.
FIG. 23 shows code recognition results.
FIG. 24 ⁸ An entry frame for the dot texture created by the type code.
FIG. 25 shows recognition experiment results.
FIG. 26 is a comparison diagram of recognition results of various code shapes.
FIG. 27 is a flowchart of an active form processing system.
FIG. 28 is a flowchart for separation of an entry frame and handwriting, and detection of an entry frame and an entry frame structure.
FIG. 29 is an explanatory diagram of a median filter.
FIG. 30 shows a separated entry frame and handwritten characters when the entry frame and handwriting overlap.
FIG. 31 is an explanatory diagram of detection of an entry frame and an entry frame structure using a histogram.
FIG. 32 is an explanatory diagram (1) of a method for detecting an entry frame structure.
FIG. 33 is an explanatory diagram (2) of a method for detecting an entry frame structure.
FIG. 34 is an explanatory diagram (3) of a method for detecting an entry frame structure.
FIG. 35 shows a method that does not consider blanks in a form and a processing result obtained by applying the considered method.
FIG. 36 is an explanatory diagram of information loss prevention processing when an entry frame is detected.
FIG. 37 is a flowchart of decoding of two types of large and small codes.
FIG. 38 is a flowchart of decoding of an 8-direction line segment code.
FIG. 39 is an explanatory diagram of calculating the size of a drawing area.
FIG. 40 shows an example of code extraction based on code size.
FIG. 41 is an explanatory diagram of a cutting direction for cutting out a figure for each code from the information bar of the entry frame.
FIG. 42 is an explanatory diagram of detection and processing of an information record in which handwriting is overlapped.
FIG. 43 is a diagram showing an example of an error in a code figure.
44 is an explanatory diagram (1) of processing for dealing with code figure damage due to an information bar position detection error; FIG.
FIG. 45 is an explanatory diagram (2) of a process for dealing with a code figure damage caused by an information bar position detection error;
FIG. 46 is an explanatory diagram (3) of processing for dealing with code figure damage caused by information bar position detection error;
FIG. 47 is an explanatory diagram of recognition of an 8-direction line segment code.
FIG. 48 shows an algorithm for recognizing a code.
FIG. 49 shows a program example for recognizing a code.
FIG. 50 is an entry frame for increasing the e-mail address recognition rate.
FIG. 51 shows an example of the processing flow of the active form processing system.
FIG. 52 shows an output file format.
FIG. 53 is a form used for an evaluation experiment.
FIG. 54 is an explanatory diagram of a method for calculating a recognition rate in character units.
FIG. 55 shows experimental results of two types of large and small code forms.
FIG. 56 shows an experimental result of an 8-direction line segment code.
FIG. 57 is a diagram showing classification of contact points, intersections, and end points.
[Explanation of symbols]
10 Horizontal bar
20 entry frame
30 Information Bar
40 trout
1 processing section
2 Form input part
3 Output section
4 display section
5 storage unit

Claims (34)

A form input step for inputting form image data including entry frame data composed of a code in which information is expressed by an inclination of a line segment or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
Cutting out each code of the detected information bar;
A step of obtaining a histogram of four directions of the cut code in the vertical direction, the horizontal direction, the right diagonal direction, and the left diagonal direction;
A form processing method including a code recognition step of determining a shape characteristic of a code based on a obtained histogram in four directions and recognizing the code. A form input step for inputting form image data including entry frame data composed of a code in which information is expressed by an inclination of a line segment or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and handwritten data are separated, and the horizontal and vertical histograms or line segment detection and / or intersection detection for the form image data, An entry frame detection step for detecting entry frame data and an information bar in the entry frame data in which predetermined information is superimposed by a code;
Cutting out each code of the detected information bar;
A step of obtaining a histogram of four directions of the cut code in the vertical direction, the horizontal direction, the right diagonal direction, and the left diagonal direction;
A form processing method including a code recognition step of determining a shape characteristic of a code based on a obtained histogram in four directions and recognizing the code. A form input step for inputting form image data including code data in which information is expressed by a dot width, a slope of a line segment, or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
A superimposition information recognition step of recognizing each code of the information bar detected in the entry frame detection step and extracting superimposition information;
A character recognition step of performing character recognition according to the attribute and / or character type of the handwritten data indicated by the extracted superimposition information for the handwritten data separated in the entry frame detecting step;
A command creation step for creating command data for causing other application systems to perform predetermined processing based on the character information recognized in the character recognition step and the superimposition information extracted in the superimposition information recognition step;
Form processing including outputting the instruction form data created in the instruction form creation step and / or storing the form image data inputted in the form input step and the instruction form data in association with each other Method. A form input step for inputting form image data including code data in which information is expressed by a dot width, a slope of a line segment, or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and handwritten data are separated, and the horizontal and vertical histograms or line segment detection and / or intersection detection for the form image data, An entry frame detection step for detecting entry frame data and an information bar in the entry frame data in which predetermined information is superimposed by a code;
A superimposition information recognition step of recognizing each code of the information bar detected in the entry frame detection step and extracting superimposition information;
A character recognition step of performing character recognition according to the attribute and / or character type of the handwritten data indicated by the extracted superimposition information for the handwritten data separated in the entry frame detecting step;
A command creation step for creating command data for causing other application systems to perform predetermined processing based on the character information recognized in the character recognition step and the superimposition information extracted in the superimposition information recognition step;
Form processing including outputting the instruction form data created in the instruction form creation step and / or storing the form image data inputted in the form input step and the instruction form data in association with each other Method. The entry frame detection step includes:
A line detection step for taking a horizontal histogram for the input form image data and detecting the form image data for each line based on the histogram;
A separation step for separating entry frame data and handwritten data for the detected form image data for each line;
Taking a histogram in the vertical direction for the entry frame data for each separated line, and detecting one or more entry frame data based on the histogram;
The form processing method according to any one of claims 1 to 4, further comprising: taking a histogram in the horizontal direction with respect to the detected entry frame data and detecting an information bar of the entry frame data based on the histogram. . The entry frame detection step includes:
Separating the input frame data and handwritten data by any of the labeling process that attaches the same label to the connected components of the connected black pixels, the morphology that contracts and expands the image, and the median filter that has the effect of smoothing the image The form processing method according to any one of claims 1 to 5. The row detection step includes:
The form processing method according to claim 5, wherein when the value of the histogram is equal to or less than a predetermined threshold value, the influence of noise is removed by ignoring the value. The entry frame detection step includes:
Taking a horizontal histogram and / or a vertical histogram for the input form image data;
If the value of the histogram is less than or equal to a predetermined threshold, ignoring the value to remove the effect of noise;
Scanning the histogram, and detecting a position where the value of the histogram falls from a predetermined value or less to a predetermined value or a position where the value of the histogram falls from a predetermined value or more to a predetermined value or less;
From the detected position, the value of the histogram is traced in the direction opposite to the direction in which the histogram was scanned or the direction outside the row or column, and the position where the histogram value is zero or less than the predetermined value, or from the detected position If the value of the histogram does not become zero or less than the predetermined value even if the traced length exceeds a predetermined threshold value, the threshold value is opposite to the direction scanned from the detected position or in the direction outside the row or column. A step where the position shifted by the amount is the start or end position of the entry frame;
6. The form processing according to claim 1, further comprising a step of detecting entry frame data based on a start position and an end position in the vertical direction of the entry frame and / or a start position and an end position in the horizontal direction. Method. The entry frame detection step includes:
The form according to any one of claims 1 to 5, comprising expanding the dots of the entry frame data, obtaining a histogram for the expanded entry frame data, and detecting each entry frame data and information bar based on the histogram. Processing method. The superimposition information recognition step includes:
A horizontal and / or vertical histogram is taken for an information bar that is composed of rows of dots of the same height but different widths, and is placed on a plurality of rows and shifted by a predetermined number of dots. Cutting out dots for each row based on the histogram;
Taking a vertical histogram of dots for each cut out line, measuring the length of the dots according to the histogram, obtaining information indicated by each dot and obtaining data of the information record;
A step of returning a line shift by a predetermined number of dots;
The form processing method according to claim 3, further comprising a step of determining data of the information record by taking a majority of values of each information record. The superimposition information recognition step includes:
A code cutting step of cutting out the code of the information bar detected in the entry frame detecting step;
A code recognition step for recognizing the cut code;
Determining whether the recognized code is a leading delimiter and searching for the leading delimiter;
Obtaining data of an information record including predetermined information based on a code from the first delimiter searched in the searching step until a terminal delimiter is recognized;
Determining the data of the information record by taking a majority vote of the data of the requested information record;
The form processing method according to claim 3 or 4, comprising: The code recognition step includes
12. The form processing method according to claim 11, wherein the code is recognized by a histogram in four directions of a vertical direction, a horizontal direction, a right diagonal direction, and a left diagonal direction. The code recognition step includes
The code is recognized by sequentially scanning a predetermined position with respect to the extracted code, or the feature value data of the extracted code is extracted, and the standard pattern of each code stored in advance is extracted. The form processing method according to claim 11, wherein the code is recognized by matching with the feature value data. The superimposition information recognition step includes:
Determining the length of the information record by taking a majority vote of the length of each information record determined;
Determining the length of each information record and deleting information records different from the determined length and / or information records whose length is not within a predetermined range;
Checking the parity code of the information record and deleting the incorrect information record;
Further including
The form processing method according to claim 11, wherein a majority vote is taken using data of an information record that has not been deleted, and data of the information record is determined. The code cutting step includes
Cut out the code for each column from left to right or right to left of the information bar, and ignore the following code figure until the delimiter is found when the interval between the previous column and the extracted column is larger than the predetermined width The form processing method according to claim 11. Detecting a horizontal bar that is included in the form image data and is composed of codes based on a horizontal histogram of the input form image data;
Detecting the position of the code of the detected horizontal bar and determining the distance between the codes based on the position of the code;
Determining the size of the code based on the determined distance between the codes,
The code cutting step includes
The code of the information bar detected in the entry frame detection step is cut out, and if the size of the cut out code is smaller than the obtained size, the code is ignored, while if larger, the obtained size or The form processing method according to claim 11, wherein a code cut out in a predetermined size is separated. Detecting a horizontal bar in which codes are arranged by the same number of lines as the information bar based on a horizontal histogram of the input form image data;
Determining the number of lines of code of the detected horizontal bar,
The code cutting step includes
Cut out the code of the information bar detected in the entry frame detection step, determine the number of codes cut out in this column and the scanned length, and whether the number of codes has reached the number of code lines of the obtained horizontal bar; and 12. The form processing method according to claim 11, wherein it is determined whether the scanned length has reached a length corresponding to the number of code lines of the horizontal bar, and the cutout process is terminated when it is determined that the scanned length has been reached. The character recognition step includes:
When the attribute of the handwritten data indicated by the superimposition information extracted in the superimposition information recognition step is a mail address, the character type information entered in the character type entry box for limiting the character type of the mail address is recognized, and the character type information The form processing method according to claim 3 or 4, wherein character recognition is performed according to the above. The code is
The information is represented by any one of a horizontal line segment passing through a predetermined point, a vertical line segment, and a predetermined number of line segments inclined by a predetermined angle. Form processing method. The code is
The information is represented by one of eight line segments, a horizontal line segment passing through the center of a predetermined area, a vertical line segment, and six line segments inclined by a predetermined angle. The form processing method according to any one of 1 to 4. The code is
2. Each of (n + m) -th power bits is expressed by each side and the presence / absence of the line segment, including each side of the n-gon and m line segments having different inclinations through the inside of the n-gon. 4. The form processing method according to any one of 4. The code is
A code that includes each side of the quadrangle and four line segments that pass through the center of the quadrangle and have different inclinations, and expresses information of 2 8 bits by the presence or absence of each side and line segment, or an octagon , And four line segments that pass through the center of the octagon and have different inclinations, and are configured by any of the codes that express information of 2 12 bits depending on the presence or absence of each side and the line segment. The form processing method according to claim 1. A form input step for inputting form image data including entry frame data composed of a code in which information is expressed by an inclination of a line segment or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
Cutting out each code of the detected information bar;
A step of obtaining a histogram of four directions of the cut code, the vertical direction, the horizontal direction, the right diagonal direction, and the left diagonal direction;
A form processing program for causing a computer to execute a code recognition step of determining a code shape feature and recognizing a code based on a obtained histogram in four directions. A form input step for inputting form image data including code data in which information is expressed by a dot width, a slope of a line segment, or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
A superimposition information recognition step of recognizing each code of the information bar detected in the entry frame detection step and extracting superimposition information;
A character recognition step of performing character recognition according to the attribute and / or character type of the handwritten data indicated by the extracted superimposition information for the handwritten data separated in the entry frame detecting step;
An instruction book for creating instruction book data for causing another application system to perform a predetermined process based on the character information recognized in the character recognition step and the information indicated by the superposition information extracted in the superposition information recognition step. Creation steps,
The computer outputs the command data created in the command creation step and / or stores the form image data input in the document input step and the command data in association with each other. Form processing program to let you. A form input step for inputting form image data including entry frame data composed of a code in which information is expressed by an inclination of a line segment or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
Cutting out each code of the detected information bar;
A step of obtaining a histogram of four directions of the cut code, the vertical direction, the horizontal direction, the right diagonal direction, and the left diagonal direction;
A recording medium on which a form processing program for determining a code shape feature and causing a computer to execute a code recognition step based on the obtained four-direction histogram is recorded. A form input step for inputting form image data including code data in which information is expressed by a dot width, a slope of a line segment, or a combination of a plurality of line segments, and written handwritten data;
Based on the form image data input in the form input step, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, A frame detection step for detecting
A superimposition information recognition step of recognizing each code of the information bar detected in the entry frame detection step and extracting superimposition information;
A character recognition step of performing character recognition according to the attribute and / or character type of the handwritten data indicated by the extracted superimposition information for the handwritten data separated in the entry frame detecting step;
An instruction book for creating instruction book data for causing another application system to perform a predetermined process based on the character information recognized in the character recognition step and the information indicated by the superposition information extracted in the superposition information recognition step. Creation steps,
The computer outputs the command data created in the command creation step and / or stores the form image data input in the document input step and the command data in association with each other. A recording medium on which a form processing program for recording is recorded. A form input unit for inputting form image data including input frame data composed of a code in which information is expressed by a slope of a line segment or a combination of a plurality of line segments, and written handwritten data;
A processing unit that detects an input frame of the input form image data and recognizes a code in the input frame;
The processor is
Based on the form image data input from the form input unit, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, An entry frame detecting means for detecting
Means for cutting out each code of the detected information bar;
Means for obtaining a histogram of four directions of the cut cord in the vertical direction, the horizontal direction, the right diagonal direction, and the left diagonal direction;
A form processing apparatus having code recognition means for recognizing a code by judging a shape characteristic of the code based on the obtained histogram in four directions. Form input unit for inputting form image data including entry frame data composed of codes in which information is expressed by dot width, line slope, or a combination of a plurality of line segments, and written handwritten data When,
A storage unit for storing form image data and / or command data created based on the form image data;
A processing unit that recognizes the handwritten information entered in the input form image data and the information superimposed on the entry frame and executes a predetermined process;
With
The processor is
Based on the form image data input from the form input unit, the entry frame data and the handwritten data are separated, and each entry frame data and an information bar in the entry frame data on which predetermined information is superimposed by a code, An entry frame detecting means for detecting
Superimposition information recognition means for recognizing each code of the information bar detected by the entry frame detection means and extracting superposition information;
Character recognition means for performing character recognition on the handwritten data separated by the entry frame detection means according to the attribute and / or character type of the handwritten data indicated by the extracted superposition information;
Instruction book creation means for creating instruction book data for causing other application systems to perform predetermined processing based on the character information recognized by the character recognition means and the superposition information extracted by the superposition information recognition means;
Means for outputting the instruction book data created by the command book creating means and / or storing the form image data inputted by the inputting means in correspondence with the instruction book data in the storage unit; A form processing apparatus. An information record in which predetermined information is superimposed by a code in which information is represented by a dot width, a slope of a line segment, or a combination of a plurality of line segments has an information bar arranged over a plurality of lines, and the code An entry box consisting of:
A form comprising the code and comprising a form or a horizontal bar containing information on the information bar. The code is a dot having the same height and different width,
30. The form according to claim 29, wherein information records on which information is superimposed are arranged with a predetermined number of dots shifted in a plurality of rows. The code is
30. The form according to claim 29, wherein information is represented by any one of a horizontal line segment passing through a predetermined point, a vertical line segment, and a predetermined number of line segments inclined by a predetermined angle. The code is
The information is represented by one of eight line segments, a horizontal line segment passing through the center of a predetermined area, a vertical line segment, and six line segments inclined by a predetermined angle. 29 forms. The code is
30. The (n + m) -th power bit information is represented by each side and the presence or absence of a line segment, including each side of the n-gon and m line segments that pass through the n-gon and have different inclinations. The form described. The code is
A code that includes each side of the quadrangle and four line segments that pass through the center of the quadrangle and have different inclinations, and expresses information of 2 8 bits by the presence or absence of each side and line segment, or an octagon , And four line segments that pass through the center of the octagon and have different inclinations, and are configured by any of the codes that express information of 2 12 bits according to the presence or absence of each side and the line segment. The form according to claim 29.

Images