JP2000076370A - Slip recognizing method and its device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily obtain document format information for key-inputting data to a data processor by referring to a format display displayed on a screen by inputting a picture including a slip written in an original document and executing a program corresponding to the kind of the slip in the inputted picture. SOLUTION: First, a paper where the format of the slip is written is read by a picture inputting device 93 to store in a picture storing area 981 as digital picture information. Next, the picture of a slip format is recognized to obtain format information converted into code data and is stored in a slip format storing area 932. In addition, slip contents data suited to the slip format is prepared and stored in a slip contents storing area 983. At the same time, outputting data of slip data by these format and contents data of these is prepared and stored in an outputting slip storing area. Then, output slip data is printed by a printer 95 or stored in a data file 99. Thus, a new data inputting format, etc., is easily executed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document data processing system using image data, and more particularly, to data input from a keyboard to a database or data output from a keyboard or a database in, for example, a general office processing field. The present invention relates to a data processing method and a data processing method for creating a document (form) having a predetermined format with a blank space for inputting or outputting character data required for the data processing.

[0002]

2. Description of the Related Art Using office equipment such as a workstation, office processor, personal computer, or word processor, for example, various application forms, data entry slips, billing slips and receipts, or accounting, statistics, breakdown, etc. When a document (form) having a predetermined format for issuance is to be issued or data is to be input to a database using the format of the document displayed on the screen, a form having a blank for data input / output is provided. Must be defined on the display screen of the above-described office equipment.

[0003] A conventional typical format definition method is to define ruled lines and characters by moving a cursor or the like on a display as described in, for example, a manual issued by Hitachi, Ltd. in a document entitled ETOILE / OP. In addition, a program for issuing the program is created and edited using a program editor or the like.

[0004]

In such prior art, in order to issue a form in a format unknown to a device or to create a data input environment using a format screen, a large amount of time and specialized knowledge about a program are required. And require experience. Unknown format is not only the initial introduction of equipment, for example, when specified by a supplier in the form of a specified slip
Occurs on a daily basis. For this reason, simplification and speeding up have been major issues.

Japanese Patent Application Laid-Open No. 61-059568 discloses a document analysis system in which format data for performing character reading by OCR is automatically generated from document image data.

In the above document analysis method, after recognizing a physical structure of a tabular form composed of line patterns included in a document from document image data, a rectangular cell area (first part) constituting the tabular form is recognized. In the specification of the application, this is called a frame), and the logical structure of the document is also recognized by analyzing the type, attribute, and dependents. However, the recognition of the logical structure performed in the above prior art is intended for application to OCR, and the type of cell (cell for setting item names or cell for setting data) is used. Only the analysis of cell attributes (types and tolerances of characters written in cells) and sub-dependencies between cells.

It is an object of the present invention to provide a form or form document in which an operator can easily obtain document format information for keying data into a data processing device by referring to a format display displayed on a screen. An object of the present invention is to provide a data processing method and apparatus.

Another object of the present invention is to provide a form suitable for printing or displaying a document in which data input by an operator or data read from a data file is set in a predetermined blank space in a format. Another object is to provide a data processing method and apparatus for a form document.

Still another object of the present invention is to provide a blank (data input / output field) for inputting or outputting data.
A data processing program for an operator to input data to a data processing device or to output a data-inputted form while referring to a format display of a predetermined format having a It is an object of the present invention to provide a method and an apparatus for automatically generating a program suitable for the above.

[0010]

In order to achieve the above object, a form or form document processing system according to the present invention is supplied directly from an image input means or indirectly through another system. Using a data processing system having a processor for processing an image of a form or form document, a display means, and an operator operating means, a form or form document including a field portion for filling fixed data or variable data is used. The image is processed by the processor according to a format recognition program prepared in advance, and the structure of the document is automatically recognized from the input image information.

For example, the format recognition of the input document is performed based on the pattern arrangement of lines, characters, or arcs, that is, the physical structure recognition for identifying the graphic structure of the input document, and the input performed based on the physical structure recognition result. Recognition of the logical structure of the document. The logical structure referred to here means, for example, a structure that extends to the meaning of a line column structure or a field of a table included in a form. By using the result of the recognition of the logical structure, for example, when the operator completes inputting data in one field on the display screen,
The cursor can be automatically moved to the next field, or data prepared in advance in the file can be correctly set in the corresponding field.

The above-described logical structure recognition will be described in further detail. In this recognition processing, for example, an input / output field for inputting variable data and a field for identifying a fixed field in which fixed data or an item name is written are described. Recognition, field relationship recognition for associating input / output fields with fixed fields or field names, and input / output field attribute recognition with reference to a knowledge base are performed.

According to the present invention, as the logical structure of the input document, the source of the data to be set in each person's output field is identified.
Recognize and process a form document image with columns for “Product Name”, “Unit Price”, “Quantity”, and “Amount”, and the operator simply inputs the product code and quantity, and the other fields are entered from the data file. It is possible to realize a data processing system for inputting data or issuing a form that can be automatically filled with read data or calculation results. In addition, by using the recognition result of the physical structure and the logical structure of the document, a program for data input or a form output process can be automatically generated.

Another feature of the present invention is that one of the logical structures of a form or a form document is based on data indicating the recognition result of a physical structure, and has a table-like dimension in a row direction and the size of each input / output field. It is an object of the present invention to obtain a normalized format display in which a digit size is modified so as to conform to a basic pitch or a character pitch of a cursor on a display screen. Apart from the structural data of the document, by generating such a normalized format display suitable for on-screen data entry,
Operator key operations for data input and output of data generated by a keyboard or program processing to a screen are facilitated.

The size, the number of characters, and the character pitch that can be entered in each input / output field on the form document are as follows:
It can be determined by the physical size of each input / output field and the type of data set in each input / output field (this is determined by the field name to be given to each input / output field). The information indicating the character size, the number of characters, and the character pitch is stored as a part of the field attribute, and the line digit normalization process is performed with reference to the field attribute.

The character data set in the input / output fields on the format display is combined with the document format defined by the physical structure recognition data of the input document with the character size and character pitch defined by the field attributes. Is output to the printer. Accordingly, it is possible to obtain a print of the same form as an input document having a different character size and format display output on the screen at the time of the data input operation, in a form in which a blank portion is filled with characters having a size corresponding to the field size. .

In the present invention, the document data obtained by the physical structure recognition can be moved or deleted by a key operation.
Create a format display on the display screen that allows you to change, add, or delete characters. Therefore, the operator performs an operation for correcting a part of the format display on the display screen, thereby causing the above-described logical structure recognition and automatic program generation for a format document partially different from the original document. Can be.

According to the present invention, a base paper including, for example, a table including a plurality of fields (columns) partitioned by ruled lines and a character string entered in or outside of these fields is used, and this is used as an image. By inputting to the input device and causing the processor to perform image processing, the structure of the input document is automatically recognized, and the format is displayed on the front screen. The operator can modify or add a part of the format data on the screen by operating the keys, thereby greatly improving the format design work.

Also, a practical program for data entry or form issuance using the above recognition result, or a program completed to such an extent that it can be put into practical use with only a slight modification by a person in charge, is automatically generated. Because it can be generated, the program development work associated with a format change or new establishment can be completed quickly.

[0020]

Embodiments of the present invention will be described below.

FIG. 1 shows an example of the hardware configuration of a form processing apparatus according to the present invention. In the figure, 91 is a console provided with a press screen and a keyboard for inputting commands from an operator, 92 is a processor for executing a form processing, 93 is an image input device for loading image data into a memory, 94 is an image input device 93
, A printer 95 for printing data on the memory, 96 a controller for the printer 95, 9
Reference numeral 7 denotes a memory for storing a program executed by the processor 92, 98 denotes a memory for storing various data used by the processor 92, and 99 denotes a memory for storing various numerical data and document (form) data. Reference numeral 90 denotes a bus for interconnecting the processor 92 and other elements. Memory 97
Are an overall control unit 970 for storing a main program for controlling the overall operation of the form processing, and an image input unit 97 for storing a subroutine program for inputting an image.
1, a form format recognizing unit 972 for storing a subroutine program for recognizing a form format, a form preparing unit 973 for storing a subroutine program for preparing a form, and a subroutine program for outputting a form. And a form output unit 974 to be executed. The memory 98 has an image storage area 98 for storing image data.
1, a form format storage area 982 for storing form data, a form content storage area 983 for storing form data, and an output form storage area 984 for storing form data to be output. , The work area 985 of the processor 92. The memory 97 has a ROM (R
read only memory), the memory 98 is RA
It can be implemented using M (Random Access Memory), or can be implemented by combining an external storage device such as a disk with a RAM.

FIG. 2 is a flowchart showing a basic embodiment of a main program for form processing stored in the memory 970. Steps 1 to 4 correspond to areas 971 to 9
74 respectively correspond to the subroutine programs stored in the subroutine program. In the image input step 1, a paper on which a form is described is read by the image input device 93 and stored in the image storage area 981 as digital image information. In the form format recognition step 2, the form format image is recognized, the format information converted into the code data is obtained, and stored in the form format storage area 982. In the form creation step 3, form content data conforming to the form format is created and stored in the form content storage area 983, and output data of the form data in these formats and content data is created.
Store in the output form storage area. As a special case of this step, it is also possible to create form data having only a form format without form contents. In the form output step 4, the output form data is printed by the printer 95 or stored in the data file 99.

FIGS. 3 and 4 show examples of form data used in the description of the following embodiment. FIG. 3EX1 shows one of the forms input from the image input device 93 in step 1 described above.
It is an example. FIG. 4EX2 is an example of a form output from the printer 95 in step 4 described above. The form processing apparatus includes a ruled line (line pattern) 100, a form name (title) 101, a date 102, an item name 103 as in EX1.
(103A-103D), etc. The purpose of this is to input an image on a form on which a form format is recorded, and to output a form including data contents 110 of each item and date, such as EX2, from a printer or the like with high quality. It is assumed that.

In the present specification, in the table drawn by the ruled line 100, each rectangular area arranged in the row or column direction is called a column or a cell.
A cell in which 3 is written is called an item cell or a fixed field cell, and a cell in which a blank area for writing data is left is called an input / output field cell.
Special fields in areas other than the table are simply called fixed fields or input / output fields.

FIG. 5 is a flowchart showing another embodiment of the form processing program stored in the memory 970.
In the program flowchart of FIG. 2, a series of operations from image input to form printing are executed.
In this program, a form definition process for processing from image input to form registration and a form issuing process for processing from form search to form printing can be selectively executed by input commands. .

In the figure, in a command input step 5a, one of a format definition command, a form issuing command, and an end command is input. If a format definition command is input, steps 1 and 2 described in FIG.
Is performed, the format data stored in the form format storage area in step 5b is registered in the disk device or the like 99 in association with the form name specified by the operator, and finally, the command input step 5a is executed. When the form issuing command is input, first, a desired format is searched from the format data registered in step 5b, and then steps 3 and 4 described with reference to FIG. 2 are executed. The input step 5a is executed. In this embodiment, in order to allow the operator to continuously issue a plurality of forms using the same format, input of a continuous form issuing command in the last command input step is permitted.

Hereinafter, steps 1 to 1, which are main parts of the present invention, will be described.
4 will be described in detail.

FIG. 6 is a detailed flowchart of the subroutine program executed in the image input step 1. In this program, first, at step 11, address information of an image storage area in the memory 98, a range of an image to be input, a sampling density of digitization, and the like are set to the controller 94 as parameters for image input. Next, in step 12, the controller 94 is instructed to input image data according to the parameters, and the image input device 93 is activated. In step 13, flag information indicating whether or not the image input processing has been completed is read from the controller 94. Step 13 if not completed
repeat.

FIGS. 7A and 7B show an input image memory 9.
8 is a diagram for explaining a method of storing data in the storage device of FIG. FIG. 1A shows a state in which a two-dimensional image is divided into an array of small points called pixels. Here, it is assumed that the value of each point is either white or black, and can be represented by one bit in the memory. In the figure, m is a pixel position in the horizontal direction, n is a pixel position in the vertical direction, M is the maximum value of the position m,
N indicates the maximum value of the position n. FIG. 7B shows a state in which these pixels are stored in the image storage area 981 of the memory 98. In the figure, w represents the number of bits in one word, a represents a memory address, and b represents the number of bits in a word. m, n,
Assuming that b is an integer starting from 1 and a is a relative address starting from 0, the pixel 120 at the pixel position (m, n) is stored at the position (a, b) in the memory where the following relationship is established. One-to-one correspondence is possible.

A = ↓ ｛M · (n−1) + m｝ / w ↓ b = M · (n−1) + m−aw The expression between the symbols ↓↓ indicates that the decimal part of the result is Means to truncate. In the following description, for the sake of simplicity, a pixel position to be referred to is represented by a two-dimensional pixel position instead of a memory address.

FIG. 8 is a flowchart showing a first embodiment of a subroutine program executed in the form format recognition step 2. In the first embodiment, the form format recognition step 2 comprises only a physical structure recognition step 21 for recognizing the physical structure of the form as a form. The “physical structure of a form” here means a simple graphic structure, such as a line segment, a character, and an arc, which does not extend to the meaning of the form, as described later in detail.

FIG. 9 shows a flowchart of a subroutine program called in the physical structure recognition step 21. The subroutine 21 is stored in an area different from the program 2 in the memory area 972.

In step 211, the given area is
The region is divided into a plurality of blocks in units of a circumscribed rectangle of a region where black pixels are connected. The given area is the whole image when it is first called in step 2, and becomes a part of the image when it is called recursively in step 21 itself as described later. In step 212, one of the blocks obtained in the block dividing step 211 is selected. In step 213, it is determined whether the type of the selected block is a table or a table other than a table. In the determination of the block type, for example, when the values of the width and the height of the block are both larger than a predetermined threshold value, the block is determined to be a block constituting a table, otherwise, an element other than the table is configured. Judge as a block. If it is determined that the block type is other than the table, the component is recognized in step 218. If the block type is determined to be a table, first, step 214
Then, the block is divided into units of a circumscribed rectangle of the white pixel connection region. Next, in step 215, one of the regions obtained in the region dividing step 214 is selected. Next, in step 216, whether the type of the selected area is a column (cell),
Recognize that it is not a column. If it is determined that the area type is other than the column, in step 217, a recognition process for a non-column area is performed. If the area type is determined to be a column, step 2
In step 1, the physical structure recognition subroutine program is recursively called, and furthermore, the physical structure of this area is recognized.
A method for executing the above-described recursive call of a program by a normal processor is known as a compiler technology of a normal programming language. Wi
This is described in detail on page 142 and subsequent pages of "Algorithm + Data Structure = Program" by Rth and translated by Takuya Katayama (published by the Japan Computer Association in 1979), and will not be described here. The above processing is repeated for all blocks and areas.

FIG. 10 is a detailed flowchart of the block dividing step 211. In the figure, the variable Ι is the number of valid temporary blocks during processing, n is the vertical pixel position described in FIG. 7, i is the number of the temporary block to be processed, N is the height of the target image or target area, , Y) is the pixel (x,
The density of y), m is the horizontal pixel position, and li and ri are the left and right pixel positions of the i-th temporary block.

In this processing, first, the above-mentioned variables are initialized, and each time the processing for one row of pixels is completed, steps 211a and 211b are repeated. In step 211a, at the end of the processing of the row, the table T1
In, it is determined that the same block has been detected as a plurality of temporary blocks. In step 211b, a temporary block newly detected in the row is registered in the table T1. The value of the variable Ι is also updated by the processing of these temporary blocks. For each temporary block, it is checked whether or not the position of the left end or the right end is updated. If it is updated, the temporary block information is updated in step 211d. If it is not updated, it is checked whether or not the temporary block has been completed in the row. A process for deleting a temporary block is performed. Finally, in step 211C,
If the circumscribed rectangles of the formal blocks overlap each other, a process of integrating them is performed.

In the above processing, the tables T1, T used for registering the information of the temporary block and the formal block
2A and 2B are shown in FIGS. The table T1 shown in FIG. 7A is a table for storing information of a temporary block, and is composed of items of a pixel position 1 at a left end, a pixel position r at a right end, and a pixel position n0 at an upper end. A table T2 shown in FIG. 7B is a table for storing information of a formal block, and includes a pixel position x0 at the left end of the block, a pixel position x1 at the right end, and a pixel position y at the upper end.
0, and each item of the pixel position y1 at the lower end. Note that the variable J represents the number of blocks finally obtained.

The state of the above block division processing will be described with reference to a specific example of FIG. Symbols a to e
Respectively correspond to steps 211a to 211e, and indicate that the corresponding step is executed at that point. First, in a row L1, information indicating a new temporary block B1 is registered in the table T1 (b). In the next row L2, the detection of the black pixel in contact with the temporary block updates the left and right pixel positions thereof (d), and registers information of another temporary block B2 appearing in the same row in the table T1. (B). In the next row L3, the pixel positions at the left and right ends of the two temporary blocks B1 and B2 are updated (d), and another temporary block B3 is registered. The temporary blocks B1 and B2 are integrated into one temporary block B1-2 after the processing of this row is completed (a). Further, in the next row L4, since the black pixels in contact with the temporary block B1-2 integrated above do not appear, that is, the temporary block is not continued, and after the processing of this row is completed, based on the temporary block information, The formal block information is generated and stored in the table T2, and the temporary block information is deleted from the table T1 (e). After the processing of all the rows is completed as described above, block information is integrated for squares circumscribing each block having a common area (overlapping each other) (c). In this example, since the two blocks B1-2 and B3 partially overlap, they are integrated into one block.

FIG. 13 is a detailed flowchart of the area dividing step 214. In the figure, the variable Ι is the number of temporary areas effective during processing, n is the vertical pixel position described in FIG. 7, i is the number of the temporary area to be processed, N is the height of the target area, p (x, y) indicates the density of the pixel (x, y), m indicates the horizontal pixel position, and li and ri indicate the left end pixel position and the right end pixel position of the i-th temporary area.

In this processing, the same table as that shown in FIG. 11 is used, and the above-mentioned variables are initially set.
Each time the processing of the pixels for one row is completed, step 214a
And step 214b are repeated. In step 214a, when it is determined that the same area is detected as a plurality of temporary areas at the end of the processing of the row,
The temporary area is integrated. In step 214b, registration processing of a temporary area newly detected in the row is performed.
By processing these temporary areas, the value of the variable の is also changed.

With respect to each temporary area, it is checked whether or not the position of the left end or the right end is updated.
In step 214d, the temporary area information is updated. If it is not updated, it is checked whether or not the temporary area is completed in the row. If it is completed, processing for registering (adding) it as a formal area in step 214e and deleting temporary area data is performed. . Finally, formally, in step 214c, the overlapping rectangles of the regions are integrated.

FIGS. 14A to 14C are diagrams for specifically explaining the progress of the physical structure recognition step 21. FIG. FIG. 9A shows a structure EX3 of an area recognized when the first block division step 211 is performed on the original image EX1.
Is shown. Here, each detected block is represented by a rectangle circumscribing it. EX4 shown in FIG.
Indicates a state of an area recognized when the area dividing step 214 is performed on a block determined as a table in EX3. In particular, each detected area is represented by a rectangle circumscribing it. EX5 shown in FIG. 9C is the state of the area recognized when the block division step 211 is performed on the area determined to be a column in EX4.

FIG. 15 is a flowchart of the area type recognition step 216. In the present embodiment, the recognition step 216 is to recognize each of the left, right, upper, and lower sides of the area.
It consists of two steps 216-1 to 216-4. In the physical structure recognition step 21, based on the recognition result, for example, it is determined that an area having a line segment of three or more sides is a column.

The details of the left side recognition step 216-1 among the four side recognition steps will be described with reference to the flowchart of FIG. In the figure, n is a variable indicating the pixel position in the vertical direction of the area, n0 and n1 are the minimum and maximum values of n, m is a variable indicating the pixel position in the horizontal direction of the area, and m0 is m at the boundary position of the area. Value of wid (n)
Is a variable indicating the number of black pixels continuous in the horizontal direction near the boundary m0 in the pixel row at the position n in the vertical direction.

First, after initializing each variable,
In step 216a, wid (n) is obtained for the outside of the area. Next, in step 216b, wid (n) is obtained for the inside of the area and added to the above value. At step 216c, the line width of the left-side line segment is determined based on wid (n) obtained for each n. The line width is, for example, (n1-n0 +
1) The most frequent value of the values of wid (n) can be determined as a line width. If the line width obtained in this way is not 0, the line segment position is specified in step 216d, and step 216e is performed.
To add data to the line segment table TBL1.

FIG. 17 shows an example of the structure of the table T3 required to execute the above-described left side recognition step. The table T3 is composed of items of a left end x0, a right end x1, and a length wi of a black pixel continuous in the horizontal direction near the boundary for each n. For each n, the equation wid = x1-x0 + 1 holds. This table is prepared in the work area 985.

FIGS. 18A and 18B are diagrams for specifically explaining the above-described left side recognition processing. In the figure, (A) shows the state of the left side of the original image. In the original image, a part of the black pixel is usually defective due to an input error of the image or an extra black pixel is generated, so that the edge of the straight line is generally uneven. FIG. 7B shows the result of processing the input image in the left side recognition step 216-1. The line width and the line position are recognized based on the line segment data with the highest frequency, and a straight line image is formed and displayed based on the line segment data with the unevenness corrected. With this recognition, it is possible to obtain position information of the straight line and the like, and to generate a straight line in which unevenness existing on the surface is smoothed.

FIG. 19 shows the configuration of a table TBL1 for storing the recognition result of a line segment generally parallel to the coordinate width, including the left side described above. The table TBL1 includes upper left coordinates (LX0, LY0) and lower right coordinates (LX1, LY) when a line segment is regarded as a rectangular area as shown in FIG.
It consists of each item of 1). This table is prepared in the form format storage area 982, and the line segment data recognized in the component recognition step 218 described later is also stored in the table.

Next, the non-column element recognition process performed in step 217 will be described.

FIGS. 20A to 20F show typical patterns of non-column elements to be handled in this embodiment. FIGS. 7A and 7B show patterns P corresponding to diagonally hatched columns.
T1 and PT2. In these patterns, immediately before step 214c of the area dividing step 214 described with reference to FIG. 13, a portion surrounded by a broken line in the drawing is a superimposed area of substantially the same size. Figures (C)-(F) are PT3-PT6 patterns corresponding to the corners of the table rounded into an arc. In these patterns, immediately before the step 214c of the area dividing step 214, a portion surrounded by a broken line in the figure is a small area superimposed inside a relatively large area. In any case, since these patterns cannot be detected as the left, right, upper, and lower four sides in the area type recognition step 216, they are determined to be non-linear areas.

Next, an example of a process for detecting the above-described arc pattern from an input image will be described with reference to FIG. In the figure, a portion surrounded by a broken line is one region detected in step 214, and this region is shown in FIGS.
One of the patterns (F) is located. First, this area is divided into 16 partial areas by six dashed lines shown in the figure. The three vertical lines V1 to V3 are set based on the area boundaries x0 and x1 and a constant α having a predetermined small value so that the respective x coordinates are x0 + α, (x0 + x1) / 2, and x1-α. You. The three horizontal lines H1 to H3 are set based on the area boundaries y0 and y1 and the predetermined constant α so that the respective y coordinates are y0 + α, (y0 + y1) / 2, and y1-α. Next, among these partial regions, in particular, the central 4 surrounded by only six dashed lines.
Check the density of black pixels inside the two areas. The density is defined by the ratio of black pixels to all pixels in each partial area. If this density is larger than a predetermined threshold, it is called a black area, and if it is smaller, it is called a white area. When the black area exists at the upper left and lower right, the pattern PT1 in FIG. Is PT2, upper left, upper right, lower left is PT3, upper left,
The upper right and lower right can be determined to be a pattern of PT4, the upper left and lower left, and the lower right can be determined to be a pattern of PT5, and the upper right, lower left and lower right can be determined to be a pattern of PT6.

In the above processing, the pattern PT1 or PT
2 are included in the information shown in FIG.
In the table TBL2 shown in FIG. This table TBL
Numeral 2 includes the leftmost pixel coordinates (SX0, SY0), the rightmost pixel coordinates (SX1, SY1), and the line width SW of the oblique lines. The leftmost and rightmost coordinates can be easily obtained by examining the coordinates of the corner point of the target area or the black pixels near the corner point.

The line width SW is, for example, the number of black pixels B in the area.
P and the length SLA of the line segment, ie, SLA = √ (SX0−SX1) 2+ (SY0−SY1)
2 can be calculated as SW = BP / SLA or a value obtained by converting this to an integer by rounding or the like. This table is a form format storage area 9
82.

For the patterns determined to be patterns PT3 to PT6 in the above processing, the arc information is stored in the table TBL shown in FIG.
3 is stored. This table TBL3 includes the center pixel coordinates (AX, AY), radius AR, line width AW, and start angle A of the arc.
A0 and the end angle AA1. The start angle and the end angle are respectively (90, 18) according to the pattern types PT3 to PT6 obtained by the above method.
0), (0, 90), (180, 270), (270,
0). The radius and the line width can be calculated by solving the following equations using, for example, the number of black pixels BP in the region, the length AL of the long side of the region, and the length AS of the short side.

AS−AW / 2 = AR AW · (π · AR / 2 + AL−AS) = BP The center pixel coordinates are as follows for each of the patterns PT3 to PT6.
The area is defined as the same point as the length of the short side from the end on the long side.
This table is also prepared in the form format storage area 982.

FIG. 24 shows an embodiment of the component recognition step 218. Here, first, it is determined whether the type of the component of the form is a straight line, a broken line, or a character, and any one of the straight line recognition step 2181, the broken line recognition step 2182, and the character recognition step 2183 is determined based on the determination result. The processing of is performed.

An example of a method for determining the type of a component will be described with reference to FIG. In this example, the block dividing step 21
The type of the component is determined based on the size and shape of the rectangle circumscribing each block obtained in 1.

In the figure, the horizontal axis and the vertical axis respectively represent the horizontal width, ie, X1-X0 + 1, and the height, ie, Y1-Y0 + 1, of the rectangle circumscribing the block. Here, ranges 81 to 83 are defined as shown in the drawing, using preset constants β and γ as boundary values. If the target block belongs to the range 81, a straight line;
If it belongs to 3, it is determined to be a character.

FIG. 26 shows a flowchart of the straight line recognition step 2181. In this step, for example, information on a straight line is obtained in exactly the same manner as in the left side recognition step described with reference to FIG. In the figure, line width determination step 2
Step 181a corresponds to steps 216a to 216c in FIG. The line position determination step 2181b also corresponds to step 216d. Further, step 2181c corresponds to step 216e, and in exactly the same way, the recognition result is registered in the line segment table TBL1 prepared in the form format storage area.

Next, the broken line recognition step 2182 will be described. In this step, it is checked whether or not the detected dashed line element extends the already detected dashed line. First, conditions for two dashed elements to form one dashed line will be described with reference to FIG. In the figure, the upper left pixel coordinates of the two dashed elements 85 and 86 are respectively (x0,
y0), (x0 ', y0'), and when the lower right pixel coordinates are (x1, y1), (x1 ', y1'), the following inequalities are satisfied with respect to the predetermined constants d1 and d2. Assume that a set of dashed line elements is an element constituting the same dashed line.

Min (| y0-y0 '|, | y1-y1' |) <d1 Min (| x1-x0 '|, | x1-x1' |) <d2 This function takes the smaller value of the values of the expression.

FIG. 28 is a flowchart of the broken line recognition step 2182. First, in step 2182z, it is determined whether or not the dashed line element extends a previously detected dashed line stored in a table TBL4 described later.
If so, the range of dashed lines is extended to include all of these elements. With respect to the range of the dashed line elements that are sequentially combined in this manner, dashed line information can be obtained as described below in exactly the same way as in the straight line recognition step described with reference to FIG. In the figure, a line width determination step 2182a corresponds to step 2181a in FIG. However, in the process of obtaining the maximum frequency of the value WID (see the description of the step 216c), the frequency of the value 0 is excluded in consideration of the existence of the interval between the broken line elements in the case of the broken line. The line position determination step 2182b corresponds to step 2181b in FIG. Step 2182
"c" corresponds to step 2181c, and is a broken line table TBL prepared in the form format storage area in the same manner as in the case of a straight line.
4 stores the recognition result.

FIG. 29 is a diagram showing one example of the configuration of the broken line table TBL4.
Here is an example. The table TBL4 includes upper left coordinates (DX0, DY0) and lower right coordinates (DX1, D2) when a set of a plurality of broken line elements forming one broken line is regarded as a rectangular area.
Y1). This table is also prepared in the form format storage area 982.

FIG. 30 is a flowchart of the character recognition step 2183. First, it is checked whether or not there is a block near the block that may be determined as a character element with the block type not yet determined. If such a block exists, the block is registered in the work area 985 as a character string element in step 2183d. If there is no block that may be determined as a character element in the vicinity, in step 2183a,
Generate a character string containing the block. In this processing, among the character string elements registered in step 2183d, blocks satisfying a character configuration condition described later are collected, and
One rectangular range including all these blocks is obtained as a character string range. In step 2183b, individual character regions are extracted from the character string range. Individual character areas, for example, in the rectangular range, create projection data of black pixels along short sides, and in areas where gaps in the projection data appearing along long sides exceed predetermined thresholds, It can be extracted by dividing the rectangular range into sub-regions. In step 2183c, a character pattern included in such an individual character area is converted into a character code by a known character recognition method. As a character recognition method, for example, a document titled “Image Processing Handbook” edited by Morio Onoe (Showa 62, published by Shokodo)
The method described on page 468 and thereafter can be used.

The determination as to whether adjacent blocks are elements constituting the same character string is performed, for example, as follows. Figure
In FIG. 31, the upper left pixel coordinates of two blocks (character elements) 87 and 88 are (x0, y0), (x0 ′,
y0 ′), and the lower right pixel coordinates are (x1, y
1), (x1 ', y1'), the following four inequalities with respect to predetermined constants D3 and D4: y0-y1 '<D3 y0'-y1 <D3 x0-x1'<D4x0'-x1 When all of <D4 are satisfied, these character elements are regarded as elements constituting the same character string.

FIGS. 32A to 32C are views showing the process of the actual processing of the character recognition step 2183 described above. In the figure, (A) illustrates a set of blocks determined to be integrated according to the above conditions.
(B) illustrates a state where a character string is actually generated in step 2183a. (C) illustrates the result of extracting individual characters from such a character string in step 2183b.

The recognition result in the character recognition step 2183 is stored in the form format storage area 982 in the form of a table TBL5 shown in FIG. The above table TBL5
Is the pixel coordinates (CX0,
CY0), the lower right pixel coordinates (CX1, CY1), and the character code CC. As the character code, for example, a character code system defined by the JIS standard is used.

Although the basic operation of the form format recognition step 2 shown in FIG. 8 has been described above, the form recognition accuracy can be further improved by adding another processing function to the above step.

FIG. 34 shows another embodiment of the form format recognizing step 2 thus extended. In this embodiment, preprocessing 20 of the input image is executed prior to the physical structure recognition step 21, and post processing 29 of the form data is executed after the physical structure recognition step 21.

FIG. 35 shows an example of a flowchart of a subroutine program called in the preprocessing step 20. Here, the inclination of the input image data is corrected as preprocessing. In steps 201a to 201d, a left side, a right side, an upper side, and a lower side of an image described later are detected, respectively. In step 202, the inclination of the entire image is calculated from the inclination information of the four sides detected in the above step. This can be calculated by, for example, an average value of four values of the left side inclination + 90 °, the right side inclination + 90 °, the upper side inclination, and the lower side inclination. In step 203, the image obtained by the angle obtained in step 202 is rotated. Step 204
Then, an area effective as a form image is cut out as necessary, and in step 205, noise information of the image is removed as necessary.

The "side" of the image detected in the preprocessing step 20 will be described with reference to FIGS. 36 (A) and (B).

FIG. 9A shows a case where the boundary of the paper included in the input image can be used as a reference line for calculating the inclination of the image. This corresponds to a case where the image input device 93 can input an image using a background color different from the background color (for example, white) of the paper surface. For example, in an image input apparatus of a system in which a document is stopped and light reflected from a paper surface is input to a sensor, this can be realized by setting the color of a document cover to black. In such a case, the top, bottom, left, and right sides are:
The detection can be performed by focusing on each boundary line of the paper surface indicated by the symbols 70a to 70d.

FIG. 9B shows a case where the boundary of the paper surface cannot be detected in the input image 71, but the outer frame of the table described in the form can be detected as a reference line for calculating the inclination. In this case, the boundaries of the table indicated by the symbols 71a to 71d are detected as the four sides. Specifically, the outermost boundary at which the white pixels change from the continuous state to the black pixels is determined, and the boundary can be determined by associating the four straight lines with these boundary lines.

FIG. 37 shows an example of a flowchart of a subroutine program called in the post-processing step 29. Here, as an example of the post-processing, the image data stored in the form format storage area 982 is corrected by the physical structure recognition processing. In step 291, the table TBL1
To the line information stored in TBL4.
In 2, the character information stored in the table TBL5 is corrected. In the above embodiment, the line correction step 291 is performed.
And the character correction step 292 are sequentially performed once each. However, as in a normal word processor or CAD system, a line or character correction can be performed at any time by an operator's instruction. You may make it repeat.

In the line correction step 291, processing such as movement, deletion, expansion and contraction of a line according to an operator's instruction is performed as performed in the graphic processing function of a word processor or a CAD system. Here, as an example of the special function of the line correction step 291, a process of automatically correcting ruled line data from a relationship with another line will be described with reference to FIG. In step 2911, intersections of all the lines stored in the form format storage area are obtained. At this time,
The distance between the end point of each line and the other line is determined, and those whose distance is smaller than a predetermined threshold value are also treated as intersections. Step 291
In step 2, the distance between the end point of each line and the end point of another line is obtained, and a set of a plurality of lines having a distance smaller than a predetermined threshold value is obtained. In step 2913, the line width data of each line is replaced with one of the line widths or an average value of the line widths of the line pairs whose line width difference is smaller than a predetermined threshold value. In step 2914, the type of each line is replaced with the most common line type for the above-mentioned line set. In step 2915, the position of the line is normalized with respect to the line set. For the horizontal lines, for example, the x coordinate is replaced with the value of the average position of each line so that the positions of the start point and end point of each horizontal line coincide with the vertical lines of the outer frame. Also, the vertical lines are aligned to the average position of each line so that the start point and the end point (y coordinate) of each vertical line coincide with the horizontal line of the outer frame. In step 2916, a line in which the distance between two adjacent lines is substantially equal, that is, a line which is within an error of a predetermined threshold is detected from a plurality of parallel lines. In step 2917, the line interval is averaged. Move each line as needed to make it equal to the value.

In the character correction step 292, processing such as movement, change, deletion, addition, size conversion, etc. of a character according to an instruction from an operator is executed as performed in a character processing function of a word processor or a CAD system. Here, a special process for automatically correcting each character from the relationship with other characters will be described with reference to FIG.
In step 2921, a set of a plurality of adjacent characters is obtained as a character string. The adjacent character is, for example, FIG.
Can be found using the method described in. In step 2922, the character string obtained in this way is compared with a standard word group prepared in the form format recognition unit 972 in advance. Each word is sequentially applied to the head of the character string, and for example, a word having the largest ratio of the number of matching characters (matching rate) among the characters of the standard word is set as the matching result. Further, the inventors have filed a Japanese Patent Application No. 1-196586, which was filed earlier.
The method described in the item can also be used. As a result of matching,
If the match rate is larger than the predetermined threshold, the character code of the character that does not match the standard word is corrected in step 2923, and the word in the character string matches the standard word. In step 2924, the values of the rectangle information CX0, CY0, CX1, CY1 registered in the table TBL5 are corrected so that the character positions and the character sizes in the same character string are aligned with each other. As a correction method, simply use the average value in each character string, or use the recognized character code, first find the size of the character frame from the shape information of the character, and then determine the position and size of this character frame. For example, a method of correcting based on the information can be used.

The word information included in each character string thus obtained is stored in the form format storage area 982 as a table TBL6 shown in FIG. The above table TBL6
Are pointer information WC1 to WC1 to table TBL5 as information on the number WN of characters constituting a word and information on each character.
It consists of each item of WCP. Here, p is the maximum value of the number of characters WN per word. The above table TBL6
Is used in the second embodiment of the form format recognition program 2 described below.

FIG. 41 is a flowchart showing a second embodiment of the subroutine program executed in the form format recognition step 2. The second embodiment includes a step 21 for recognizing the physical structure of a form described above and a step 22 for recognizing the logical structure of the form described below. In the present invention, the “logical structure of a form” refers to a structure that extends to the meaning of the form, such as a line / column structure of a field of a form described later and information on fields.

FIG. 42 shows a first embodiment of the subroutine program called in the logical structure recognition step 22.
The program 22 is prepared in an area different from the program 2 in the form format recognition unit 972. In the first embodiment, the subroutine 22 comprises only a step 221 for normalizing the positions and sizes of characters, ruled lines, and the like entered in the form so as to be arranged in rows and columns.

FIG. 43 is a detailed flowchart of the line / column normalization step 221. In step 2211, the reference position of each character stored in the table TBL5 and the coordinates of all the intersections and endpoints of the line segments stored in the tables TBL1, TBL2, and TBL4 are determined.
This is stored in the table TBL7 shown in FIG. The reference position of the character pattern will be described later in detail. Table T
BL7 indicates the character reference position, the intersection of the line segment,
The end point is composed of the coordinates (MX, MY) and each item of the information code MC. The information code MC is
The character information indicates a character code indicating a character pattern, and the line information indicates a character code indicating a pattern (shape) of an end point or an intersection as described later with reference to FIG. In step 2212, the data stored in the table TBL7 is sorted according to the value of the X coordinate MX.
Sorting is performed in ascending order (arrangement in ascending order of data), and its algorithm is well-known. Since it is described in detail on page 62 and subsequent pages of the document by Mr. Wirth, the description is omitted. In step 2213, the table T after sorting
The data of BL7 is sequentially taken out, and first, an amount d to be normalized for the data is obtained. When the width of a row and a column to be normalized is defined as an RC pixel,
An integer multiple of and closest to the value of MX is selected and calculated as the difference between this and the MX value. In step 2214, the above d is added to the MX value of all data. In step 2215, as in step 2212, TBL7 is sorted according to the value of MY.
Thereafter, in steps 2216 and 2217, the values of MY are sequentially normalized as in steps 2213 and 2214, respectively. After normalization of all coordinate data is completed, in step 2218, a character code indicating a line segment intersection or a line segment pattern between end points is generated together with the position information. Finally, in step 2219, a process of contracting the redundantly expanded column or the like while maintaining the geometric structure is performed. In the above embodiment, the normalized values d of the coordinates MX and MY are all positive values, but a negative value can be allowed by simply adopting a value closest to a multiple of RC.

FIGS. 45A and 45B show characters, intersections of line segments,
It is a figure which shows the position reference point RP (black circle) of an end point pattern. FIG. 7A shows an example of a reference point of a full-width character pattern, and FIG. 7B shows an example of a reference point of a half-width character pattern.

FIG. 46 is a flowchart of a second embodiment of the subroutine program called in the logical structure recognition step 22. The second embodiment has a step 222 for recognizing a field position of a form. The second embodiment includes only the step 222 as shown in FIG. Digit normalization step 22
1 is executed, the field position recognition step 22 is executed.
2 is executed.

FIG. 47 shows the above field position recognition step 2.
22 is a detailed flowchart of FIG. Step 2221
Then, in order to easily recognize the field position, the table shape is normalized. Figure shows an example of table shape normalization processing
48 (A) and (B). In the figure, (A) is a representative example of a process of adding ruled lines 42A and 42B to a table 41 in which ruled lines are omitted. This processing checks, for example, the presence or absence of a vertical line segment connected to each end point of the horizontal line segment, and if not, generates a new horizontal line segment of another vertical line segment intersecting the horizontal line segment. Can be easily realized. More simply, for a set of parallel horizontal line segments, two new vertical line segments are generated between the start point and the end point of the uppermost line segment and the lowermost line segment. It can also be realized by: FIG. 4B shows Table 43 in which each corner of the outer frame has an arc shape, and Table 4 in which the outer frame has a straight line.
4 is a representative example of a process of converting the data into a C.4. This processing can be easily realized by, for example, replacing the arc data with two pieces of vertical and horizontal line segment data indicating the corners of the table. More specifically, for each element of the stored table TBL3 (FIG. 23), for example, when AA0 = 0, LX0 = AX +
AR, LY0 = AY, LX1 = LX0, LY1 = AY−
AR vertical line segment, LX0 = AX + AR, LY0 = AY
-A vertical line segment of AR, LX1 = AX, LY1 = LY0R is generated in TBL1. The same processing is performed for AA0 of 90, 180, and 270.

In step 2222 of FIG. 47, a standard field, that is, a rectangular area (field) surrounded by four line segments and containing no other field is sequentially extracted. In this process, for example, the horizontal line segments are sorted in the order of their vertical positions, and the first horizontal line is first selected as a reference horizontal line according to the order. Next, check in order from the left vertical line, select a set of two vertical lines that intersect with the reference horizontal line and extend downward, and then cross the set of these vertical lines and are the closest horizontal line to the reference horizontal line To find an area in which the two horizontal lines and the two vertical lines having the reference horizontal line as the upper side are four sides. When the above operation is repeated and all the fields having one reference horizontal line as the upper side are found, the next horizontal line is selected as the reference horizontal line, and the same operation is repeated, so that all the fields formed by the ruled lines are sequentially output. Can be found. By checking whether or not each of the quadrilaterals thus detected satisfies the standard field conditions described above, only the standard fields can be extracted.

In step 2223, step 29
In the same manner as in step 21, character string information is created from the character information stored in the table TBL5, and steps 2224 and 2225 are repeated for each character string.

In step 2224, the extracted character string is collated with a word which can generate a special field registered in advance. Word matching is performed in the same manner as in step 2922. If there is a word that satisfies the predetermined condition as a result of the comparison, in step 2225, a special field extraction process is performed according to a predetermined procedure. Here, the special field is a virtual field that is not necessarily surrounded by line segment information but indicates a position where content data is printed when a form is issued. For example, as shown in examples EX6 and EX7 described later, a special field is defined for "year" at a circumscribed rectangle and at a position the same size as this and a predetermined distance to the left of the rectangle. In this processing, for each word that can generate a special field, conditions for actually generating the special field, and information such as a position and a size to be generated are described as the processing of this step 2225. Can be realized by The above conditions for actually generating a special field include, for example, a case where the ratio of the interval between the characters "year" and "month" to the size of each character is larger than a predetermined value. Describe the conditions for determination.

In step 2226, step 22
22 and whether each field extracted in step 2225 is an input / output field in which data may be entered at the time of form issuance or a fixed field in which no new data is entered. I do. This determination can be made simply by checking whether or not a character string exists in a field (area range), for example, and the program stores in advance the meaning of the character string as a word. It is also possible to make a determination by collating with a list of words that may exist in the input / output field and referring to whether or not the character string position in the field is biased.

At step 2227, the field definition information of the field determined as the input / output field is registered in a table TBL8 described later. Step 22
At 28, the field definition information is registered in the table TBL8 for the field determined as the fixed field.

FIG. 49 shows the structure of the field table TBL8. The table TBL8 includes upper left coordinates (FX0, FY0) and lower right coordinates (FX1, FY) of the field area.
1) and a field name FN. As for the field name, a character string described in the area range can be used as it is for a fixed field, and a symbol indicating an input / output field is registered for an input / output field. The table is formed in the form format storage area 982.

FIGS. 50A and 50B show an example of the execution result of the field position recognition step 222. FIG. EX6 shown in FIG. 7A shows the position of the fixed field recognized from the original image EX1 shown in FIG. 3, and EX7 shown in FIG. 7B shows the position of the input / output field recognized from the original image EX1. It was done.

FIGS. 51A and 51B are flowcharts showing a third embodiment of the subroutine program called in the pre-recording structure recognition step 22. FIG. This third embodiment includes the field position recognition step 222 described above,
The method includes a step 223 for recognizing a relationship between a plurality of fields using a recognition result of a field position, and a step 224 for logically recognizing an attribute of each field. Depending on whether the field position recognition step 222 is executed directly or after the row / digit normalization step 221 is executed, there are embodiments of the configuration shown in FIG.

FIG. 52 shows the above-described field relationship recognition step 2.
23 shows a detailed flowchart of the embodiment. Step 2231
Then, one fixed field is selected from the field table TBL8. In step 2232, it is checked whether or not an input / output field exists below the selected fixed field. If there is an input / output field, the determination as to whether or not another input / output field is present further below the input / output field is repeated until there is no input / output field adjacent below (steps 2233A to 2233A).
233B). If there is no input / output field below the fixed field selected first, it is checked whether there is an input / output field in contact with the right side (step 2234). The operation of selecting the input / output field located to the right of the fixed field is repeated until the field no longer exists (step 2).
235A-2235B). If there is no input / output field on either the lower or right side, step 22
At 36, exception processing is performed. In the exception processing, for example, the input / output field extracted as a special field in step 2225 is combined with a fixed field which is a character string serving as a key for extracting the special field. A process that cannot be associated and a process of extracting a fixed character string having no corresponding input / output field are performed. In step 2237, a unit is generated by combining the fixed field selected first and the input / output field selected in step 2233A or 2235A. When the exception process 2236 is performed, a unit for a special field, a unit configured with a fixed field alone, or a unit including a field not surrounded by a dashed line is generated. The above processing is repeated for all fixed fields of the field table TBL8.

FIG. 53 shows the above-mentioned unit generation step 223.
7 shows the structure of a unit table T4 for storing the unit information obtained in step S7. The table T4 is a field table TB indicating fixed fields belonging to the unit.
A pointer FF to L8, the number of input / output fields PN belonging to the unit, and pointers PF1 to PF1 to field table TBL8 indicating each input / output field belonging to the unit
It consists of each item of PFp. Here, p indicates the maximum value of the number PN of input / output fields in each unit. The table T4 is prepared in the work area 985. Fig. 54
Shows an example of the state of the unit obtained by executing the field relation recognition step 223 on the field position recognition results EX6 and EX7. Step 22
Unit U consisting of the input / output field selected in 33
V1 to UV4 are vertical units, units UH1 to UH4 each consisting of the input / output field selected in step 2235.
Is a horizontal unit, the unit UI1 in which the fixed field is recognized as being isolated in step 2236 is an isolated unit, and the units UE1 to UE3, which are also composed of fields not surrounded by ruled lines in step 2236, are shown as exceptional units. In this example, the characters "year",
For three characters (string) of "month" and "day", UE1-
The three units of UE3 have been created,
One unit for the character string “year / month / day” may be generated by the method of defining the process of H.225.

FIG. 55 shows a field attribute recognition step 224.
3 is a detailed flowchart of the first embodiment. At step 2241, one unit is selected from the unit table T4. In step 2242, the name of the fixed field of the unit is compared with the unit knowledge table registered in the form format recognition unit 972 in advance, and the function of the unit is obtained. If the function cannot be determined only by the field name, in step 2243, information such as the semantic relationship with another unit and the arrangement of the unit on the form is collated with a predetermined rule to determine the function. If the function of the unit obtained by these processes is a function input unit that calls a function, calculates a value, and substitutes the value into an input / output field, the function input attribute is stored in a table T described later.
BL9 and TBL10 (Steps 2244a to 2244a).
2245a). If the unit function is a file input unit that searches for a file to obtain a value and substitutes it for an input / output field, the file input attribute is set in the tables TBL9 and TBL11 (step 22).
44b-2245b). When the function of the unit is a calculation input unit that substitutes a value obtained by calculation based on a value substituted into another input / output field into the input / output field, the computation input attribute is set in the table TBL9,
Set to TBL12 (steps 2244c to 2245)
c). If the unit function is a unit repeat unit having the meaning of repeating the same set of items, the repeat attribute is set in the tables TBL9 and TBL13 (steps 2244d to 2245d). If the unit function is not any of the above, it is determined that the key input unit sets an input value from the keyboard in the input / output field, and the key input attribute is set in the table TBL9,
It is set to TBL14 (step 2245e).

FIGS. 56 to 59 show the structures of various knowledge tables referred to in determining the unit function in the name matching step 2242 described above.

The knowledge table KNW1 shown in FIG. 56 has a name UN of a fixed field of a unit having a function input attribute.
And the name FUN of the function to be called is managed as the item.

The knowledge table KNW2 shown in FIG. 57 shows an example of a file to be searched by a unit having a file input attribute. In this example, "item name" and "unit price" are included as the items, and in particular, the unit price can be uniquely searched from the item name.

The knowledge table KNW3 shown in FIG. 58 includes a fixed field name UN for a unit having a calculation input attribute, a calculation type OP, and fixed field names OP1 to OP of other units as parameters used for calculation.
N is managed as the item. Here, it is assumed that the type of parameter required for calculation can be uniquely determined by the type of calculation OP. Knowledge table KNW4 shown in FIG.
Is a table for managing attributes to be set in the unit having the key input attribute, and includes a fixed field name UN,
The attribute has, for example, a symbol AN indicating whether it is numerical information or character information as an item.

FIGS. 60 to 65 show table structures for attribute setting used in steps 2245a to 2245e of the field attribute recognition processing 224 described above.

The table TBL9 shown in FIG. 60 is a table for managing the entire unit created based on the unit work table T4. The above table is a field table TBL8 indicating fixed fields belonging to the unit.
FF, the number of input / output fields PN belonging to the unit, and the pointers PF1-PFN to the feed table TBL8 indicating the input / output fields belonging to the unit.
, A function type UA of the unit, and a pointer UAP to tables TBL10 to TBL14 storing detailed function information thereof. Here, N indicates the maximum value of the number PN of input / output fields per unit. The above table T4 has the following tables TBL10
Along with the TBL 14, it is prepared in the form format storage area 982.

A table TBL10 shown in FIG. 61 is a table for storing information indicating the details of the function of the function input attribute unit.
UNN, number of parameters required to call the function R
N and its items PAR1 to PARN. Here, N indicates the maximum value of the number of parameters RN.

The table TBL11 shown in FIG. 62 is a table for storing information indicating the details of the function of the file input attribute unit. Number D of
N and its items TM1 to TMN. Here, N indicates the maximum value of the number of parameters DN.

The table TBL12 shown in FIG. 63 is a table for storing information indicating the details of the function of the calculation input attribute unit, and includes, for example, the calculation type CALN, the number CLN of parameters required for calculation, and the actual state thereof. An O
It consists of each item of P1-OPC. Here, C indicates the maximum value of the number CLN of parameters.

A table TBL13 shown in FIG. 64 is a table for storing information indicating details of the function of a unit having a repetition attribute.
1 to RFFN, the number M of units to be repeated, and pointers RU1 to RUM to the TBL 9 which are the actual items.

The TBL 14 shown in FIG. 65 is a table for storing information indicating the details of the function of the key input attribute unit. DH, an interval DD between adjacent characters, and items of format information DJ indicating distinction such as left alignment, center alignment, and right alignment.

FIG. 66 shows an original image EX1 obtained by the above processing.
As an example of the form logical structure recognition result generated from the table, a specific example of the tables TBL9 to TBL14 is described as TBL9-EX.
ＴTBL14-EX.

FIG. 67 is a flowchart showing a third embodiment of the subroutine program executed in the form format recognition step 2. In the third embodiment, a step 21 for recognizing the physical structure of a form described above, a step 22 for recognizing a logical structure of the form, and a step for automatically generating a form creation program described below are performed. Step 23 is comprised. The form creation program referred to here means a program executed in the form creation processing 3 thereafter.

FIG. 68 shows a form creation program generation step 2
3 is a flowchart showing one embodiment of a subroutine program called in 3; This subroutine program is stored in an area different from the above program in the form format recognition unit 972. Hereinafter, the processing content of the above subroutine will be described in detail with reference to the actual processing result EX9 shown in FIG.

EX 9 is an original image EX of the form shown in FIG.
1 is actually generated using the tables TBL8-EX to TBL14-EX shown in FIG. In the present embodiment, the grammar of the generated program conforms to the programming language “pascal”.

First, in step 231, a fixed part of a program that does not depend on the result of recognizing the logical structure of a form is generated. In the example EX9, statements such as l1, l2, l16, and l32 are generated as the program fixing unit. The generation of the program is actually realized by defining a character string as shown in the above example in a predetermined memory area defined in the subroutine program 23. In step 232, a part for declaring variables used in the program is generated. In the example EX9, the generation of the statements 13 to 15 corresponds to this. The variables used in the above program are determined from the number of units, the number of repeating units, and the like in the result of the logical structure recognition. Next, in step 233, the example EX
As shown in statements 16 to 115 in FIG. 9, a part that declares a function or procedure used in the above program is generated. These functions or procedures are determined from the function name of the function input attribute in the logical structure recognition result. This program portion is generated by extracting a character string determined in the above step from several function declaration strings defined in the program 23 in advance,
This is realized by transcribing this to the creation program storage area on the memory. In the above function definition, by providing a function of displaying a cursor at a position where an input character string is to be inserted particularly in the statements 16 to 17, the user can easily perform the following form issuance processing. In step 234, a repetition control unit is generated by referring to the table TBL13. The repetition control units are generated by the number of repetition units until the end is detected in the determination step 230c. As a result, in the example EX9, the statements 117, 118,
124 is generated. In step 230a, input / output fields belonging to the repetition unit are selected one by one, and in step 235, a processing unit for the input / output field is generated. In the example EX9, this causes the statement l1
9 to 123 are generated.

The generation of all repetitive parts is completed (step 230c), and then the generation of non-repeated parts is performed. First, in step 230d, one of the input fields of the non-repeated part is selected, and then in step 235, a processing part for the input / output field is generated. By repeating this, statements 125 to 131 of the example EX9 are generated. When the processing units have been generated for all the input / output fields of the non-repeating unit (step 230d), the program is shaped in post-processing step 236, and step 237 is performed.
Then, a character string called such a source program is represented by C
It is translated into a machine language that can be directly executed by the PU 92. The translation processing into the machine language is realized by activating a known language compiler (program) prepared in a part of the memory 97 in advance.

FIG. 70 is a flowchart showing the detailed processing contents of the input / output field processing unit generation step 235. In this processing, first, in step 2350a, the attribute of the given input / output field is determined by referring to the table TBL9. If the result of the determination is a function attribute, the attribute of the given input / output field is determined by referring to the table TBL10 in step 2351. If the result of the determination is a function attribute, a processing unit for the function attribute is generated by referring to the table TBL10 in step 2351. In the example EX9, l29,
Three rows of l30 and l31 are generated by this step.
If the result of the determination is the file input attribute, step 2
At 352, a processing unit for the file input attribute is generated by referring to the table TBL11. In the example EX9, l
21 lines are generated by this step. If the result of the determination is a key input attribute, a key input attribute processing unit is generated by referring to the table TBL14 in step 2353. In the example EX9, two rows of 119 and 120 are generated by this step. If the result of the determination is the calculation input attribute, the type of operation is obtained by referring to the table TBL12 in step 2350b. If the operation type is a product, a processing unit for the product operation is generated in step 2354 by referring to the table TBL12 again. In the example EX9, one line of 122 is generated by this step. When the operation type is the sum, in step 2355, the processing unit of the sum operation is generated by referring to the table TBL12 again. In the example EX9, four lines 125-128 are generated by this step.

In the above description of the form format recognition step, for simplicity, it has been described that recognition is generally performed automatically. However, in actual applications, erroneous recognition is performed depending on, for example, the state of image input. May occur. In order to cope with such a case, for example, a confirmation operation by an operator and a processing step for manually correcting the automatic recognition result as needed may be inserted at each stage of the recognition. Especially physical structure recognition step 2
1. Immediately after each of the line digit normalization step 221, the field position recognition step 222, the field relation recognition step 223, the field attribute recognition step 224, and the form creation program generation step 23 in the logical structure recognition step 22, the above-described processing results It is convenient to insert a processing step for confirmation and correction.

The form format recognition step 2 has been described above, and an embodiment of the form creation step 3 will now be described.

FIG. 71 is a flowchart showing a first embodiment of the form creation step 3. In this embodiment, the form creation step 3 uses the physical structure (that is, the information registered in the tables TBL1 to TBL5) obtained in step 21 in FIG. It consists only of the generating step 39.
The generated output form format data is stored in the output form storage area 9.
84.

FIG. 72 shows a flowchart of a subroutine program called in the output format data generating step 39. This subroutine program is prepared in a memory area different from the program 3 in the form creation unit 973. In the first step 391, format data for printing these line segments by, for example, the printer 95 is generated by sequentially referring to the table TBL1 in which the recognition results of the line segments (straight lines) are registered. In step 392, the format data necessary for printing these oblique lines by the printer 95 is generated by sequentially referring to the table TBL2 in which the oblique line recognition results are registered. In step 393, format data for printing these arcs with the printer 95 is generated by sequentially referring to the table TBL3 in which the recognition results of the arcs are registered. At step 394, format data for printing these broken lines by the printer 95 is generated by sequentially referring to the table TBL4 in which the recognition results of the broken lines are registered. In step 395, a table TBL5 in which the character recognition result is registered
Are sequentially referred to to generate format data for printing these characters by the printer 95.

In the above step 39, print format data finally obtained in the print form storage area 984 is generated in a data format conforming to the specification of the printer 95 to be connected. The data format may be, for example, a bitmap format that a normal printer has as an external interface, or a command sequence format.

FIGS. 73 (A) and 73 (B) are explanatory diagrams in the case where print format data is generated in the bitmap format in step 391. FIG. FIG. 7A shows one piece of line segment data stored in the table TBL1, and FIG. 7B shows the corresponding bitmap print format data. In the data of FIG. 6B, an array of white pixels corresponding to the bits to be printed in one-to-one correspondence is prepared in the memory, and a portion corresponding to the line segment defined in the table TBL1, ie, FIG. Can be easily obtained by substituting black pixels into the rectangular area 5 indicated by oblique lines. With respect to oblique lines, arcs, and broken line data, print format data can be created in the same manner as in the case of the above line segments. Note that the character data should be created in such a manner that a character font pattern prepared separately in advance is directly or after modification such as enlargement / reduction as necessary, and copied to a predetermined position on the white pixel array. Can be.

On the other hand, FIG. 74 is an explanatory diagram in the case where print format data is generated in the command sequence format in step 391. (A) shows one line segment data stored in the table TBL1, and (B) shows print format data in a command sequence format for drawing a corresponding line segment. The data in FIG. (B) is stored in the table TBL1.
Can be easily obtained by defining in advance in step 391 a rule for generating a predetermined command sequence according to the value of each item. Oblique lines, arcs, dashed lines, and character data can be created in the same manner as in the case of the above line segments. Note that the printer 95 is
If the contents of BL1 to TBL5 cannot be output with the graphic type and precision as they are, the procedure for obtaining approximate data suitable for the printer from the print format data is also defined in the print format data generation step 39.

FIG. 75 is a flowchart showing a second embodiment of the form creation step 3. In the present embodiment, prior to the above-described print format data generation step 39, first, the form format data stored in the tables TBL1 to TBL5 is displayed on the console 91 (step 31), and the format data is edited according to the instruction of the operator, ie, Processing such as addition, update, and deletion of data is performed (step 32). The display of the form data on the console 91 (step 31) can be realized by, for example, expanding the print format data in the bitmap memory of the console 91 in the same manner as in the print format data generating step 39. The editing process performed in step 32 is the line correction step 2 in the post-processing step 29 described with reference to FIG.
As in the case of 91 and the character correction step 292, techniques widely known in word processors and CAD systems can be applied, and a description thereof will be omitted.

According to the above embodiment, the displayed form EX is displayed.
It is also possible to add the edit to 1 to generate the print format data of the form EX10 whose layout has been changed as shown in FIG. 76, for example. According to the second embodiment,
For example, it is also possible to create a form EX11 as shown in FIG. 77 and generate its print format data. in this case,
The form data is printed as a format. When only printing is intended, there is no need to distinguish between format data and content data, and such a usage method is possible.
The print format data finally obtained by the above processing is
As in the first embodiment, the print form storage area 984
Registered in.

The above two embodiments have been described on the premise that the physical structure recognition result in the form format recognition step 2 is used. However, the table TBL7 which is a part of the logical structure recognition result, that is, the row digit normalization, is used. Similar processing can be performed based on the information on the result of step 221. The processing in this case can be realized by the same method as the processing for handling the character code in the above embodiment. The format data to be printed at this time does not necessarily match the format of the input form geometrically, but since all the elements of characters and ruled lines are arranged in a lattice, the form format editing processing 22 Is very easy for the operator.

FIG. 78 is a flowchart showing a third embodiment of the form creation step 3. In this embodiment, first, in step 33, form content data conforming to the form format recognition result is created using the form format recognition result described above, and stored in the form content storage area 983. The form content data can be stored, for example, in the same format as the table TBL5. Next, in step 34, the data is converted into data for printing (output) in the same manner as the form format data in the previous embodiment, and stored in the output form storage area 984.

FIG. 79 is a flowchart showing a first embodiment of the form content data creation step 33. Step 3
In step 1, similar to the second embodiment, the form data of the form is displayed on the console 91. In step 331, the key information input by the operator from the console 91 is read. The key information includes, for example, cursor movement, content data to be entered in a form, commands for instructing various functions, and the like. If the information is other than the end command, in step 332, input processing of form content data is performed according to the input key information. In the input process, for example, a process of newly defining a character or a character string specified at a cursor position, and a process of changing or deleting such input data thereafter can be performed. Through the above input processing, for example, content data in the same format as the character table TBL5 is newly defined in the form content storage area 983.

According to the form content data creation step 33, for example, based on the form formats EX1 and EX10 or the format EX12 shown in FIG.
Content data such as EX2, EX11, and EX13 shown in FIG. 81 can be created.

FIG. 82 is a flowchart showing a second embodiment of the form content data creation step. Step 3
At 33, an arbitrary input / output field is selected from the form. The above selection may be made automatically,
It may be performed by an instruction from the operator. Step 33
In step 4, content data corresponding to the selected input / output field is searched from a file or the like. This processing can be performed by automatically searching for the corresponding file based on the name of the unit to which the selected input / output field belongs, or by the instruction of the operator. As a result of the search, if there is data corresponding to the input / output field, the data is input to the input / output field in step 332. That is, in the same manner as in the input process 332 in FIG. 79, for example, a format and content data similar to the character table TBL5 are newly defined in the form content storage area 983.

An example of the processing according to the second embodiment is shown in FIG.
This will be described by taking the form EX12 shown in FIG. 80 as an example. The file searched in step 334 has, for example, a configuration as shown in FIG. This file is composed of management information F1 of the entire file, a title file F2, a name file F3, a summary file F4, and a body file F5.

First, when the input / output field relating to the title of the format EX12 is automatically selected, the corresponding data is read out from the title file F2, and as shown in the form EX13, the data corresponding to the input / output field is located at the position corresponding to the input / output field. Defined as content data. In the same manner, the contents of the input and output fields of the name and the gist can be defined in the same manner. According to the above-described embodiment, simply by inputting format data such as EX12 as an image, the content data can be automatically generated from a basic file, and a form such as EX13 can be easily created.

FIG. 84 shows a third embodiment of the form content data creation step 33. In the present embodiment, a form is created as shown in FIG.
And the field attribute recognition result of step 224 described in FIG. In the first step 31, the form of the form is displayed on the console 91. In the next step 333,
One of the input / output fields of the form to which the processed flag is not added is selected. The processed flag is an array of one for each field. If the attribute of the selected input / output field is a function input attribute, a function is calculated in step 335 to newly define the function as content data and add a processed flag. When the attribute of the selected input / output field is the file input attribute, first, it is checked whether or not all the parameters necessary for searching the file of the field, that is, TM1 to TM (DN-1) in the table TBL11 have been input. I do. If input has been completed, the content data is retrieved from the file in step 334, this is defined as new content data, and a processed flag is added. When the attribute of the selected input / output field is the calculation input attribute, first, it is determined whether or not all the parameters necessary for executing the calculation formula of the field, that is, all of OP1 to OP (N-1) in the table TBL12 have been input. Confirm. If input has been completed, the corresponding calculation formula is executed in step 336, the result is defined as new content data, and a processed flag is added. If the attribute of the selected input / output field is the key input attribute, data input processing from the keyboard is performed in step 331, the result is newly defined as content data, and a processed flag is added. When one input / output field process is completed, the format and updated content data are displayed on the console 91 in step 31. Thereafter, the above processing is repeated until the above-mentioned processed flag is added to all the fields.

FIG. 85 is a fourth embodiment of the form content data creation step 33. In the present embodiment, the result of the form creation program generation processing in step 23 is used to create the form. In step 337, the form creation program obtained in step 23 is edited. With this processing, the operator can correct a recognition error or extend the recognized program as necessary. In step 338,
Activate the form processing program. In the following, since a form is created under the control of this program, the program may monitor the end thereof.

In the above embodiment, the normalization process of the row digits is performed on the result of the physical structure recognition, and the other logical structure recognition processes such as the field position recognition are executed.
As shown in FIG. 51A, a logic recognition process such as the above-described field position recognition may be performed subsequently to the result of the physical structure recognition, and line digit normalization of data indicating the result of the logic recognition may be performed.

The description of the basic embodiment of the present invention is completed above, and a supplementary explanation will be given below mainly on the operation example of this embodiment.

FIG. 86 is a diagram showing the operation procedure of the operator and typical screen transitions according to the present invention. In the figure,
A201 is an example of a form sheet. Such paper,
In step A202, the image is input to the image input device 93 (for example, an OCR reading device, a facsimile, etc.). Step A2 above
In 02, ruled lines, characters,
Information such as an image is recognized from the image data, physical structure data of the document described in detail in the above embodiment is generated, and document information based on the physical structure data is displayed on the display screen as a screen A203. The operator compares and confirms the format displayed on the display screen with the content of the read sheet in step A204.

If there is a correction or addition, the cursor is set at the correction or addition position on the display screen, and the correction and addition work is performed while observing the result of the correction and addition. When confirmation and correction are completed, a continuation key is input from the console 91 (keyboard or the like), and the logic recognition process is started. In the logic recognition processing, the shape of a ruled line, a character string, and the like are cut out using a knowledge base, and the position, size, attributes, and the like of the input / output field are determined based on the content and meaning. The result is A20
The input / output field determined is associated with the data displayed on the display screen of No. 3 and the determined input / output field is displayed on the display screen of A205 in a reverse form in which white and black are inverted with respect to other parts, for example. The displayed contents are checked in A206, and if there are corrections and additions to the position, size, and attributes of the input / output fields, the corrections and additions are performed by interactive processing while observing the results.
At this time, if necessary, trial printing is performed, and the position of the input / output field is finely adjusted. After the confirmation and correction are completed, the continuation key is input from the keyboard or the like of the console 91, and the processing shifts to the next processing program creation processing.

First, processing logic is generated for each input / output field using the nature, position, and knowledge base 340 of the input / output field.

For example, the data handled by the data processing system is stored in the file 800 in the order of item number, item name and unit price, while the knowledge base 340 associates the item number, item name and unit price with each other. Is stored in advance. Further, in the above association, it is assumed that the knowledge base stores the knowledge that the product number is a key, and that the amount = unit price × quantity and the total = amount + amount +... + Amount.

When the character group such as the product number, product name, unit price and the like recognized from the form paper is compared with the knowledge base 340, it can be seen that by inputting the product number, other values can be obtained by file search. . As a result, it is possible to generate a processing procedure for inputting a product number and then a processing procedure for searching the file 800 to obtain a product name and a unit price. However, in practice, the terms may not exactly match and may differ from each other, such as a product number instead of a product number. Therefore, in the knowledge base, not only the product number but also the product number, product number, product name number, product name No.
By registering such similar words in advance, it is possible to generate a processing procedure even when the item name does not exactly match the item name read from the form.

Referring to the knowledge base for the character group of quantity and amount, it can be seen that by inputting the amount, the value of the amount can be obtained by calculating unit price × quantity. From this, it is possible to generate a processing procedure for inputting the quantity and then a processing procedure for calculating the amount.

Further, since there are a plurality of item columns such as a product number, the above-described process of inputting the product number and quantity, repeating the process of obtaining the product name, unit price and price, and finally calculating the total of each price is performed. It can also be generated. By these methods, the form format A201 indicated
A processing procedure (program) for obtaining each value to be output is generated. The created program is stored on a disk or the like as in A207. The created program can be immediately executed in step A208. A209
Shows a screen during execution of the created program.

The operator inputs, for example, a plurality of combinations of the product number (code) of the product and the quantity of the product from the keyboard. The program searches the file 800 prepared in advance for the product name and unit price corresponding to the product using the product number input by the operator as a search key, calculates the price by multiplying the unit price and quantity for each product number, Calculate the total amount of money in a transaction consisting of multiple sets. These execution results can be printed at the finely adjusted positions of the input / output fields contained as recognition information, as indicated by A210.

FIG. 87 shows an execution sequence of a main program module corresponding to the operation sequence of FIG. 86.
Hereinafter, the processing and the flow of data will be supplemented with reference to FIG.

FIG. 88A shows the flow of the recognition process A303. FIG. 88 (B) shows an example of a form of a slip handled in the recognition processing. Here, for example, the symbols CE1, C
As shown by E2, three sides or four sides are ruled lines 100
The area surrounded by is referred to as a cell. First, in step A401, a cell group is recognized from the ruled line information input.
In step A402, a meaningful character string is searched for and recognized from characters in the cell and characters outside the cell. Next, in step A403, an input / output field in the cell, an input / output field outside the cell, and the like, which are input / output fields, are determined from the type of the cell and the meaning of the character string. Thereafter, the attribute and property of the input / output field are determined from the meaning of the character string, and stored in the data file 99 (disk, etc.) in step A404 together with information on the position and size of the input / output field as recognition information.

FIG. 89 is a diagram showing the flow of the cell recognition step A401. In step A501, first, a line segment is corrected. FIGS. 90A to 90F show examples of line segments to be corrected. FIGS. 7A to 7C are examples of correction of the corners of the table. The angle is corrected by extending the vertical and horizontal line segments and correcting the arcs shown in FIGS. (A) and (B) and the cut shapes shown in FIG. (C) into shapes in which vertical and horizontal straight lines are orthogonally connected. . Line segments that are not completely in contact with each other, as shown in FIGS.

Next, in step A502, the start coordinates (X, Y) of the line segments are rearranged in the order of (Y, X), the area is divided one by one, and cells are created. An operation example of this region division processing will be described with reference to FIG. In this example, state A70
As shown by symbols a to g in 8, the table is assumed to be composed of seven line segments.

First, in the state A701, the top horizontal line segment a is selected, and then in the state A702, the vertical line segment b intersecting with the above line segment is placed. Divided into two areas. When the next vertical line segment c is placed in the state A703, the region indicated by the symbol A is divided, and the symbols A, A,
There are three areas shown by c. When the next vertical line segment d is placed in the state A704, the area indicated by the symbol c is divided into four areas indicated by the symbols a, a, c, and d. In this way, for example, while the horizontal line segment is selected from top to bottom, and the vertical line segment is selected from left to right, line segments that intersect each other are placed one after another. It is divided into nine areas indicated by A to K. Where:
Since the cell is not considered unless surrounded on all sides, the three regions A, D, and K are dropped in the post-processing, and as a result, the state A
The six regions 708 are recognized as cells.

After the recognition of the cell, in step A402, a character string inside or outside the cell is recognized. A meaningful character string is cut out from the input character information, and characters that slightly overlap the width of the first character on the basis of the left-to-right direction are regarded as a character string in the same row or column.

FIGS. 92 (A) to 92 (D) show an operation example of this processing. In other words, even if the line start positions of the characters should be originally aligned as shown in FIG. 7A, but the line start positions of the characters are not aligned as shown in FIG. The subsequent characters “R”, “thing”, and “item” over the character width of the character “” are regarded as being all on the same line. If one character extends over two lines as shown in FIG. 3C, it is regarded as a character on the upper line. Therefore, in this example, “store” is determined to be a character on the same line as “book”. As shown in FIG. 4D, even if the character string has a line start position that should be aligned, but the character line start position is not uniform, a character string within one character width is considered to be in the same character string. to decide.

Next, with reference to FIGS. 93A to 93C, an operation example of a process of extracting a meaningful character string from characters in the same row and the same column will be described. The cutout processing is performed by converting the character spacing and matching with terms and words in the knowledge base 340. For example, if the character space is widened in the middle, as in the case of the space between “place” and “Mr.” in FIG. "Address" is cut out as one character string. Conversely, when the character spacing changes from the character spacing A to the narrow character spacing A as shown in FIG. 8B, the character "telephone" before the change is cut out as one character string. In the example shown in FIG. 9C, the intervals A and B are the same, and the interval C changes narrowly, and the interval A increases. In the case of this example, if it is determined that the change in the interval is the boundary of the character string, “commodity unit” is cut out. If the interval is once narrowed and then widened as in (c) and (d), a character string is cut out at the once changed boundary, and the cut out character string is matched with a term or word in the knowledge base 340. . If the character string exists in the knowledge base 340, it is determined that the extraction of the character string has been correct up to that point. If the corresponding term or word does not exist, it is determined that the last character is connected to the next character, and it is delimited before the character “Single” with a narrow space between “U” and “Product” as one character string. cut. By using the character spacing and the collation of the knowledge base in this way, it is possible to cut out the correct character strings “product”, “unit price”, and “sales” even in the case of FIG. The same applies when characters are arranged vertically.

When the cutout processing for each row or column is completed, it is determined whether adjacent character strings in a plurality of rows and columns in the same cell are combined into a character string having one meaning. Judge. FIGS. 94A to 94E show an operation example of the processing. In the example of FIG. 9A, “customer” and “code” are cut out, but in fact, “customer” and “code” should be one character string “customer code”. As described above, in a process of integrating two or more character strings into one character string, the interval between lines is within the vertical width α of the standard character CH, and the first or last position of each is aligned. If you do. However, if the displacement is within the width β of the standard character, the error is treated. In the case of FIG. 7A, the vertical interval between the first characters "T" and "U" of each character string is Wα, the horizontal error is Wβ, and the horizontal error between the last characters "First" and "D" is W. If W'β, then Wα
If <α and Wβ <β or W′β <β, these are recognized as a character string to be integrated into one, and “customer code”
And

The character string is cut out basically by the above method. As shown in FIGS. (B) and (C), the cell is vertically long and has two columns and n rows (n> 2). As shown in FIG. 4D, the following method is used for the one having two rows and two columns. That is, when the cell is vertically long and has 2 columns and n rows, if the intervals W1, W2,... Between the first characters of each row are larger than the vertical size α of the standard character as shown in FIG. If it is small as shown in FIG. 7B, characters are cut out in the vertical direction. Thus, in the case of FIG. 9B, the character string “use period date” is cut out. In the case of two rows and two columns, as shown in FIG.
340 is determined by matching. In the case of FIG. 3D, the horizontal reading is "prefecture" and the vertical reading is "prefecture". In this case, "prefecture" of vertical reading is adopted by collation with the knowledge base. If it cannot be determined by matching the knowledge base 340, the horizontal reading is used as a reference.

In addition to the cases described above, there is a case where a character in a character string has one meaning in a plurality of lines. "No" and "to" at the end of the extracted character string or at the beginning of the next line
If there are continuation characters such as "or""and""and"
Regardless of the interval, it is one character string. In FIG. (E), the first line ends with "handler's" and the character "no" indicating continuation is attached, so that the next line is combined with "handler's address" and cut out. .

The cell recognition step A40 described above.
1. After the character string recognition cutout step A402, a processing step A403 for setting an input / output field to be used for inputting / outputting data from these pieces of information is performed. The cell is
For example, FIG. 95 (A) to (C) of FIG.
Are classified into three types of cells. FIG. 1A shows a cell in which a cell is made of only a character string and has no room for creating a field, and is called an item cell or a heading cell. This is the heading of the input / output field. FIG. 6B shows a cell with a heading in which a character string is present in the cell but is less than half the area, and it is determined that an input / output field is formed in the cell. FIG. (C) shows an input / output field cell which does not include any word other than a word representing a unit such as "circle" or "kg" in the cell, and the cell itself becomes an input / output field.

FIG. 96 shows a processing routine for classifying into the above three types of cells, which is executed in step A403 of FIG. 88 (A). In step A1202, the cell of the heading and the cell to be the field are linked. Next, step A
In step 1203, input / output fields in the field cell with a heading are set. In step A1204, the heading of the field cell which was not finally connected to the heading cell is searched for and connected to the character string, and in step A1205, the cell is not surrounded by a ruled line. Set input / output fields from the character string outside the cell. The link between the heading cell and the input / output field is made rightward and downward from the heading cell.

FIGS. 97 (A) to 97 (F) show an operation example of this processing. In the case of FIGS. (A) and (B), the heading cell “CE1” is linked to the field cell CE2. When the field cell is divided into a plurality of areas as shown in FIGS. (C) and (D), the entire part CE2 of the field cell is connected to the heading cell CE1. In the case where the connection between cells overlaps as in the example of FIG. 10E, the field cell CE2 is connected to the heading cell CE1. In the case of the heading cell CE3, the field cell CE4 and the field cell CE6
Is tied. The heading cell CE5 is linked to the field cell CE6 below. Here, field cell CE6
Are linked from the two header cells CE3 and CE5. In the case of such an overlap, priority is given to the one with the smaller number of connected field cells. Therefore, in this example, since the number of field cells connected to the cell CE3 is large, the field cell CE6 is
It will be linked to E5.

However, this rule is not applied to a matrix type example as shown in FIG. In this case, for each input / output field, the heading on the left side indicates the type, and the heading on the upper side indicates the content. Therefore, it is necessary to link the left and upper directions from both directions. Figure (F)
In the example, the header cells CE1 and CE3 are provided for the input / output field CE5, the header cells CE2 and CE3 are provided for the input / output field CE7, and the header cells CE1 and CE4 are provided for the input / output field CE6. , And for the input / output field CE8, the header cells CE2 and C2
E4 binds.

The setting of the input / output field in the heading field cell is determined by the position of the blank area and the contents of the character string in the cell.

FIG. 98 is a typical example of a field to be processed. FIG. (A) shows an example in which the right half is empty, and FIG.
In this example, the lower half is empty. However, as shown in FIG. 9C, even if the area is left more than half, if the cell is divided into four and the character string extends over three or more divided cells, it is treated as a heading cell and No generation is performed. When setting the position of the field, the left, top, and bottom are determined according to the content of the middle character string. If the content cannot be determined, an empty area is set.

The confirmation of the contents of the character string is performed in the knowledge base A34.
This is performed by matching with 0. FIG. 99 illustrates a simple example of this processing, and FIG. 100 illustrates an operation example thereof. The knowledge base 340 stores field position information corresponding to character strings. For example, "sama" means "left", "10,000 yen"
Contains information such as "left" and "one" as "left and right". The field creation position is determined according to the correspondence information between the character string and the field position. FIG. 100 (A),
In (B) and (D), it is made on the left, and in (C) and (E), it is made on the right. If there is no character string corresponding to the knowledge base 340, the right or lower part of the empty part is set as the input / output field.

As a result of the above-described connection, there are input / output field cells that have not been connected yet. The input / output field cell is searched for a character string serving as a heading existing therearound, and this is linked with the field cell. An example of the operation of this processing will be described with reference to FIGS. Basically, as shown in FIG. 9A, a character string in a range S between an upper and lower interval α and a horizontal interval β is searched for the input / output field CE. In the example of FIG. 7B, “money” is a character string within the search range S, and “money” is a heading (item name) for the field CE1. When there are a plurality of character strings serving as headings as shown in FIG. 3C, the character strings are linked in the order of left, right, and bottom. In the case of the above example, the heading is “Technical Department”. However, when a character that can be a unit such as "circle" or "kg" is added around the input / output field as shown in FIGS. (D) and (E), a direction opposite to the direction in which the unit is added is used. Use a character string as a heading. In the case of FIG. (D), the heading is “Technical Department”, and in the case of FIG. (E), the heading is “Amount”.

Finally, an input / output field is set from a character string not surrounded by ruled lines. FIG. 102 shows an example of a result obtained by operating this processing. In the figure, the symbols a, a,
The characters indicated by c and d are not surrounded by ruled lines, and input / output fields are set by matching these character strings with the knowledge base 340. As described above, the setting of the input / output field and the heading for the setting are linked.

For the set input / output field, the attribute is determined based on the contents of the character string of the linked heading, and the information is stored in the recognition information file while generating the recognition information. Figure
103 describes an example of the content of the recognition information. A position a, a length b, a width c, the number of characters d in the input / output area, a field area name e, and an attribute f are set for each input / output field (A to O ...). a to c are, for example, 1/10 mm
Set in units. The number of characters in the field of d is the number of characters to be put in the field, and is calculated and set from the size of the field and the standard character size.

FIGS. 104A to 104C illustrate how to determine the number of characters. In FIG. 7A, the symbols l and m added to the field CE are the vertical and horizontal lengths of the field, and α and β represent the standard spacing of characters. When the standard character size is represented by Q and P as shown in FIG. 3B, the number of characters is obtained by the following equation.

As shown in FIG. (C), when the field is composed of a series of fine cells, the number of cells is the number of characters. The obtained number of characters is a case of the standard character size, and the operator can change the number of characters later. The larger the number, the smaller the size of the characters to be printed, and the smaller the number, the larger the characters to be printed.

As the field name, a character string of a heading is set. For example, in the case of a matrix type as shown in FIG. 105, the left and upper cells are associated with input / output fields. In general, in such a case, the character string on the left represents the type, and the character string on the upper side represents the property. Therefore, “left heading + upper heading” is set as the item name.
Therefore, the input / output field name indicated by the symbol CE1 is “ballpoint pen purchase unit price”, and the field name indicated by the symbol CE2 is “ballpoint pen retail unit price”. Similarly, the symbol C
The field name indicated by E3 is “fountain pen purchase unit price”, and the field name indicated by symbol CE4 is “fountain pen retail unit price”. These set field names are used for assembling the attributes and properties of the field and the processing procedure.

Based on the field name determined by the above method, a match is searched from the knowledge base 340, and its attribute is set as recognition information. Here is the knowledge base 3
40 will be described with reference to FIG. Knowledge base 34
0 is divided into a term part 340A and a common sense word part 340B. The term part 340A contains general terms.
The common sense word part 340B is a collection of basic ending words in business terms. In general, the term at the end of a job indicates the meaning and content of the term at the end of the term. For example, the term “sales amount” expresses the property by the word “amount” at the end. Common sense word part 340B
Is a collection of words with properties. Also,
The data structure of these term part and common sense word part is the same format, and in addition to the search key term or word 341, the synonym 3
42, information such as inputs and outputs, attributes 343 such as alphanumeric characters and kanji, and arithmetic expressions (execution processing rules) 344. For other information registered in the knowledge base,
I will explain later.

Using this knowledge base 340, the process proceeds by matching the contents of the character string with terms and words. As the matching method, the following three methods are sequentially performed. First, 1: 1 matching between the character string and the term part 340A is performed. Next, 1: 1 matching between the character string and the word of the common sense word part 340B is performed, and the ending word of the character string is cut out using the common sense word part, and the corresponding word is found. The attribute of the input / output field is searched and set using the field name.

FIG. 107 is a diagram for explaining the matching processing with the knowledge data mainly on an operation example. In the figure, in step A2201, the field name is the term part 3 of the knowledge base.
It is checked whether it matches the term registered in 40A. If so, the attribute 343 in the corresponding knowledge base is set for the field name. If not, step A2
In 202, it is checked whether or not the field name matches a word registered in the common word unit 340B. If not, it is checked in step A2203 whether the word of the common sense word part 340B is included in the field name, and the included common sense word is selected. At the end of the word selected in step A2260, a word having as many characters as possible is adopted, and the attribute of the adopted word is set in the field name. For example, Step A2201 and Step A22
In the process of 02, if the field name is the sales product name,
The “sales product name” is neither in the term part of the knowledge base nor in the common sense word part. Therefore, the process of the third step A2203 is performed, and three of “article name”, “article name”, and “article” are candidates. The “product name” having the most ending and the number of characters is selected, and the field “sales product name” is included in the attribute 343 included in the record having the “product name” in the common sense word part 340A as a word. Is set.

As described above, after obtaining from the positions of all the input / output fields to the attributes, these information are written in the file A 350 as recognition information. The file A350 can be formed, for example, on the data file 99 (disk or the like) in FIG. At the same time, the input / output fields set together with the format are reversed on the display, the screen of A205 in FIG. 85 is displayed, and the recognition processing ends.

Next, a program is created based on the recognition information created as a result of the recognition processing. The created program is
It is output in a form that can be executed on the spot. The name of the file used in the creation of the program is specified by the operator after the creation is started, or the
It is specified to correspond to the term in 0 or common sense word. Here, of the input / output fields, the search key 3 of the knowledge base
The item 41 will be referred to as “search item” hereinafter in the program creation processing.

In general, a processing program using a slip or a form is made up of processing for each item of the slip or the form. This item processing is roughly divided into FIG.
A file search is performed based on the data input in the portion indicated by the symbol A or the symbol K, the corresponding record is selected, and the contents of the record are substituted into the portion indicated by the symbol A, C, K,
The process is divided into the processes of substituting the value obtained by the operation, such as the symbol や and the symbol シ. The former is referred to as search item processing, and the latter is referred to as arithmetic processing.

To create a program, first, in step A2301, the processing logic of the retrieval item is assembled. Next, in step A2302, the processing logic of the operation item is assembled. Thereafter, in step A2303, optimization of the number of times of opening the same file and optimization of redundant processing are attempted.

FIG. 109 shows the flow of the procedure for processing the search item A2301. First, in step A2401, all items that can be search items are selected. Next, step A2
A file to be searched and an item in the file to be searched are searched for for each search item selected in 402. Using these, the processing logic is assembled in step A2403.

Selection of a search item is performed based on information in the knowledge base 340. In addition to the items shown in FIG. 106, the knowledge base 340 contains information 345 indicating whether or not each term and common word correspond to each term and common word as shown in FIG. It is determined whether each field name becomes a search item by matching with this knowledge base. FIG. 111 is a flowchart of the search item selection process A2401, and FIG. 112 shows an example of the contents of a knowledge base for explaining the operation. In step A2601 and step A2602, matching between the item name (field name) and the term in the knowledge base 340 or the common sense word 341 is performed. If there is a matching term or word in the knowledge base 340, the search information 34 in step A2605.
Check if it can be a search item by looking at 5. If the matching term or word is not in the knowledge base, the ending word of the item name is checked using common sense words in step A2603, and it is checked whether the corresponding common sense word can be found and becomes a search item. For example, as shown in FIG. 112, when the item name is indicated by the symbol 103 and the term part and common sense word part of the knowledge base have the contents indicated by the symbol 341.
The search items are “customer code” and “part code”. Next, for the selected search item, a file to be searched and a search item on the file are selected, and logic is generated (step A2403).

The search processing for the file and item to be searched is as follows.
Matching processing of all specified file items and search item names, conversion of search item names to synonyms and matching processing of all file items, extraction of item names using common sense words, many words extracted This is performed in three stages, ie, a process of selecting an item of the file to be held.

FIG. 113 (A) and FIG. 113 (B) are diagrams for explaining the flow of the search processing. In step A2701, a file containing the item name as a data item 1 is searched from the designated files 800 to 800C. If so, that file and item are what you want. If not, step A
At 2702, the item names are converted into synonyms using the knowledge base 340. In step A2703, a file containing the converted synonym as a data item is searched. If not, in step A2704, common words are used and the ending word is extracted from the item name using common words. Step A270
In step 5, a file having the extracted word as a data item is searched for. If only one file is found, it is what you want. If not, assemble the logic using the file name and item as the "?" Mark. If two or more files are selected as candidates, a common word in the knowledge base is found from the item name in step A2706. In the next step A2707, the name of the data item in the file which has been selected as a candidate and which contains the most words cut out in the above step A2706 is found. If two or more candidates cannot be narrowed down yet, in step A2708, the extracted words are converted into synonyms, and the names of the candidate data items are checked again using these synonyms.
9 to narrow down. In this way, one search file and one search item are finally determined for one search item.

FIGS. 114 (A) and 114 (B) are operation examples of the above-described search processing. Here, the product code having the value indicated by the symbol 700 is the search item. A symbol 810 indicates a data item in the data files 800A and 800B. Symbols 341 and 342 indicate words registered as common sense words in the knowledge base and their synonyms. Step A
At 2801, the search item name "product code" is matched with the item of the file. Since there is no match, the synonym conversion of the "product code" is tried next, but in this case, since there is no "product code" in the knowledge database, matching with the synonym is not performed. Next, in step A2803, the ending word of the search item name is cut out using common sense words. In this example, a character string “code” is cut out. When the data item 810 of the file is searched for in step A2804 based on the above ending word, “article code” and “code” are selected as candidates. In step A2805, the “product code” is divided into “product” and “code” using common sense words. Since “code” has already been processed as an end word, in step A2806, a data item including “product” is searched from among the candidate data items. In this case, neither “article code” nor “code” includes “article”. Therefore, in step A2807, "product" is converted into a synonym "product" using the synonym 342 registered in the common sense word part. In step A2808, a candidate data item including the synonym “product” is selected from the candidate data items. As a result, the "article code" is selected. In this example, the search file name corresponding to the search item “product code” is determined to be “A”, and the item name in the file used for the search is determined to be “article code” by the above processing method.

The logic is assembled at step A2403 in FIG. 109 based on the determined search file name and data item name. FIG. 115 shows an example of the assembled logic. In the figure, step A2901 is a file open process, and step A2902 is a process of reading a corresponding record from a file based on data input by an operator. If there is no corresponding record in step A2903, an error message is output in step A2903, and the process waits for re-input in step A2904. If there is a corresponding record, logic is assembled to close the search file in step A2905. The code read from the file is stored in a work area on the memory.

In step A2302 of creating the calculation item processing procedure in FIG. 108, the calculation items are basically created in accordance with the execution processing rules 344 registered in advance in the knowledge base 340 corresponding to the terms and words except for the total items. If the rule is not described in the knowledge base 340, a logic for setting the corresponding item from among the records retrieved from the data file in the previous retrieval item processing and displaying it on the screen is incorporated. The calculation rules of the knowledge base 340 include a range rule 344A and an execution processing rule 344B as shown in FIG. The calculation items are 1: 1 matching between the name and the term or word registered in the knowledge database, 1: 1 matching between the synonym and the term or word, and extraction of the name based on the word in the common sense word part To search for the applicable rule.

The execution processing rules for knowledge data include date and time rules and operation rules using item names and properties. The date / time rule is a function processing rule for displaying a calendar or obtaining and displaying a time. In an operation rule using an item name, a term expression is entered using an item name in a file, so the item name described is searched for, and logic is assembled as described. The rule of the calculation using the property is that the calculation can be performed with the sum of the items in the slip, the form, and the file having the specified property as one. FIG. 116 shows an example in which the sum of all the items having the property of “unit price” is multiplied by the sum of the items having the property of “quantity”. The property 340 registered in the knowledge base indicates the characteristic of the term or word corresponding to the term or word. The above properties include, for example, eight types shown at 346 ', and an operator can be added or changed. An operation rule with this property is expressed by “Σ property”.

The range rule is a rule for checking the range of the input value of the item, and an example of the generated logic is shown in FIG. In the figure, step A3101
Is a check process of whether or not it is within the range, step A31
02 is a process for outputting an error message, step A3
Reference numeral 103 denotes a process of waiting for re-input.

The item of “total” which is one of the operation items is
Logic can be generated without an arithmetic expression in the knowledge database. "Sum" generally indicates what items are summed at that position. For example, FIG.
In the column indicated by the symbol a in the table shown in FIG. 7A, the sum of the amounts of money at the position above the total column may be calculated. Further, the sum indicated by the symbol a in the table shown in FIG. 2B is the sum of the amount of money above the same as in the above (A), and the total indicated by the symbol D is the sum of the tax amount of the symbol c above and the symbol You only need to take the total sum. In this way, the sum generates logic that takes the sum of the items located above and “the items having the properties of the sum”. When there is a total as one of the items of the specification as in the symbol o in the table shown in FIG. 9C, the logic is generated according to the execution processing rule. Further, when the total column is distant as indicated by the symbol F in the table shown in FIG. 4D, the item to be processed is searched for from the total item name, and logic for obtaining the sum of the items is generated. that's all,
The business program is generated from the recognition information by the method described above.

In the printing step A307, as shown in FIG. 119, the execution result of the program is printed on the designated sheet at the finger position using the position, width and length of the input / output field in the recognition information. it can. A slip indicated by the symbol EX100 is input and recognized, and recognition information indicated by the symbol A305 is generated. The screen indicated by the symbol A209 'is displayed to execute the program, and the execution result is stored in the data file A370 together with the item name. In printing, the recognition information that matches each item stored as the execution result is searched from the recognition information file A350, and the result data is printed at the area position. At this time, the size of the character font to be printed is obtained from the number of characters included in the area in the recognition information, and the result data can be printed in the size of the character. According to this method, the operator can print at a desired position with a desired character size without having to assemble a data stream for printing.

Next, a description will be given of a modification of the device configuration of the form processing system according to the present invention.

FIG. 120 is a block diagram showing a case where the system is configured using one data processing device 92.
The data processing device 92 includes a document structure recognition mechanism 921, a program generation mechanism 922, and an execution mechanism 923. An image input device 93 for inputting an image, an output device 95 for print output, and a display terminal device (console) 91 having a display 911 and a keyboard 912 for inputting or correcting data are connected to the data processing device 60. I have.

With such a configuration, the program can be developed and changed by one processing device.

FIG. 121 shows an example in which the form processing system according to the present invention is constituted by one data processing device 92 and a plurality of display terminal devices 91A and 91B.

The data processing device 92 includes a document structure recognition mechanism 921, a program generation mechanism 922, and an execution mechanism 92.
3 are included. The terminal devices 91A and 91B are connected to the data processing device 92 by data transmission paths 40A and 40B, respectively, and have a display and a keyboard for performing data input or correction. Further, the image input device 93 is connected to the terminal device 91A, and the input image data is transmitted to the data processing device 92 via the terminal device 91A and the data transmission path 40A. An output device 95 for performing print output is connected to the terminal device 91B, and output data from the data processing device 92 is output via the data transmission path 40B and the terminal device 91B.

By adopting such a configuration, it is possible to use one terminal device for development work and the other terminal device for executing or changing processing. In addition, by increasing the distance between the data transmission paths 40A and 40B, development, execution, and change at a remote location become possible.

FIG. 122 is an example in which the form processing system according to the present invention is composed of a plurality of data processing devices.

An image input device 93 is connected to the data processing device 92A to input an image.

The data processor 92B has a recognition mechanism 921.
And converts the image data from the data processing device 92A into code format data and attribute data.

The data processing device 92C includes a generation mechanism 922.
Are generated, and the generation mechanism 922 generates a processing procedure (program) based on the data in the code format and the attribute data.

The data processing device 92D includes an execution mechanism 923
And execute the processing procedure. Therefore, an input device 912D for inputting data and an output device 95 for printing out are connected.

The data processing device 92E includes a display 9
11E and a keyboard 912E.
21 output information is corrected.

The data processing device 92F is connected to the display 9
11E and a keyboard 912E.
The processing procedure output from 22 is corrected.

These data processing devices are connected by a transmission line 41 such as a local area network. For data transmission between the data processing devices, it is also possible to adopt a method in which output data is temporarily stored in a floppy disk medium to carry the data.

With this configuration, the load can be distributed. Further, it is possible to use a lower-cost substitute machine instead of each data processing device.
For example, a facsimile device may be used instead of the data processing device 92A, or an inexpensive personal computer may be used as the data processing device 92E.

[0197]

As described above, according to the present invention, the format information (ruled line, arc, oblique line, Dashed lines, characters, field positions, field attributes, etc.)
Since it can be automatically defined, there is an effect that a new data input format and a form issuing process can be easily performed. Also, since format information input as an image can be automatically converted into graphic code information and character code information, this format can be displayed or printed with high image quality.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram showing an overall configuration of a form processing system according to the present invention.

FIG. 2 is a flowchart showing a basic operation of a form processing program executed by the system shown in FIG. 1;

FIG. 3 is a diagram showing an example of a form input as an image

FIG. 4 is a diagram showing an example of a form output from the system of the present invention.

FIG. 5 is a diagram showing an example of a practical flowchart of a form processing program.

FIG. 6 is a flowchart of a subroutine executed in an image input step 1;

FIG. 7 is a diagram for explaining a relationship between an input image and an input image memory;

FIG. 8 is a flowchart showing a first embodiment of a subroutine executed in a form format recognition step 2;

FIG. 9 is a flowchart showing details of a subroutine executed in a physical structure recognition step 21 of FIG. 8;

FIG. 10 is a flowchart showing details of a block dividing step 211 in FIG. 9;

FIG. 11 is a table T1 used in a block dividing step;
Showing the configuration of T2 and T2

FIG. 12 is a diagram showing an example of an image for specifically explaining block division;

FIG. 13 is a flowchart showing details of an area dividing step 214 in FIG. 9;

FIG. 14 is a diagram for specifically explaining an execution process of a physical structure recognition step 21;

FIG. 15 is a flowchart showing details of an area type recognition step 216 in FIG. 9;

16 is a flowchart showing details of a left side recognition step 216-1 in FIG.

FIG. 17 is a diagram showing a configuration of a table T3 used in a left side recognition step 216-1.

FIG. 18 is a diagram showing a specific example of an original image processed in the left side recognition step and a recognition result.

FIG. 19 is a diagram showing a configuration of a table TBL1 used to store a recognition result of a line segment;

FIG. 20 is a diagram showing a typical pattern of a non-column element.

FIG. 21 is an explanatory diagram of a process for detecting an arc pattern from an input image.

FIG. 22 is a table TBL2 for storing oblique line information.
Configuration diagram

FIG. 23 is a table TBL3 for storing arc information.
Configuration diagram

FIG. 24 is a flowchart showing details of a component recognition step 213 in FIG. 9;

FIG. 25 is a diagram for explaining the type determination of a component;

FIG. 26 is a flowchart showing details of a straight line recognition step 2181 in FIG. 24;

FIG. 27 is an explanatory diagram related to the condition of a component indicated by a broken line.

FIG. 28 is a flowchart showing details of a broken line recognition step 2182 in FIG. 24;

FIG. 29 is a configuration diagram of a broken line table TBL4.

FIG. 30 is a flowchart showing details of a character recognition step 2183 in FIG. 24;

FIG. 31 is an explanatory diagram relating to conditions of components of a character string.

FIG. 32 is a view for explaining a processing procedure of a character recognition step;

FIG. 33 is a table TB for storing character recognition results;
Configuration diagram of L5

FIG. 34 is a flowchart showing another embodiment of the form format recognition step 2;

FIG. 35 is a flowchart showing details of preprocessing step 20 in FIG. 34;

FIG. 36 is a diagram showing an example of an input image processed in a preprocessing step

FIG. 37 is a flowchart showing details of post-processing step 29 in FIG. 34;

FIG. 38 is a flowchart showing details of a line correction processing step 291 in FIG. 37;

FIG. 39 shows a character correction processing step 292 in FIG. 37.
Flow chart showing details of

FIG. 40 is a table TBL6 for storing word information.
Configuration diagram

FIG. 41 is a flowchart showing another embodiment of the form format recognition step 2;

FIG. 42 is a flowchart showing a first embodiment of a logical structure recognition step 22 in FIG. 41;

FIG. 43 is a flowchart showing details of a row / column normalization step in FIG. 42;

FIG. 44 is a table TBL used in a row / column normalization step;
Configuration diagram of 7

FIG. 45 is a view for explaining reference points of a pattern such as a character;

FIG. 46 is a flowchart showing another embodiment of the logical structure recognition step.

FIG. 47: Field position recognition step 2 in FIG. 46
Flowchart showing details of 22

FIG. 48 is an exemplary view for explaining tabular normalization processing performed to facilitate field position recognition;

FIG. 49 is a configuration diagram of a field table TBL8.

FIG. 50 is a view for explaining an example of an execution result of field position recognition;

FIG. 51 is a flowchart showing another embodiment of the logical structure recognition step 22 in FIG. 41;

FIG. 52: Field relation recognition step 2 in FIG. 51
Flowchart showing details of 23

FIG. 53 is a table T4 for storing unit information.
Configuration diagram

FIG. 54 is a view showing an example of a field relationship recognition result;

FIG. 55: Field attribute recognition step 2 in FIG. 51
24 is a flowchart showing the details of

FIG. 56 is a view showing the configuration of KNW1 in various tables referred to in the name matching step 2242 in FIG. 55;

FIG. 57 is a configuration diagram of KNW2 in various tables referred to in the name matching step 2242 of FIG. 55;

FIG. 58 is a view showing the configuration of KNW3 in various tables referred to in the name matching step 2242 in FIG. 55;

FIG. 59 is a configuration diagram of KNW4 in various tables referred to in the name matching step 2242 in FIG. 55;

FIG. 60 is a configuration diagram of various attribute information setting tables TBL9 referred to in a field attribute recognition step 224.

FIG. 61 is a configuration diagram of various attribute information setting tables TBL10 referred to in a field attribute recognition step 224.

FIG. 62 is a configuration diagram of various attribute information setting tables TBL11 referred to in a field attribute recognition step 224.

FIG. 63 is a configuration diagram of various attribute information setting tables TBL12 referred to in a field attribute recognition step 224.

FIG. 64 is a configuration diagram of various attribute information setting tables TBL13 referred to in a field attribute recognition step 224.

FIG. 65 is a configuration diagram of various attribute information setting tables TBL14 referred to in a field attribute recognition step 224.

FIG. 66 is a diagram showing a specific example of the contents of tables TBL9 to TBL14 formed by form logical structure recognition;

FIG. 67 is a flowchart showing a third embodiment of the form format recognition step 2;

68 is a flowchart showing details of a form creation program generation step 23 in FIG. 67.

FIG. 69 is a view showing an example of a program generated in a generation step 23;

70 is a flowchart showing details of an input / output field processing generation step 235 in FIG. 68.

FIG. 71 is a flowchart showing one embodiment of a form creation step 3 in FIG. 2;

72 is a print format data generation step 3 in FIG. 71.
9 is a flowchart showing details of

FIG. 73 is a detailed diagram of generation of print format data in a bitmap format.

FIG. 74 is an explanatory diagram of generation of print format data in a command sequence format.

FIG. 75 is a flowchart showing another embodiment of form creation step 3;

FIG. 76 is a view showing an example of a form printed and output;

FIG. 77 is a view showing an example of a form printed and output;

FIG. 78 is a flowchart showing still another embodiment of the form creation step 3;

FIG. 79 is a flowchart showing an embodiment of a form content data creation step in FIG. 78;

FIG. 80 is a diagram showing an example of a form serving as a manuscript

FIG. 81 is a diagram showing an example of an issue form to which contents have been added;

FIG. 82 is a flowchart showing a second embodiment of the form content data creation step 33;

FIG. 83 is a view showing an example of a file searched in step 334 of FIG. 82;

FIG. 84 is a flowchart showing a third embodiment of the form content data creation step 33;

FIG. 85 is a flowchart showing a fourth embodiment of the form content data creation step 33;

FIG. 86 is a flowchart of a form and a screen for explaining an example of the operation of the system of the present invention.

FIG. 87 is a data flow diagram corresponding to the operation in FIG. 86;

FIG. 88 is a flowchart showing the flow of recognition processing and a diagram showing an example of a form of a slip handled in the recognition processing.

FIG. 89 is a flowchart showing the flow of a cell recognition process;

FIG. 90 is a diagram showing an example of a line segment to be corrected;

FIG. 91 is a schematic view showing an intermediate state of the area division processing;

FIG. 92 is a diagram showing an example of a character string to be recognized;

FIG. 93 is an explanatory diagram for explaining a process of extracting a meaningful character string from characters in the same matrix;

FIG. 94 is an explanatory diagram for describing a process of cutting out a character string when a plurality of matrices have one meaning

FIG. 95 is an exemplary view showing an example of a positional relationship between a cell and a character string.

FIG. 96 is a flowchart showing input / output field setting processing;

FIG. 97 is an explanatory diagram for describing a process of linking a heading cell to an input / output field;

FIG. 98 is an exemplary view showing an example of setting fields in a cell with a heading.

FIG. 99 is an explanatory diagram for describing field setting processing based on the meaning of a character string;

FIG. 100 is an exemplary view of a field set according to the meaning of a character string;

FIG. 101 is an explanatory diagram for explaining a linking process between a heading character and a field cell;

FIG. 102 is a diagram showing an example of a field created by a character string not surrounded by cells

FIG. 103 is an exemplary view showing contents of recognition information.

FIG. 104 is an explanatory diagram for explaining how to obtain the number of characters;

FIG. 105 is an explanatory view showing how to assign field names in the case of a matrix type.

FIG. 106 is a diagram showing an example of the contents of a knowledge base.

FIG. 107 is a view for explaining a matching process with a knowledge base.

FIG. 108 is a flowchart showing processing program creation processing;

FIG. 109 is a flowchart showing a file search item processing procedure creation process;

FIG. 110 shows an example of a file search item and a processing item.

FIG. 111 is a flowchart showing the flow of a search item selection process;

FIG. 112 is an explanatory diagram for explaining search item selection processing;

FIG. 113 is a flowchart showing the flow of a search process for a file to be searched and an item to be a search item;

FIG. 114 is an explanatory diagram for explaining search processing of a search item and a flowchart showing the flow of search processing for a search item;

FIG. 115 is a diagram showing an example of a flowchart of a logic assembly result based on the determined search file and item.

FIG. 116 is an explanatory diagram for describing the description of a knowledge-based calculation rule;

FIG. 117 is a view showing an example of a flowchart showing a result of the range check logic assembling process;

FIG. 118 is a view for explaining how to calculate “total”;

FIG. 119 is an exemplary view for explaining the flow of print processing;

FIG. 120 is a diagram showing a configuration example of a modification of the form processing system according to the present invention;

FIG. 121 is a diagram showing a configuration example of a modification of the form processing system according to the present invention.

FIG. 122 is a diagram showing a configuration example of a modification of the form processing system according to the present invention;

[Explanation of symbols]

91 console, 92 processor, 93 image input device, 95 printer, 1 image input program, 2
Form format recognition program, 3 ... form creation program, 4
… Form output program.

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] October 18, 1999 (1999.10.
18)

[Procedure amendment 1]

[Document name to be amended] Drawing

[Correction target item name] All figures

[Correction method] Change

[Correction contents]

FIG. 2

FIG. 3

FIG. 4

FIG. 8

FIG.

FIG. 6

FIG. 7

FIG. 11

FIG.

FIG.

FIG.

FIG. 5

FIG. 14

FIG.

FIG.

FIG.

FIG. 41

FIG. 59

FIG. 9

FIG.

FIG. 27

FIG. 28

FIG. 29

FIG. 10

FIG. 21

FIG. 23

FIG. 24

FIG. 25

FIG. 26

FIG. 13

FIG. 31

FIG. 32

FIG. 33

FIG. 34

FIG. 35

FIG. 36

FIG. 78

FIG. 16

FIG. 37

FIG. 38

FIG. 39

FIG. 40

FIG. 42

FIG. 44

FIG. 45

FIG.

FIG. 46

FIG. 48

FIG. 49

FIG. 50

FIG. 51

FIG. 53

FIG. 56

FIG. 57

FIG. 65

FIG. 80

FIG. 43

FIG. 58

FIG. 60

FIG. 61

FIG. 62

FIG. 63

FIG. 67

FIG. 47

FIG. 54

FIG. 64

FIG. 71

FIG. 73

FIG. 52

FIG. 74

FIG. 75

FIG. 76

FIG. 79

FIG. 81

FIG. 55

FIG. 66

FIG. 70

FIG. 77

FIG. 83

FIG. 89

FIG. 68

FIG. 72

FIG. 90

FIG. 69

FIG. 92

FIG. 95

FIG. 96

FIG. 100

FIG. 82

FIG. 85

FIG. 86

FIG. 99

FIG. 102

FIG. 108

FIG. 84

FIG. 87

FIG. 91

FIG. 93

FIG. 97

FIG. 88

FIG. 98

FIG. 101

FIG. 105

FIG. 120

FIG. 94

FIG. 103

FIG. 104

FIG. 106

FIG. 107

FIG. 109

FIG. 110

FIG. 112

FIG. 116

FIG. 111

FIG. 113

FIG. 114

FIG. 115

FIG. 117

FIG. 118

FIG. 119

FIG. 121

FIG. 122

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Jin Suzuki 1 Ikegami, Haruoka-cho, Owariasahi-shi, Aichi Prefecture Inside the Asahi Plant of Hitachi, Ltd. (72) Inventor Toshihiko Matsuda 1 in Ikegami, Haruoka-cho, Owariasahi-shi, Aichi Prefecture In-house Hitachi, Ltd. (72) Inventor Hiroshi Fujise 5030 Totsuka-cho, Totsuka-ku, Yokohama-shi, Kanagawa Japan Stock Company (72) Inventor Keisho Kuno 890 Kashimada, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture 12 Inside Hitachi Information Systems Factory (72) Inventor Ichitaro Koai 4-6-6-1 Kanda Surugadai, Chiyoda-ku, Tokyo Hitachi, Ltd.

Claims (2)

[Claims] 1. A form recognition method comprising: inputting an image including a form described in an original document; and executing a program corresponding to the type of the form in the input image. 2. A form recognition apparatus comprising: input means for inputting an image including a form described in an original document; and control means for executing a program corresponding to the type of form in the input image. .