JPS61193281A - Document input system - Google Patents

Abstract

PURPOSE:To enable decision automatically of the format information of document subjected to identification by understanding the construction of the image from the remaining characters and frame after identification of characters existing within the frame from an image of a document to be identified and eliminating all characters except that are to be entered in the specimen document. CONSTITUTION:Read out construction data and physical data of the specimen document in sequence to the memory 53 and compare with the construction data and physical data of the input document of memory 54; thence determining which specimen document the input document coincides with. Edit the character recognition result and the necessary parts in the construction data and write out in the output device 9. As an example of editing, e.g. if the item title of the frame belonging to the data frame is 'applicant', and the result of recognition is 'YAMADA TARO', 'applicant'='YAMADA TARO', which is written as record 1.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Application of the Invention] The present invention relates to a document processing method, and particularly to a document input method suitable for reading characters on a tabular document.

[Background of the invention]

In conventional document reading devices (hereinafter abbreviated as OCR), it is necessary to print the reading area using colors (dropout colors) that OCR does not sense, which has the problem of increasing printing costs. Furthermore, it is necessary to describe the position of the character reading area as the distance from the edge of the document, and specify the number of characters in that area, character type, checking method, etc.
It was complicated. Hereinafter, information such as the position information of the character reading area, the number of characters in the area, character type, and checking method is referred to as format information.The character recognition function is not only used for dedicated OCR as in the past, but also for document files and OA workstations. However, the above-mentioned problems have prevented its widespread use.

In the prior art, for example, published patent publication 1982-207
No. 184 (published on December 2, 1984), a method for removing a fixed image stored in memory in advance from an input image, and a method for identifying the type of document using the fixed image. is disclosed. However, since this method stores the memory image, the amount of memory increases, and there is also a risk that if the document is deformed (stretched/contracted, rotated, misaligned, etc.), it may not be removed correctly.

[Purpose of the invention]

The purpose of the present invention is to automatically understand the structure of an input document and automatically generate a document format.

The object of the present invention is to provide a means for greatly expanding the range of target documents and simplifying the effort required to create format information.

Another object of the present invention is to provide a means for automatically identifying the type of a formatted input document and using pre-stored format information to enable continuous reading of formatted documents of different types. There is a particular thing.

[Summary of the invention]

In order to achieve this purpose, the present invention uses a document to be identified, extracts frame parts and relationships between the frame parts from an image of the document to be identified, and further identifies the characters existing inside the frame. Delete characters other than those that should be written in a blank document (hereinafter referred to as a sample document) of the same type as the document to be identified, and understand the structure of the image from the remaining characters and the frame. By doing so, the format information of the document to be identified is automatically determined.

[Embodiments of the invention]

Hereinafter, the present invention will be explained in detail using the drawings.

FIG. 1 is a block diagram showing the configuration of an apparatus that implements the document understanding method of the present invention. Each part of the device is connected to a bus 1, and the overall operation is controlled by a controller 2. Information on the document 3 (document image) is scanned by the photoelectric conversion device 4, further digitized, and stored in the memory 51 via the bus 1. The memory 51 forms a part of the memory 5 along with 52, 53, and 54, which will be described later.When digitizing, known high-efficiency encoding processing may be performed, thereby saving the storage capacity of the memory for storing document images. can.

Further, in the following explanation, it is assumed that each pixel is binarized into one bit, but one pixel may be expressed in multi-values, or color information may be added by photoelectric conversion using a color scanner.

Document processing includes a registration mode and an identification mode. Mode selection is performed through man-machine interaction with the control section 2 using the keyboard 6 and display 7.

First, the case of registration mode will be explained. A flowchart of the registration mode process is shown in FIG. 2, and this process is executed by the software of the control unit 2. In FIG. 2, 201 is an input of a document image to be identified, and the image on this document is stored in a memory 51.
Store it inside. At 202, normalization such as position correction processing and tilt correction processing is performed on the input document image, and the resulting image is stored in the memory 52. Line extraction processing is performed on this normalized image in step 203, and the extracted line pattern is stored in the memory 53. FIG. 3 is a diagram illustrating the processing contents of line pattern extraction.

By extracting horizontal lines from an input image like (A), (
Obtain a horizontal line pattern such as B), obtain a vertical line pattern such as (C) by vertically extracting lines from (A), and obtain a vertical line pattern such as (C) by logically adding (B) and (C). get an image like this. Hereinafter, (D) will be referred to as a line pattern, and details of the line pattern extraction process will be described later. In step 204, coordinate point sequences on the contour are extracted from the line pattern by contour extraction, and as many coordinate point sequences as there are contours are obtained. In the following, only the inner contours among the contours will be processed. Here, the inner contour is the boundary of the black background surrounding the white background, and refers to the contour that goes around the contour clockwise with the white background on the right and the black background on the left.In 205, inside this inner contour Extract rectangles from. Details of rectangle determination will be described later. Hereinafter, the inner contour of the rectangle will be referred to as a frame. At step 206, the coordinates of the four corners of the frame are used to rearrange the frames from top left to bottom right.

In 207, when there are a plurality of extracted frames, the mutual relationship between the frames, that is, the relationship code between frame m and frame n Cm n
Extract. An example of the definition of the relational code Cm n is shown below.

Cmn=0: All of the following conditions do not hold = 1: n includes m = -1: m includes n = 22 m is directly above n = -2: m is directly below n = 3: m is n Direct pressure = -3: m is directly to the right of n ~ = 4: m is above n = -4: m is below n = 5: m is to the left of n = -5: m is to the right of n At step 208, images of areas corresponding to the extracted frames are extracted from the original image, and these images are sequentially sent to the character recognition unit 8 to recognize the characters within the frames. Since only the image inside the frame is sent to the character recognition unit, there is no interference from non-dropout color display parts or unnecessary parts outside the frame, and since the target is a band-shaped area, it is easy to cut out and recognize characters. After recognizing the characters in all frames, display the recognition results for each frame 7
to be displayed. When it is determined that the displayed characters do not represent the structure of the target document, the character string is erased using the keyboard 6. For example, in FIG. 3(A), Hitachi Abe, Tokyo, 30, 1.1, etc. are deleted. Also, when changing the contents in the character string representing the structure, 6, 7
Use the six or more corrected data that is changed using as the data of the sample form, and continue the following process. Further, as a means for erasing character strings other than character strings representing the structure of the target document, it is possible to automatically erase character strings other than the above-mentioned keyboard method.

For example, after understanding the structure of the input document described below, it can be automatically deleted by deleting character strings within a frame determined to be a data frame based on the result of the understanding. 209
Then, in the frame m, the number of recognized characters Nm, the character string Km, and the relational code Cmn are compared with knowledge regarding the structure of the document to understand the structure of the input document. Knowledge is r i f = then
...'' format, that is, a condition and a conclusion. Examples of knowledge are: a, b, and Q indicate frame numbers.

1゜ (Condition) Ka = 'Applicant' (Conclusion) Frame a is an item frame Item name of frame a = 'Applicant' 2゜ (Condition) Na = 0 & Kb = 'Applicant'& Ca b = -3 (Conclusion) Frame a is the attribute of data frame bar a that is subordinate to frame =
゛Name (Na = O indicates a blank) 3. (Condition) Na = O & Kb = 'Prefecture'& Ca b = -4 Kc = 'Registered address'& Cac = -3 (Conclusion) Frame a is , Attribute of data frame bar a belonging to c = ゛Address: Prefecture' 4, (Condition) Ka = 'Showa year Showa year month sun wheel) Frame a is an independent data frame Unit name of frame a = ``Date and time frame a Attribute = “Time 2 years, month, day” 5, (condition) Na = 0 (conclusion) Frame a is an independent item ・Item name of data frame bar a = ``Miscellaneous'' Attribute of frame a = ``Miscellaneous'' Here, item frame is a box that is used as a heading and is not normally filled with data. A data frame is a frame in which characters (character strings) representing the contents of an item are to be written, and an attribute is a type of character to be written in the frame, a limited range, etc. The specific method for realizing the above knowledge is the r i f of the program.
=then~” is a subroutine written as a statement,
Inquiry with knowledge is a.

Substitute m and n for b, check whether the condition holds, and give the result to the outside as an argument. 0 If the condition holds, the conclusion is also given to the outside as an argument. Knowledge for example 1. If frame numbers m and n that apply to conditions a and b are found, frame number m will be added to frame number a in the conclusion.
The substituted value is registered as the structure data of the input document. The structural data consists of the above-mentioned frame relationship codes, frame types (item frame, data frame, or both), dependent relationships, item names, attribute codes, and the like. If the knowledge applies to multiple pieces of knowledge, for example, it may be decided to give priority to the knowledge that appears earlier. By determining the remedy (5 in the above case) when the knowledge does not apply to other knowledge as the knowledge with the lowest priority, the structure for all frames will be determined. When all the frames are compared with knowledge and the structures are determined, the registration of the structural data is completed, and in step 210, the structural data, physical data of the frames (coordinates of the four corners of the frame, etc.), and mask data are output to the device. Write it down at 9. The mask data indicates the position of the caulked characters (corresponding to the unit name such as the year, month, and day of Showa in 4. above) within the data frame. Structural data.

The frame physical data and mask data correspond to conventional OCR format data. When writing, the keyboard 6 and display 7 are used to input the identification number and file name for the input document.

Next, details of the line pattern extraction process will be explained. Figure 4 shows
2 is a flowchart showing a process for extracting a line pattern in PAD format. Line pattern extraction includes horizontal line extraction and vertical line extraction, but horizontal line extraction will be described here, and vertical line extraction will be omitted since it is the same. In FIG. 4, 401 is an entrance to line extraction, and the normalized image Q stored in the memory 52 is given. Q is expressed as two-dimensional data Q(itj)t (i=o to I-1, j=o-J-1) as shown in FIG. 3(A). In step 402, the processes in steps 403 to 409 are repeated for scanning line number j.

A two-dimensional pattern A(i*j) as shown in FIG. 3(B) is obtained. 403 is initialization, in which A(i, j) is cleared to 0 and B, which will be described later, is set to 0. 404 is a loop that repeats only the number of sensing elements in the scanning line. 405 is Q(x,j)
Determines whether is 1 or O, and if it is 1, it is 406 and black slow flame B
- If Q(l v j) is O, then
As a result of the judgment in step 407, the slow flame B up to the previous pixel is the threshold value C.
When larger, 8 of A(i Bo j) to A(i l, j) in 408
Processing is performed to set the pixels to 1 (black), and at 409, the slow flame B
Reset. Processes 410 to 411 are similar to processes 407 and subsequent steps at the final point (i=I-1) of the scanning line. By adding the judgment in 407, line extraction is performed only when there is a sufficiently long black horizontal line segment, but since the characters and symbols written on the document are made up of short line segments, they are almost never extracted. It is the exit of the process, gives the pattern A(z, j) to the output, and stores it in the memory 53. As can be seen from the explanation of 0 or more, the pattern A(itj) is a pattern that reflects the existence of a line segment.

Next, details of the rectangle extraction process will be explained. FIG. 5 is a diagram showing the details of the process of extracting a rectangle.

The coordinates of the inner contour are the maximum and minimum values of (X(i) + Y(i)), (X(i) - If we find the points that give the maximum and minimum values of Y(i)), these points can be found in P4. P2
．． P3. Corresponds to PL. P1 to P4 are the four corner points when the frame is a rectangle.

Find the points that give the maximum and minimum values of X between P1 and P2 and calculate this in Q12. Let it be Qll. Find the point that gives the maximum and minimum values of Y between P2 and P3 and calculate this in Q22. Let's call it Q21. Do the same for the other sides. PI, Qll, Q
12. When the difference in the X coordinate values of the four points of P2 is small, P1~
P2 is determined to be a straight line, and the other three sides are similarly determined to be straight lines, and when the four sides are straight lines, this frame is determined to be a rectangle.

Next, details of the relational code extraction process will be explained. Figure 6 shows
It is a flowchart which shows the process of extracting a relational code in PAD format. In FIG. 6, 601 is an entrance, and the coordinates of the four corners of the frame and the outline length R stored in the memory 52 are given. The four corners of frame m are (X 1 (m), Y 1 (m))
~(X 4 (m), Y 4 (m)), 60
2,603 is a double loop regarding frame numbers m and n.

In step 604, the initial value O is set in Cm_n. The condition 605 detects that frame m includes n, and when the condition is satisfied, 1 is assigned to Cmn and -1 is assigned to Cnm. The related codes are determined in the same manner below. 606 provides a matrix C consisting of relational codes at the output.

This concludes the explanation of the registration mode. Although this registration mode processing is performed in advance to store format information such as structural data, it is also possible to perform it in combination with the identification mode processing described below.

Next, processing of an input target document in the identification mode will be explained. FIG. 7 shows a flowchart of identification mode processing.
701 is importing an image of an input document.

702 is normalization, 703 is line pattern extraction, 704 is contour extraction, 705 is rectangle extraction, 706 is frame ordering,
707 is a relational code extraction.

This is similar to 202 and 207 in FIG. In step 708, the structural data and physical data of the sample document are sequentially read into the memory 53 and compared with the structural data and physical data of the input document in the memory 54 to determine which sample document the input document matches. Using the structural data and physical data of this sample document, the input document is read as follows. 7
09 is a loop that repeatedly executes the following steps 710 to 717 for the frame. 710 is a determination to read only the data frame. In step 711, only the area inside this data frame is extracted. At step 712, characters previously printed using mask data are erased (this is done only when mask data exists).

At 713, a character type determined by the attribute code is selected.

Set the recognition dictionary (standard pattern for character recognition). In step 714, characters inside the frame are recognized.

At 715, a word dictionary determined by the attribute code is selected,
In step 716, word matching is performed to correct misreading or misreading of the character recognition results. In step 717, the character recognition result and the necessary portions of the structural data are edited and written to the output device 9. As an example of editing, suppose that the item name of the item frame to which the data frame belongs is "Applicant" (corresponding to 2. of the above knowledge), and the recognition result of this frame is 1 Yamada Abe'.

``Applicant'' = ``Yamada Abe'' and write it out as a code.

Also, in the independent item frame/data frame, item name = 'date date and time,
Since the unit name = ``Showa date and time'' (corresponding to knowledge 4゜) and the unit name part was erased by mask data, if the recognition result was 58529, the editing output would be ``Date and time'' = 58, 1982. Let it be the 29th day of the month. In the latter example, the position coordinates of the character are taken out from the recognition unit 8, and the position is determined and edited by comparing the coordinates with the mask data.

One embodiment of the present invention has been described above. It is easy to imagine that processing performed by a normal character recognition device or graphic processing device, such as displaying and modifying the frame extraction results and displaying and modifying the character recognition results, can be added to this embodiment.

In addition, we have set multiple thresholds for line extraction and rectangle determination during frame recognition, and when automatic judgment results are difficult, we can manually select the optimal value or change the binarization level and try again. Furthermore, if the type of document is determined in advance in the identification mode, the number of the sample document may be given from the keyboard, and the verification of structural data etc. may be omitted.

Note that in this example, a line pattern is extracted from a document and a frame is extracted from it, but it is also possible to directly extract an outline from the original document pattern and then extract a frame. Other contours can be used later for other processing, such as character segmentation and character recognition.

Further, in this embodiment, the description of the document structure is limited to a frame format, but it may be expanded to use ruled lines such as solid lines or broken lines, 1 yen, etc. For example, a solid ruled line can be detected by extracting an elongated outer contour, and an underlined title character can be recognized by cutting out the area above the ruled line. A circle can be detected by displaying the contour coordinate series in polar coordinates and detecting a straight line in this space.

It is also possible to read characters outside the frame and use them for structural description. It is also effective to add color attributes to the structural description using color information.

Furthermore, the present invention is not limited to direct photoelectric conversion of documents, but can also be applied to document images read from image files.

〔Effect of the invention〕

As explained above, according to the present invention, even when there is no sample document, it is possible to automatically understand the structure of an input document and automatically generate a document format.

There is no need for humans to create formatting information as in the past, and since tabular documents printed in black can be read, there is no need to reprint them in dropout color, simplifying the preparation work and making it easy to The scope of the document can be expanded significantly.

Further, according to the present invention, the type of a fixed-form input document can be automatically identified and format information stored in advance can be used, making it possible to continuously read different types of fixed-form documents.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of an apparatus that implements the document processing method of the present invention, FIGS. 2.4 and 6.7 are flowcharts for explaining the processing in the control section of FIG. The figure is a diagram illustrating the processing contents in FIG. 2. 1...Bus, 2...Control unit, 3...Document, 5...
- Memory, 6...Keyboard, 7...Display, 8...Character recognition unit, 9...Output device.

Claims (1)

[Claims] 1. means for inputting a document image; means for extracting a frame portion from the document image; means for identifying characters existing inside the frame portion; means for making corrections such as deletion, insertion, replacement, etc.;
A document input method comprising means for understanding the structure of an image from the frame portion and the modified characters.