JP3586949B2 - Form recognition device - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention recognizes a frame ruled line structure in a document image including a frame ruled line and characters such as a form, identifies a form, cuts out a specific character area written in the form, and automatically recognizes characters. The present invention relates to a form recognition device.
[0002]
[Prior art]
In recent years, with the digitization of document information, there has been an increasing demand for a character recognition technology such as an OCR (Optical Character Reader) and a document image processing, and a technology for automatically reading a tabular document such as a form is one of them. In particular, when processing a form in which multiple formats are mixed, it is necessary to classify the form manually by type and then have it read by an OCR device. To reduce time and effort, both form identification and character reading are automated. It is requested to do.
[0003]
As a form identification technique, for example, there is a method of identifying a form type based on a difference in a local ruled line structure and reading an area designated in advance, as disclosed in Japanese Patent Application Laid-Open No. 2-217977. , FIG. 21 and FIG. For example, as shown in FIG. 19A, six line segments k1 to k6 are detected as shown in FIG. You. On the other hand, when line segments are similarly extracted from the form 320 shown in FIG. 20A, seven line segments m1 to m7 are detected as shown in FIG. Therefore, it is possible to set a line segment detection area in which a plurality of forms can be identified in advance, and to detect and compare the number of line segments in the area, the mutual positional relationship, and the length, thereby enabling the forms 310 and 320 to be identified. The specified character reading area, that is, the character areas 311 and 312 in the form 310 of FIG. 19A, and the character areas 321 and 322 in the form 320 of FIG. To achieve reading.
[0004]
Various methods have been proposed for recognizing cut-out characters. For example, a character recognition method using a neural network (Mori: "Handwritten Kanji Recognition by PDP Model", IEICE Transactions, Vol. J73- D-II, No. 8 pp. 1268-1274 1990), and the recognition rate has reached a practical level. However, this time, since it is a form frame line recognition device, it is limited to preprocessing of character recognition. Therefore, the character recognition is omitted.
[0005]
[Problems to be solved by the invention]
However, in the above-described conventional configuration, if a form 330 as shown in FIG. 22A is mixed to detect a local difference in the frame ruled line structure of the form, line detection is performed as shown in FIG. As shown, seven line segments n1 to n7 are detected, which is the same as the case shown in FIG. 20B, and it is difficult to specify the character reading areas 331 and 332 in FIG. It is necessary to visually confirm and set the range of the detection region in advance, and there is a problem that the operation is complicated.
[0006]
The present invention solves the above-mentioned problems of the prior art, and provides a highly reliable form recognition device that identifies forms even when forms in various formats are mixed, and accurately detects a character reading area for each form. The purpose is to provide.
[0007]
[Means for Solving the Problems]
To solve this problem, the present inventionFrom the binary image,A corner detecting means for detecting a corner,The components of the frame rule are determined from the combination of the shapes of the corners of the detected frame rule.Component detecting means for detecting, and connecting the components,Rectangle detecting means for outputting as frame structure information, andThe mostDetect the outer frame of the outer shellOutsideOuter frame detection means for extracting a feature of the structure of the frame as an outer frame structure feature;For each extracted outline frameFeature extraction means for extracting internal structural features as internal structural features;The outline frame and internal structure features are registered in advance as the format of one or more forms.A frame structure reference table;A frame structure that specifies the form type by searching the detected outer frame structure feature and the frame structure reference table to determine a form candidate, and comparing the determined candidate with the detected internal structure feature. Collation meansAre provided.
Further, an image memory for storing a binary image including a frame ruled line, and a character cutout unit for cutting out a character area from the image memory based on the coordinates of the character reading target area from the detected outer frame structure feature and internal structure feature, And character recognition means for recognizing the cut-out character.
[0008]
As a result, even when forms in various formats are mixed, the forms can be identified, the character reading area of each form can be accurately detected, and a highly reliable form recognition device can be realized.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
The invention according to claim 1 of the present inventionA corner detecting means for detecting a corner of the frame ruled line from the binary image; a component detecting means for detecting a component of the frame ruled line from a combination of the detected corner shapes of the frame ruled line; A rectangle detection unit that outputs as frame structure information, an outline frame detection unit that detects an outermost outline frame based on the frame structure information, and extracts a feature of the outline frame structure as an outline frame structure feature,For each extracted outline frameFeature extraction means for extracting internal structural features as internal structural features; a frame structure reference table in which an external frame and internal structural features are registered in advance as one or more forms of the form; Frame structure matching means for searching a frame structure reference table to determine a form candidate, and identifying the form type by comparing the determined candidate with the detected internal structure featureAnd a form ruler that determines the format category of the form based on the structure of the outline frame in the form, and further, for each outline frame, becomes a sample frame rule line structure registered in a reference table in advance. Because of the collation, the form type can be specified accurately.WhenIt has the effect.
[0010]
The invention described in claim 2 isFurther, the image processing apparatus further includes a ruled line extracting means for extracting a frame ruled line by detecting runs having a predetermined length or more in the horizontal and vertical directions from the binary image including the frame ruled line and the character. The feature is that the character is erased, the corner can be detected only from the frame ruled line, and the component can be detected with high accuracy.
[0011]
The invention according to claim 3 is:The constituent element detecting means detects the constituent elements of the L-shaped element, the T-shaped element, and the cross element as the constituent elements by bending or intersecting the ruled line from the combination of the corner shapes. Since the component can be stably detected even from the form read by tilting by detecting, the type of the form can be specified accurately, and the character reading area for each form can be accurately detected.
The invention according to claim 4 is characterized in that the outer frame detecting means connects the vertex coordinates and extracts each of the independent outer frames as an outer frame structural feature, and connects the vertex coordinates of the outer frames. By doing so, the outer frame structure can be easily extracted.
The invention according to claim 5 is characterized in that the outline frame structure feature is to extract the total number, position, and shape of the outline frames in the target form, and to easily connect the vertex coordinates of the outline frames. This has the effect that the outer frame structure can be extracted.
The invention according to claim 6 is characterized in that the feature extracting means extracts the number, position, width and height of internal frames as internal structural features for each external frame. There is an effect that the type of the form can be accurately specified using the internal structure of each outer frame.
The invention according to claim 7, further comprising: an image memory for storing a binary image including a frame ruled line; and the image memory based on the coordinates of the character reading target area based on the detected external frame structural features and internal structural features. It is characterized by having a character extracting means for extracting a character area and a character recognizing means for recognizing the extracted character, so that the type of the form can be specified accurately and the character reading area for each form can be accurately detected. Has an action.
The invention according to claim 8, wherein a step of detecting a corner of the frame ruled line from the binary image, a step of detecting a component of the frame ruled line from a combination of the shapes of the corners of the detected frame ruled line, Connecting each other and outputting as frame structure information; detecting the outermost outer frame based on the frame structure information; and extracting the features of the outer frame structure as outer frame structure features; Extracting a structural feature inside the frame as an internal structural feature, a frame structure reference table in which an external frame and internal structural features are registered in advance as one or more forms, and the detected external frame structure Features andSearch the frame structure reference table to determine form candidates, and for the determined candidatesSaid detectedBy matching with internal structure featuresRecognizing the type of the form by identifying the type of the form, and determining the format category of the form based on the structure of the outline frame in the form, and furthermore, a reference table for each outline frame in advance. Since the collation with the frame ruled line structure as a sample registered in the above is performed, there is an effect that the type of the form can be specified accurately.
[0012]
Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 21.
(Embodiment 1)
FIG. 1 shows a block diagram of a form recognition apparatus, wherein 1 is an image input means for reading a form to obtain a binary image, 2 is an image memory for storing the binary image, and 3 is a horizontal and vertical direction of the binary image. Ruled line extracting means for extracting ruled lines; 4 is a corner detecting means for detecting a corner of a binary figure; 5 is a component for detecting an L-shaped element, a T-shaped element, and a cross-shaped element by bending or intersecting a ruled line from a combination of corner shapes. Detecting means, 6 is a rectangular detecting means for connecting the constituent elements and outputting the connected form as frame structure information, and 7 is detecting the outermost outer frame including the rectangular structure in the form, and forming a global frame of the form. Outer frame detecting means 8 for extracting the structural features as outer frame structural features, 8 is a feature extracting means for extracting features such as the number, position, width, height, etc. of the inner frames surrounded by the individual outer frames as internal structural features. , 9 is the outline of the outer frame A frame structure reference table for classifying the form based on the outer frame structure feature, which is a feature, and registering the outer frame structure feature and the internal structure feature as the structural features of each outer frame, 10 is a frame structure on the frame structure reference table. Frame structure matching means for recognizing the type of the form and detecting a character reading target frame by comparing with a feature, 11 is a character extracting means for extracting a character area of the reading target frame, and 12 is a character recognizing means for recognizing the extracted character. is there.
[0013]
The operation of the form recognition device configured as described above will be described.
The form is read by the image input means 1, converted into a binary image having a character portion of 1 and a background of 0, stored in the image memory 2, and scanned by the ruled line extracting means 3 in the horizontal and vertical directions. A line in which the value “1” is continuous over a predetermined length is extracted, a corner detecting means 4 detects a corner of a pattern composed of horizontal and vertical ruled lines, and a component detecting means 5 detects a bent portion of the ruled line from a combination of corner shapes. And L-shaped, T-shaped, and cross-shaped components at the intersections and crosses are detected by the rectangular detection means 6 to detect the form of connection, and the outer frame detection means 7 includes the rectangular structure in the form. The outermost outer frame is detected, and global features such as the total number, position, and shape of the outer frames are extracted. The number, position, width, Features such as height are extracted, In the reference table 9, the structure of the outer shape frame of the form as a sample and the internal characteristics of each outer shape frame are registered in advance, and the frame structure collation unit 10 stores the frame structure information of the target form and the frame structure reference table 9. The form is identified by collating with the frame structure as a sample, the frame to be read in the form is recognized, the coordinates of the vertices are detected, and the character cutout unit 11 cuts out the character area based on the coordinates of the four vertices of the frame to be read. The character recognition unit 12 recognizes the extracted character.
[0014]
Next, the operation of each component in FIG. 1 will be described in detail. The image input means 1 which reads a form and outputs a binary image has a reading linear density of about 400 dpi, illuminates a form as an original with an LED (light emitting diode) or the like, reads the reflected light thereof with a one-dimensional CCD camera, This is to output a binary image having a character portion of value 1 and a background of value 0 by binarizing with an arbitrary threshold value. In addition, the illumination varies depending on the frame line of the document as a manuscript and the color of the entered characters, but, for example, numerals, symbols and characters are written in black, blue, etc. for the frame lines such as blue, black and red. In this case, an LED having a green or yellow-green wavelength (around 550 to 570 nm) is often used. In the binarization processing, a fixed threshold method or a floating threshold method (Mori, Otsu: “Binarization as a recognition problem and various methods”, Information Processing Society of Japan, Image Processing 15-1, Nov. 1977) is good. This is a known binarization processing method, and an arbitrary binarization processing method may be selected according to a document.
[0015]
The binarized image data is stored in the image memory 2, and the image data on the memory is subjected to ruled line extraction and character cutout described below.
[0016]
Next, the ruled line extracting means 3 will be described with reference to FIG. FIG. 2 shows a block diagram of the image processing in the ruled line extracting means 3, wherein 20 is a binary image from the image memory 2, 21 is a horizontal shrinking means for shrinking the pattern in the horizontal direction, and 22 is a pattern extending in the horizontal direction. Horizontal extension means, 23 is a vertical contraction means for contracting the pattern in the vertical direction, 24 is a vertical extension means for extending the pattern in the vertical direction, 25 is a NOR operation of outputs of the horizontal extension means 22 and the vertical extension means 24 Is a NOR circuit that performs the following. By shrinking the binary image in the image memory 2 by h pixels in the horizontal direction by the horizontal shrinking means 21, lines and characters having a width of h pixels or less disappear in the horizontal direction. , A horizontal line segment longer than h pixels is extracted. Similarly, by shrinking v pixels in the vertical direction by the vertical contraction means 23, lines and characters having a width of v pixels or less disappear in the vertical direction, and v pixels are extended by v pixels in the vertical direction by the subsequent vertical extension means 24. Vertical lines longer than the pixel are extracted. When the NOR operation of the outputs of the horizontal extension means 22 and the vertical extension means 24 is performed by the NOR circuit 25, a binary image is obtained in which characters are deleted and only the frame ruled lines remain. It has a value of "1".
[0017]
The contraction and extension processing in the horizontal and vertical directions in FIG. 2 will be described in more detail with reference to FIGS.
[0018]
FIG. 3 is a flowchart showing the operation of the horizontal and vertical contraction means 21 and 23. The binary image is sequentially scanned one line at a time in the horizontal or vertical direction and processed until the end line. When the pixel contraction process is performed, assuming that the count value of the run length is C, the count value C is set to 0 at the start of scanning of each line in step 31, one pixel data is read in step 32, and the pixel value is read in step 33. It is determined whether the value is 0 (white) or 1 (black). If the value is 0, the process proceeds to step 34, where the count value C is set to 0. Further, the process proceeds to step 35, and the value 0 is output because it is not a black run. If the value in step 33 is 1, the flow advances to step 36 to determine whether or not the count value C is equal to or more than n. If the value is less than n, the flow advances to step 37 to increment the count value C. Since a predetermined black run does not exist up to the position, a value 0 is output. When the count value C is equal to or greater than n in the determination in step 36, the process proceeds to step 38, and a value 1 is output because a black run having a run length of n pixels or more exists up to the scanning position. By performing the above processing until the end of one line, the black run on that line is reduced by n pixels. When processing the next line, the same processing is repeated from step 31 again. When scanning of the entire screen is completed in this way, a line segment having a run length of n pixels or more in the horizontal or vertical direction is extracted.
[0019]
Similarly, FIG. 4 is a flow chart showing the operation of the horizontal and vertical extension means 22 and 24. The binary image is sequentially scanned one line at a time in the horizontal or vertical direction, and processed until the end line. Assuming that the count value of the run length is C when performing the extension process of n pixels at step S, the count value C is set to n at the start of scanning of each line in step 41, one pixel data is read in step 42, and It is determined whether the value of the pixel is 0 (white) or 1 (black). When the value is 1, the process proceeds to step 44, where the count value C is set to n. . If the value in step 43 is 0, the process proceeds to step 46 to determine whether or not the count value C is equal to or less than 0. If the value is greater than 0, the process proceeds to step 47 to decrement the count value C. Since the position is within n pixels from the black run, a value of 1 is output. When the count value C is equal to or less than 0 in step 46, the process proceeds to step 48, and the value 0 is output because the scanning position is farther than n pixels from the black run. By performing the above processing until the end of one line, the black run on that line is extended by n pixels. When processing the next line, the same processing is repeated from step 41 again. When scanning of the entire screen is completed in this way, the run length is extended by n pixels in the horizontal or vertical direction.
[0020]
Next, a series of processes in the corner detecting means 4, the component detecting means 5, and the rectangular detecting means 6 will be described. The details of these processes are described in Japanese Patent Application No. 7-016682 filed by the same applicant. The description is omitted, and the operation is briefly described.
[0021]
First, the corner detecting means 4 will be described with reference to FIGS. FIG. 5 is a diagram showing a result of converting an input image into a direction-coded image as preprocessing for detecting a corner, FIG. 6 is a diagram showing a correspondence relationship between direction codes 1 to 8 and actual directions, and FIG. It is a figure showing the example of the corner to detect. In FIG. 5, reference numeral 51 denotes a pixel of a frame ruled line, 52 denotes a pixel of a background, and a numeral denotes a direction code assigned to an outline point. In this case, a tracking direction when the outline is traced clockwise with respect to the background pattern. Are assigned as direction codes 1 to 8. A change point of the direction code, that is, a corner is detected from the image thus coded. For this purpose, a position that is not the same direction code as the target pixel is detected in the direction indicated by the target position (center pixel) code near 3 × 3. In FIG. 5, the circled position indicates a change point of the direction code. For example, at the position 53, the pixel arrangement is as shown in FIG. The direction code of the position is “1”, which means that the direction of the outline changes from “3” to “1”. Change codes) are detected as one set. Similarly, the pixel positions 54, 55, and 56 correspond to (b), (c), and (d) of FIG. 7, and are given direction change codes of "17", "75", and "53", respectively. For these corner points, the x-coordinate, y-coordinate, and direction change code are notified to the component detection unit 5 as a set of feature information.
[0022]
Next, the component detection means 5 will be described with reference to FIGS. FIG. 8 is a diagram showing a determination condition for detecting a component from a combination of corner points, and FIG. 9 is a diagram showing a description format of the component. In FIG. 8, (a), (b), (c), and (d) are examples of detecting an L-shaped element, (e), (f), (g), and (h) are examples of detecting a T-shaped element, and (h) is a cross-shaped element. An example of detection is shown, in which a plurality of corner points having x and y coordinates within a predetermined distance are grouped using the characteristic information of the corner points from the corner detection means 4, and the directions of the corner points which are members of the group are grouped. The type of the component is associated with the combination of the change codes. The shape of the component detected in this way is described by a 4-bit code (hereinafter referred to as a shape code) as shown in FIG. 9A, and each bit is represented by S, B in FIG. In which direction W, N, or E the arm is present is shown. For example, the L-shaped element shown in FIG. 8A has arms in the S direction and the E direction, and is described by a bit pattern of “1001”. The x-coordinate and y-coordinate of the component give the average value of the x-coordinate and y-coordinate of the corner point which is a member of the group, and the component detecting means 5 calculates the x-coordinate and y-coordinate and the shape of the component. The code is notified to the rectangle detecting means 6 as feature information.
[0023]
Next, the rectangle detecting means 6 will be described with reference to FIG. FIG. 10 is a diagram showing a connection relationship between components. FIG. 10A shows an example of an L-shaped element, a T-shaped element, and a cross element detected by the component detecting means 5, and the rectangle detecting means 6 describes a connection relationship of these components as rectangle information. Generate and output a table. First, identification labels e1 to e20 are given to each component, and then, based on the feature information (x coordinate, y coordinate, shape code) from the component detection means 5, the arm indicated by the shape code in the x direction and y A direction is searched, and a component having a connectable arm and located at the shortest distance is detected. FIG. 6B shows a connection table showing the connection relationship of the components, and describes to which component each component is connected in each of N, S, E, and W directions. For example, in the case of the L-shaped element e1, there are e15 and e2 as components corresponding to the arms S and E, and in the case of the T-shaped element e2, e8, e1, as components corresponding to the arms S, W, and E. And e3 will be present. The attribute flags shown in FIG. 3B are parameters used in the feature extracting means 8, and the specific contents will be described later.
[0024]
Next, the outer frame detection means 7 will be described with reference to FIGS. FIG. 11A is a diagram illustrating an example of an outline frame, FIG. 11B is a diagram illustrating a path from tracking the outline frame to a clockwise direction and returning to the start point, and FIG. 12 is a flow of a process of detecting the outline frame. FIG. In FIG. 11A, rectangles 62, 63, and 64 indicated by thick lines are outline frames, and are formed of outermost ruled lines of a closed frame structure in the form 61. For example, with respect to the outline frame 62, as shown in FIG. 6 (b), the tracing path from the start point element f1 to the outline frame in the clockwise direction and back to the start point is f1, f2, f6, f10, f12, f11, f9, f5, and f1, and the vertices of the outer frame become components that change the tracking direction as indicated by circles, and are f1, f2, f12, f11, and f1, and these are called vertex paths. In the following description, the tracking route is represented by [...] And the vertex route is represented by {...}. For the outer frame 63, the tracking route is [f3 f4 f8 f14 f13 f7 f3], and the vertex route is {f3 f4 f14. f13 f3}, the tracking path of the outer frame 64 is [f15 f16 f17 f20 f24 f29 f33 f32 f31 f30 f25 f21 f18 f15], and the vertex is {f15 f17 f33 f32f31 f30 f15}. Therefore, in the example of the form shown in FIG. 11A, the total number N of the outer frames is three. Next, a process for extracting these outer frames will be described with reference to FIG. First, variables used in the flowchart will be described. The input variables are the component connection table shown in FIG. 10B, and the identification label is ei, the x coordinate is x (ei), the y coordinate is y (ei), and the shape is the i-th component on the table. The code is code (ei), the direction pointer is dk (ei), and k = 0, 1, 2, and 3 correspond to each direction of S, W, N, and E. i takes a value from 0 to (M-1), where M is the total number of components on the table. Next, as output variables, the total number of outline frames is N, the tracking path is Bj = [bj (0) bj (1)...], And the vertex path is Cj = ｛cj (0) cj for the j-th outline frame. (1) ...｝ However, j takes a value from 0 to (N-1). The control variables are a component pointer sp that has started tracking of the outer frame, a component pointer np for tracking the next, a variable dir indicating the current tracking direction, and a tracking flag f of the outer frame. First, at 71, the total number N of outer frames is set to 0, at 72, a flag f indicating whether or not the outer frame is being tracked is set to 0, and at 73, 0 is set to control variables i and j. Hereinafter, the individual outline frames are tracked clockwise while referring to the component connection table. If the control variable i is equal to or greater than the total number M of components in 74, the process ends; otherwise, the process proceeds to 75. At 75, it is determined from the value of the flag whether or not the outer frame is currently being tracked. When the flag value is 0, that is, when the starting point of the outer frame is being searched, the process proceeds to 76, and when the flag value is 1, that is, at the outer frame. Is being tracked, the process proceeds to the determination at 84. The determination in step 76 is to determine whether or not the currently focused component is the starting point of the outer frame tracking, and to determine whether or not the left shoulder element of the outer frame, that is, the component having arms in the S and E directions. If so, the flow advances to 78 to set the tracking flag f, and if not, the flow advances to 77 to update the control variable i and check the next component. Steps 79 and 80 are for giving initial values to the control variables. At 79, a start point pointer is set. At 80, a value 3 indicating the direction E is set to the control variable kk and a variable dir indicating the current tracking direction. Next, at 81 and 82, the identification code ei of the component of interest at present is registered in the tracking paths Bj and Cj. Then, in step 83, the value of the E-direction pointer of the component currently focused on is set to the next point pointer np. Next, the process returns to the determination 75 again, and proceeds to the determination 84 since the tracking flag f is set this time. Reference numeral 84 denotes a determination as to whether the next point pointer np and the start point pointer sp coincide with each other. If they match, it means that the frame has been closed as an outline frame, and the routine proceeds to 94 where the total number N of outline frames is incremented. j is incremented, the tracking flag f is reset at 96, the control variable i is updated at 77, and the determination at 74 is performed again. When the next point pointer and the start point pointer do not match in the determination at 84, the pointers in the S, W, N, and E directions in the component pointed to by the next point pointer np are searched to determine the next np. Reference numeral 85 denotes a process for determining from which direction to search in the directions of S, W, N, and E. In the expression,% indicates a modulo operation. That is, by adding 3 to the current direction kk and performing a modulo operation, the clock direction is checked in the clockwise direction from the direction immediately adjacent to the current direction clockwise. At this time, in the determination at 86, it is determined whether or not the user has an arm in the direction. If the user does not have an arm, the value of kk is updated at 87 and the process proceeds to 85 again. . At 88, a determination is made as to whether or not the arm direction matches the current tracking direction. If not, the tracking direction changes. Therefore, a new tracking direction is set to the variable dir at 90, and attention is paid to 91 at 91. The registered component is registered in the vertex route Cj as a direction change point, that is, a vertex, and is also registered in 89 at the tracking route Bj. If the tracking direction does not change in the determination at 88, registration is performed only on the tracking route Bj. Subsequently, the identification code of the component in the direction k obtained is set to the next point pointer np, the value of k is set to the control variable kk indicating the current direction, and the determination of 75 or less is repeated again. By executing the above processing until the end, all the outline frames in the form are detected, and the total number N, the tracking path Bj, and the vertex path Cj are obtained.
[0025]
Next, the feature extracting means 8 will be described with reference to FIGS. 13 and 14 are flowcharts for detecting the attribute of the component to the external frame, and FIG. 15 is a flowchart for detecting the internal frame of each external frame from the component connection table. First, variables used in the flowchart of FIG. 13 will be described. The input data is the component connection table shown in FIG. 10B and the tracking route Bj (j = 0 to N−1) obtained by the processing of FIG. 12, and the identification label, shape code, and direction pointer are The notation is the same as that in FIG.
The total number of elements bj (h) belonging to the tracking route Bj is denoted as pj. On the other hand, the output data is an attribute flag g (ei) indicating which outline frame each component belongs to. That is, it is to determine which outer frame the constituent element shown in FIG. 10B belongs to. Prior to the determination, it is assumed that the attribute flags g (ei) of all the constituent elements are set to -1 in advance. In FIG. 13, first, control variables j, h, i, and k are set to 0 in 101 to 104, and a component on the tracking path is set at the position of the direction pointer of the arm of the component of interest in the determination of 109. Determine whether they match. If they match, the flow advances to step 114 to set the value of j, that is, the number of the outer frame to which the attribute flag g (ei) belongs. On the other hand, if they do not match, the process proceeds to determination of 111 or less. At 111, it is determined whether or not the control variable i has advanced to M-1. If the condition is satisfied, the process from 104 is repeated, and if not, the process proceeds to 112 and below. At 112, it is determined whether or not the control variable h has advanced to the number of elements pj of the tracking path Bj. If the condition is satisfied, the processing from 103 is repeated, and if not, the processing proceeds to determination at 113. At 113, it is determined whether or not the control variable j has advanced to N-1. If the condition is satisfied, the process from 102 is repeated, and if not, the process proceeds to the process shown in FIG. By the above processing, the attribute flag g (ei) of the component on each outer frame is set. The process shown in FIG. 14 refers to the component connection table itself, and determines whether or not an element whose attribute has not yet been determined matches the direction pointer of the component whose attribute has already been determined. This is a process of giving a flag. First, at 121, a control variable i is set to 0 and a flag f is set to 0. Next, at 128, it is determined whether or not the attribute of the component ei is determined. If it has not proceeded to 1, i is updated and the determination of 128 is performed again. If the attribute has not been determined, the process proceeds to step 122, where the flag is set to 1, the process proceeds to step 123, and the control variable s is set to 0. Then, in step 129, a component whose attribute is already determined is searched. If the attribute has not been determined yet, the process proceeds to the determination of 131, and if the control variable s has not advanced to M-1, s is updated and the determination of 129 is performed again. If there is a component whose attribute is determined, the control variable r is set to 0 at 124, and it is determined at 130 whether or not the component ei of interest exists in the direction pointer. Goes to 135 and sets g (es) in the attribute flag g (ei). If the condition of 130 is not satisfied, r is incremented at 125 to see another direction pointer. If the control variable i has advanced to M-1 in 133, the process proceeds to 134, and the value of the flag f is checked. If the value is not 0, the process is repeated from 121 again. If the value of the flag remains 0 in the determination of 134, it means that the attributes of all the constituent elements have been given, and the process ends. Next, a processing flow for detecting a frame inside each external frame from the component connection table will be described with reference to FIG. First, variables used in FIG. 15 will be described. The data to be input is the component connection table shown in FIG. 10B, and the identification label, the direction pointer, and the affiliation flag have the same notation as in FIGS. On the other hand, the output data is the internal frame structure table Rj for each external frame and the total number tj of internal frames for each external frame. Where j takes a value from 0 to (N-1)
And
[0026]
When the upper left point of the four vertices of the frame is defined as the start point, the internal frame structure table Rj has the identification number of each frame, the x and y coordinates of the start point, the width w and the height h of the frame as elements. And First, in steps 141 to 143, 0 is set to the control variables i and j and the total number tj of internal frames. In the determination of 145, it is determined whether or not the target component belongs to the outer frame of interest. If it does not, the process proceeds to determination 159, and if it does, the process proceeds to determination 146. Reference numeral 146 determines whether or not the target component is the starting point of the frame. The determination condition is whether or not there is a component to be connected to the pointer in the E direction and the S direction. If it is the start point, the process proceeds to 147 and below, but the method of recognizing it as a frame from the start point is basically to follow a connecting line clockwise from the start point in the order of E → S → W → N, and to reach the previous element if a dead end is reached. Return A new component is selected, the search is repeated, and finally a set of components that can return to the starting point is defined as one frame. First, at 147, it is checked whether or not components are connected in the E direction of the target point. If no, the process proceeds to 159, and if yes, the process proceeds to 148, and the connected component in the E direction is set as the target point. Next, in 149, it is checked whether or not components are connected in the S direction of the target point. If no, the process returns to 147. If yes, the process proceeds to 150, and the connected components in the S direction are set as the target points. Subsequently, in 151, it is checked whether or not the components are connected in the W direction of the target point. If no, the process returns to 149. If yes, the process proceeds to 152, and the connected components in the W direction are set as the target points. Subsequently, in 153, it is checked whether or not the components are connected in the N direction of the target point. If no, the process returns to 151, and if yes, the process proceeds to 154, and the connected components in the N direction are set as the target points. Subsequently, in 155, it is checked whether the target point of 154 coincides with the start point. If no, the process returns to 153. If yes, one frame is detected and the process proceeds to 156. In 156, the width w of the frame is calculated from the difference between the x coordinate of the starting point and the x coordinate of the component in the E direction, and the height h of the frame is calculated from the difference between the y coordinate of the starting point and the y coordinate of the component in the S direction. I do. Next, at 157, the x coordinate, y coordinate, width w, and height h of the start point of the frame are registered in the internal frame structure table Rj, and at 158, the total number of internal frames tj is incremented. Thereafter, the flow advances to 159 to determine the control variable i. If i has not progressed to the total number M-1 of components, the flow advances to 160, where i is incremented and the flow again advances to the determination at 145. If i is M or more, the flow advances to 161. . At 161, the control variable j is determined. When j does not advance to the total number N−1 of outline frames, the process proceeds to 162, j is incremented, and the process returns to 142 again. When j is N or more, the process ends. By the outer frame detecting means 7 and the feature extracting means 8 described with reference to FIGS. 12 to 15, the total number of outer frames, vertex positions, and the inner structural characteristics of the individual outer frames are used as the outer frame structural features of the form. The total number, the starting point position, the width, and the height of the frame are detected, and the next frame structure collating means 10 collates with the frame structure of the form registered in the frame structure reference table 9 to identify the form and determine the character reading area. I do.
[0027]
Hereinafter, the frame structure reference table 9 and the frame structure matching means 10 will be described with reference to FIGS. FIG. 16 is a diagram showing a specific example of a form registered in the frame structure reference table 9, and FIG. The form shown in FIG. 16A has three outer frames 211, 212, and 213, and each outer frame has 13 inner frames, each of which has an identification number of (1) to (13). Has been given. Also, the form shown in FIG. 2B has two outer frames 221 and 222, and each outer frame has 11 inner frames, and identification numbers (1) to (11) are assigned to the respective outer frames. Have been. Also, the form shown in FIG. 3C has one outer frame 231 and 11 inner frames, each of which is given an identification number (1) to (11). Further, the form shown in FIG. 4D has one outer frame 241 and 13 inner frames, and are assigned identification numbers (1) to (13), respectively. The features of the frame structure of these forms are registered in the tables shown in FIGS. 17A shows a table for registering the outer frame structure, and FIG. 17B shows a table for registering the inner frame structure. In FIG. 3A, the elements registered in the table include the total number of outer frames, the number of vertices, the total number of internal frames in each outer frame, the identification number of the outer frame, and the vertex coordinates of the outer frame. By searching the table using these characteristics in, as parameters, form candidates are determined. Next, the detailed structure of the inner frame shown in FIG. That is, the starting point coordinates, width, and height of each frame are determined to be coincident, and when they match, the form is considered to be specified, and the starting point coordinates, width, and height of the input form frame corresponding to the identification number of the frame to be read are determined. It notifies the character extracting means 11. For example, in the case of the form shown in FIG. 16D, the reading target frames are (6) and (7), and the start point, width, and height of each frame are notified to the character cutout unit 11.
[0028]
Next, the character extracting means 11 will be described. The character extracting unit 11 extracts an actual character binary image from the image memory 2 and sends it to the character recognizing unit 12. The processing will be described with reference to FIG. FIG. 18 shows a binary image in which characters are drawn in the form frame. Here, reference numeral 251 denotes a starting point (xs, ys) of the frame to be read, 252 denotes a character reading area, and 253 denotes a character cutout area. Based on the start point (xs, ys), the width w, and the height h notified from the frame structure matching unit 10, the width (w−2dx) and the height from the start point 254 (xs + dx, ys + dy), taking into account the margins dx and dy. The area of (h−2 · dy) is determined as the reading area 254. The cut-out area for each character is determined by taking projections in the horizontal and vertical directions in this reading area, and an area having a predetermined frequency or more in the horizontal and vertical directions is set as a character cut-out area 253. Since character segmentation using projection is a general technique, a detailed description is omitted. The individual characters cut out in this way are transferred to the character recognition means 12 at the subsequent stage, and the character string in the designated frame can be recognized.
[0029]
【The invention's effect】
As described above, according to the present invention, a ruled line is extracted from a binary image of a form read by an image input unit, components that are intersections of the ruled lines are detected, and each component is connected to each other to form a frame. Generating the structure, detecting the outer frame of the frame structure, classifying the form based on its characteristics, further collating the frame structure inside each outer frame with a sample, specifying the type of the form, and detecting the frame to be read. Accordingly, even when forms of various formats are mixed, the form can be identified, the individual character reading area of each form can be accurately detected, and the highly reliable character recognition can be performed.
[Brief description of the drawings]
FIG. 1 is a diagram showing block connection of a form recognition apparatus according to an embodiment of the present invention;
FIG. 2 is a diagram showing block connection by ruled line extracting means in the form recognition apparatus according to the embodiment;
FIG. 3 is a flowchart showing a processing procedure in a line segment contracting unit in the form recognition apparatus according to the embodiment;
FIG. 4 is an exemplary flowchart showing a procedure of processing by a line segment extending means in the form recognition apparatus according to the embodiment;
FIG. 5 is a view for explaining direction coding and direction code change point detection by a corner detecting means in the form recognition apparatus of the embodiment.
FIG. 6 is a diagram showing a correspondence relationship between a direction code and an actual direction by a corner detecting means in the form recognition apparatus according to the embodiment;
FIG. 7 is a diagram showing a specific example of a direction change code in a corner detection unit in the form recognition device of the embodiment.
FIG. 8 is a diagram showing a specific example of a component based on a combination of corner points in a component detection unit in the form recognition apparatus of the embodiment.
FIG. 9 is an exemplary view for explaining the description of the form of the component in the component extracting means in the form recognition apparatus of the embodiment.
FIG. 10 is a conceptual diagram showing a connection relationship between components in a rectangle detection unit in the form recognition apparatus of the embodiment.
FIG. 11 is a diagram showing an example of outer frame detection by the outer frame detector in the form recognition apparatus according to the embodiment;
FIG. 12 is a flowchart showing a procedure of an outer frame detection process by an outer frame detector in the form recognition apparatus according to the embodiment;
FIG. 13 is a flowchart showing a procedure of a process of detecting an attribute of a component by a feature extracting unit in the form recognition apparatus of the embodiment.
FIG. 14 is a flowchart showing a procedure of a process of detecting an attribute of a component by a feature extracting unit in the form recognition apparatus of the embodiment.
FIG. 15 is a flowchart showing a procedure of a process of detecting an internal frame structure in the feature extracting means in the form recognition apparatus of the embodiment.
FIG. 16 is a diagram showing a specific example of a form registered in a frame structure reference table in the form recognition apparatus of the embodiment.
FIG. 17 is a diagram showing a table structure in a frame structure reference table in the form recognition apparatus according to the embodiment;
FIG. 18 is a diagram showing processing of a character cutout unit in the form recognition apparatus of the embodiment.
FIG. 19 is a diagram showing a processing operation of a conventional form recognition device.
FIG. 20 is a diagram showing a processing operation of a conventional form recognition device.
FIG. 21 is a diagram showing a processing operation of a conventional form recognition device.
[Explanation of symbols]
1 Image input means
2 Image memory
3 Ruled line extraction means
4 Corner detection means
5 Component detection means
6 rectangle detection means
7 Outer frame detection means
8 Feature extraction means
9 Frame structure reference table
10 Frame structure collation means
11 Character extraction means
12 Character recognition means
20 binary image
21 Horizontal contraction means
22 Horizontal extension means
23 Vertical contraction means
24 Vertical extension means
25 NOR circuit
51 Frame ruled line pixels
52 Background Pixel
53-56 corner points
61 Reports
62-64 outer frame
210 Form A
211-213 Outer frame
220 Form B
221-222 Outer frame
230 Form C
231 Outer frame
240 Form D
241 outer frame
251 start of frame
252 character reading area
253 character cut-out area
Starting point of 254 character reading area

Claims (8)

A corner detecting means for detecting a corner of the frame ruled line from the binary image; a component detecting means for detecting a component of the frame ruled line from a combination of the detected corner shapes of the frame ruled line; , a rectangle detection means for outputting a frame structure information, the frame structure information to detect the outline frame outermost based, and outline frame detecting means for extracting a feature of the structure of the outer shape frame as outline frame structure, wherein, feature extracting means for extracting the internal structural features as an internal structural characteristic for each extracted outline frame, the outer frame and inner structural features previously registered frame structure reference table as the format of one or more of the form, is the detection contour frame structure, wherein the determining the candidates of the frame structure by referring to searching a table form, specifying the type of a form by collating with the detected internal structural features with respect to a predetermined candidate frame Form recognition apparatus comprising a granulation checking means. Further, the image processing apparatus further includes a ruled line extracting means for extracting a frame ruled line by detecting runs having a predetermined length or more in the horizontal and vertical directions from the binary image including the frame ruled line and the character. 2. The form recognition device according to claim 1, wherein the form is detected . 3. The component detecting unit according to claim 1, wherein the component detecting unit detects an L-shaped component, a T-shaped component, and a cross-shaped component from the combination of the shapes of the corners as components formed by bending or intersecting ruled lines . Form recognition device. 2. The form recognition apparatus according to claim 1, wherein the outer shape frame detecting means connects the vertex coordinates and extracts each of the outer shape frames as an outer shape structural feature. 5. The form recognition apparatus according to claim 4, wherein the outline frame structure feature extracts the total number, position, and shape of the outline frames in the target form. 2. The form recognition apparatus according to claim 1, wherein the feature extracting means extracts the number, position, width and height of the internal frames as internal structural features for each external frame. Further, an image memory for storing a binary image including a frame ruled line, and a character cutout unit for cutting out a character area from the image memory based on the coordinates of the character reading target area from the detected outer frame structure feature and internal structure feature, 7. The form recognition apparatus according to claim 1, further comprising a character recognition unit that recognizes the cut-out character. Detecting a corner of the frame ruled line from the binary image; detecting a component of the frame ruled line from a combination of the detected corner ruled shape of the frame ruled line; connecting the components to each other to form frame structure information; Outputting, and detecting the outermost outer frame based on the frame structure information, and extracting a feature of the outer frame structure as an outer frame structure feature.
Extracting the internal structural features as internal structural features for each of the extracted external frames; a frame structure reference table in which the external frames and internal structural features are registered in advance as one or more forms of the form; The form type is determined by searching the outer frame structure feature and the frame structure reference table to determine a form candidate, and identifying the form type by comparing the determined candidate with the detected internal structure feature. And a recognition step.

Images