JP2016115088A - Document definition device, document definition method, and document definition program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate definition information for associating a keyword to be associated with attributes of characters to be entered in a character frame of a document to be optionally selected with the character frame for the document.SOLUTION: Image data of a document is generated by optically reading the document, document image data is generated by binarizing the image data, corrected document image data is generated by performing correction for compensating a defect generated by binarization to the document image data, rectangular or approximate rectangular shapes are detected as squares from the corrected document image data, whether or not the squares are character frame square is determined, the character frame squares are classified based on adjacent relation of the character frame squares, each of classified categories is detected as a field, characters included in the document image data are detected, a keyword with which attributes of the characters to be entered in the field is preliminarily associated is detected from the characters, the field corresponding to the keyword is detected based on position relation between the keyword and the field, the detected field is stored by associating it with the keyword.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a form definition device, a form definition method, and a form definition program.

  Generate an image to detect characters and character frames printed on blank forms before reading the characters handwritten on the form using a scanner that optically reads characters printed on paper media. Then, only information on characters and character frames printed from the image is read and stored. Thereafter, a form in which characters are handwritten in a character frame is read, and only handwritten characters are extracted from the read image and stored. Then, when you want to check the contents of the handwritten characters, separate the contents displayed in the form such as displaying the information of the stored characters and character frames and the information of the handwritten characters on the screen. Thus, there is a technique for performing processing (see, for example, Patent Document 1).

  By the way, in the form, by showing the print characters such as “name” and “account number” as keywords around the character frame, the character who fills in each character frame of the form is shown to the writer. I am doing so. If such a keyword can be used in a form processing by a computer, it is possible to check whether or not the contents handwritten in the character frame are clearly wrong. For example, if a numerical value is written in the character box corresponding to the keyword even though the keyword "name" exists, it is checked that the character is clearly mistaken during character recognition processing by the computer. be able to.

Japanese Patent No. 4269059

Conventionally, in order to use a keyword printed on a form as described above in a form processing using a computer, a user can manually enter a keyword and a character frame from the stage of form creation using a computer. It was necessary to define the position to be arranged and to further define attributes indicating the type of character described in the character frame.
However, since the work for defining as described above is based on the manual work of the user, there is a problem that it is a work that requires a great deal of labor in order to deal with each form having various styles. . Further, even with the above-described conventional technology, there is a problem that definition information that associates a keyword with a character frame cannot be generated from an image obtained by reading a form.

  The present invention has been made to solve the above-described problem, and an object of the present invention is to associate a keyword associated with an attribute of a character written in a character frame of the form with the character frame for an arbitrarily selected form. The object is to provide a form definition device, a form definition method, and a form definition program capable of generating definition information.

  In order to solve the above problem, an aspect of the present invention includes an image reading unit that optically reads a form and generates image data of the form, and binarization that generates form image data by binarizing the image data. A correction unit that generates correction form image data by correcting a defect generated by the binarization with respect to the form image data, and a rectangular or substantially rectangular shape from the correction form image data as a rectangle A rectangle detection unit to detect, a rectangle determination unit that determines whether or not the rectangle is a character frame rectangle, and classifies the character frame rectangles based on the adjacent relationship of the character frame rectangles, and each of the classified categories A field detection unit for detecting a character as a field, a character detection unit for detecting a character included in the form image data, and a keyword associated with a character attribute to be written in the field in advance. A keyword detection unit that detects from a written character; and the field corresponding to the keyword is detected based on a positional relationship between the keyword and the field, and the detected field is associated with the keyword in a form definition information storage unit. A form definition device comprising a field search unit for writing.

  Further, according to one aspect of the present invention, in the above-described invention, a character string or a character to be detected is predetermined as the keyword, and the keyword detecting unit detects the keyword from the character. Only the keyword that completely matches or partially matches the predetermined character string or character to be detected may be detected.

One aspect of the present invention is the above-described invention, further comprising a dictionary attribute information storage unit that stores in advance attribute information indicating the attribute corresponding to the keyword, and the keyword detection unit includes:
When the keyword is detected from the characters, the attribute information corresponding to the keyword is read from the dictionary attribute information storage unit, and the read attribute information is written in the form definition information storage unit in association with the keyword. Also good.

  Further, according to one aspect of the present invention, in the above-described invention, the positional relationship between the keyword and the field is a tendency in a direction in which the field exists in advance when the keyword is vertically written, or When a keyword is written horizontally, it is indicated by a tendency in a direction in which the predetermined field exists, and when the field satisfies the tendency, the field search unit detects the field as the field corresponding to the keyword. You may do it.

  Further, according to one aspect of the present invention, in the above-described invention, the rectangle determination unit includes a keyword corresponding rectangle detection unit that detects the rectangle surrounding the keyword from the rectangle that is determined not to be the character frame rectangle. When the rectangle surrounding the keyword exists, the field search unit detects the field corresponding to the keyword by using the positional relationship between the rectangle surrounding the keyword and the field as the positional relationship between the keyword and the field. You may make it do.

  Further, according to one aspect of the present invention, in the invention described above, a ruled line erasure corresponding image dividing unit that divides the form image data, a ruled line erasing unit that erases ruled lines from each of the divided form image data, A ruled line erasure-compatible image composition unit that synthesizes the divided form image data after the ruled line erasure, and the ruled line erasure-compatible image division unit exists at the boundary of the division when dividing the form image data The ruled line erasure corresponding image synthesizer is configured such that the ruled line is erased by the ruled line erasure unit and divided so that there is an overlapping region so that there is an overlapping region at each boundary of the divided form image data. Deletes the extended area of each of the divided form image data and combines them, and the character detection unit erases the ruled lines output from the ruled line erasure corresponding image combining unit From bookkeeping form image data may be detected and the character.

  According to another aspect of the present invention, in the invention described above, the field search unit may associate a plurality of keywords with the field.

  In order to solve the above problem, according to one aspect of the present invention, a form is optically read to generate form image data, the image data is binarized to generate form image data, and the form image data is Whether or not a rectangle or a substantially rectangular shape is detected as a rectangle from the corrected form image data by correcting the gap generated by the binarization to generate a corrected form image data. Determining whether or not to classify the character frame rectangle based on the adjacency relationship of the character frame rectangle, detecting each of the classified sections as a field, detecting a character included in the form image data, A keyword associated with a character attribute to be entered in advance is detected from the character, and the field corresponding to the keyword is defined as the position of the keyword and the field. Detected on the basis of the engagement, a form definition method characterized in that in association with the field detected in the keyword.

  In order to solve the above-described problem, an aspect of the present invention provides a computer with a procedure for optically reading a form to generate image data of the form, a procedure for generating the form image data by binarizing the image data, A procedure for generating correction form image data by correcting the form image data to fill in the defects generated by the binarization, a procedure for detecting a rectangle or a substantially rectangular shape from the correction form image data as a rectangle, the rectangle A procedure for determining whether or not a character frame rectangle, a procedure for classifying the character frame rectangle based on the adjacency relationship of the character frame rectangle, and detecting each of the classified segments as a field, the form image data A procedure for detecting included characters, a procedure for detecting a keyword associated with an attribute of a character entered in the field in advance from the characters, the keyword The corresponding field, the procedure for detecting based on the positional relationship between the keyword and the field, the procedure in association with the field detected in the keyword, a form definition program for executing.

  According to the present invention, it is possible to generate definition information for associating a keyword associated with the attribute of a character entered in a character frame of the form with the character frame for an arbitrarily selected form.

It is a block diagram which shows the structure of the form definition apparatus by 1st Embodiment of this invention. It is a figure which shows the data structure of the rectangular information storage part in the embodiment. It is a figure which shows the data structure of the field information storage part in the embodiment. It is a figure which shows the data structure of the character information storage part in the embodiment. It is a figure which shows the data structure of the dictionary attribute information storage part in the embodiment. It is a figure which shows the data structure of the form definition information storage part in the embodiment. It is a flowchart (the 1) which shows the form definition process in the embodiment. It is a flowchart (the 2) which shows the form definition process in the embodiment. It is a figure which shows the image of the form which the image reading part of the embodiment read. It is a figure which shows the process by the thread | sled of the binarization part and correction | amendment part of the embodiment. It is FIG. (1) explaining the expansion process by the correction | amendment part of the embodiment. It is FIG. (2) explaining the expansion process by the correction | amendment part of the embodiment. It is FIG. (3) explaining the expansion process by the correction | amendment part of the embodiment. It is FIG. (4) explaining the expansion process by the correction | amendment part of the embodiment. It is FIG. (5) explaining the expansion process by the correction | amendment part of the embodiment. It is FIG. (6) explaining the expansion process by the correction | amendment part of the embodiment. It is FIG. (7) explaining the expansion process by the correction | amendment part of the embodiment. It is a flowchart (the 1) which shows the field detection process in the embodiment. It is a flowchart (the 2) which shows the field detection process in the embodiment. It is FIG. (1) explaining the field detection process in the embodiment. It is FIG. (2) explaining the field detection process in the embodiment. It is FIG. (3) explaining the field detection process in the embodiment. It is FIG. (4) explaining the field detection process in the embodiment. It is FIG. (1) explaining the image division | segmentation by the ruled line deletion corresponding | compatible image division part of the embodiment. It is FIG. (2) explaining the image division | segmentation by the ruled line deletion corresponding | compatible image division part of the embodiment. It is a figure which shows the process by the thread | sled of the ruled line deletion part of the embodiment. It is a figure explaining the keyword circumscribed rectangle in the embodiment. It is a figure explaining the rectangle which encloses the keyword in the embodiment. It is a flowchart which shows the field search process in the embodiment. It is FIG. (1) explaining the field search process by the embodiment. It is FIG. (2) explaining the field search process by the embodiment. It is FIG. (3) explaining the field search process by the embodiment. It is FIG. (4) explaining the field search process by the embodiment. It is FIG. (5) explaining the field search process by the embodiment. It is FIG. (6) explaining the field search process by the embodiment. It is FIG. (1) explaining the difference of the expansion process by the embodiment. It is FIG. (2) explaining the difference of the expansion process by the embodiment. It is a figure (the 3) explaining the difference of the expansion process by the embodiment. It is a figure (the 4) explaining the difference of the expansion process by the embodiment. It is FIG. (5) explaining the difference of the expansion process by the embodiment. It is FIG. (6) explaining the difference of the expansion process by the embodiment. It is FIG. (7) explaining the difference of the expansion process by the embodiment. It is FIG. (8) explaining the difference of the expansion process by the embodiment. It is FIG. (1) explaining the effect of the ruled line deletion corresponding | compatible image division part of the embodiment. It is FIG. (2) explaining the effect of the ruled line deletion corresponding | compatible image division part of the embodiment. It is FIG. (3) explaining the effect of the ruled line deletion corresponding | compatible image division part of the embodiment. It is a block diagram which shows the structure of 2nd Embodiment of this invention. It is a figure which shows the data structure of the reference information storage part of the embodiment.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a form definition device 1 according to the first embodiment of the present invention. In the form definition device 1, the image reading unit 10 is applied with, for example, a scanner, and is read as a color image such as a diagram, figure, or character that is optically printed on a paper medium or written by handwriting. The image is output as color form image data. Here, as an example, a case will be described in which each pixel of the read color form image data has 24-bit color information. The image dividing unit 11 divides the color form image data output from the image reading unit 10 into four equal parts by a line segment connecting the midpoints of the upper and lower sides and a line segment connecting the midpoints of the left and right sides.

  The binarization unit 12 converts the divided color form image data divided by the image dividing unit 11 from 24 bits to 1 bit, that is, converts from color to black and white to generate divided form image data. To do. Here, the binarization processing performed by the binarization unit 12 performs binarization so that the character lines remain thin and clear in order to increase the character detection rate in the character detection unit 24. The correction unit 13 performs correction for expanding the line on the divided form image data output from the binarization unit 12. As described above, the binarization processing performed by the binarization unit 12 is a process for converting each pixel to be represented by 1 bit of black and white and making the character line remain thin and clear. When this processing is performed, a defect, that is, a discontinuous portion may occur in a shape such as a rectangle formed by continuous line segments. There is an algorithm for detecting a rectangle by interpolating the defect even when the line segment has a defect. However, if the defect is not generated, the rectangle detection rate by the rectangle detector 15 can be increased. By the correction process by the unit 13, the line is expanded to correct the defect.

  The image composition unit 14 divides the divided form image data into four equal parts by the image division unit 11 and binarized by the binarization unit 12 and binarized form image data (hereinafter binarized). Generated form image data is referred to as form image data). Further, the image composition unit 14 divides the image into four equal parts by the image division unit 11, binarization by the binarization unit 12, and corrects the divided form image data corrected by the correction unit 13 to generate corrected form image data. To do. The image composition unit 14 outputs the generated form image data to the rectangle erasing unit 20 as image data for character detection, and outputs the generated corrected form image data to the rectangle detection unit 15 as image data for rectangle detection. .

  The rectangle detection unit 15 detects, by image processing, a rectangular shape or a substantially rectangular shape included in the corrected form image data as a rectangle, and coordinates information on the upper left X axis and Y axis of the detected rectangle, and a lower right The coordinate information of the X axis and the Y axis is output. Here, the substantially rectangular shape includes a shape in which all four inner corners are not necessarily a right angle but are curved at some or all corners. The origin of the XY coordinates in the image data read by the image reading unit 10 is a coordinate system in which the upper left is the origin, the right direction is positive in the X axis direction, and the lower direction is positive in the Y axis direction. The same applies to the following description. The rectangle determination unit 16 determines whether the rectangle detected by the rectangle detection unit 15 is a character frame rectangle in which a character is to be entered or other rectangles based on a predetermined size of the character frame rectangle.

  The rectangular information storage unit 50 stores rectangular information having the data configuration shown in FIG. 2. The rectangular information includes “rectangular ID (Identification)”, “upper left X and Y coordinates”, and “lower right X, Y”. It has items of “Coordinate”, “Rectangle type” and “Processing state”. The rectangle ID is an ID that can uniquely identify each rectangle in one form image data, and is generated every time the rectangle detector 15 detects a rectangle. The item “rectangle ID” is written by the rectangle determination unit 16 that has received the rectangle ID from the rectangle detection unit 15. The items of “upper left X, Y coordinates” and “lower right X, Y coordinates” include the upper left X, Y coordinate information of the rectangle detected by the rectangle detector 15 and the lower right X, Y coordinates. The information is written by the rectangle determination unit 16 that has received the information from the rectangle detection unit 15. In the item of “rectangle type”, the type determined by the rectangle determining unit 16, that is, “character frame” indicating that the character is a character frame rectangle, or “other” indicating that the rectangle is other than that is a rectangle determination. Written by unit 16. The item “processing state” is used by the field detection unit 17 and processed when it is selected as a reference character frame rectangle by the field detection processing by the field detection unit 17 or when it is classified into any of the fields. As the rectangle, “completed” is written by the field detection unit 17. In the “processing state” item, when the rectangle determination unit 16 writes a rectangle ID and generates a record, a blank “-” is written by the rectangle determination unit 16 as an initial value.

  The field detection unit 17 reads out the rectangle information in which the item of “rectangle type” is “character frame” from the rectangle information storage unit 50, and classifies the character frame rectangles based on the adjacent relationship of the read character frame rectangles. Each of the classifications when classified is detected as a field. Here, the field is a set of character frame rectangles. For example, when entering a name in katakana etc. on a form, it is common to arrange rectangles of the same size at the same height and at the same interval, and enter the names in the rectangles one by one in katakana. It is. A set of character frame rectangles in which the name is written is called a field. Note that the set of character frame rectangles constituting the field includes the case where there is one character frame rectangle. In addition, the above-mentioned adjacency relationship is whether or not the value of the upper left Y coordinate of the adjacent character frame rectangle is the same, whether or not the adjacent character frame rectangle exists within a predetermined range in the X-axis direction, Adjacent character frames such as whether or not the adjacent character frame rectangles are arranged at the same or substantially the same interval, the length of the character frame rectangles in the vertical direction, that is, whether the heights of the character frame rectangles are the same, etc. It shows the relationship with the rectangle. The field detection unit 17 detects the upper left X and Y coordinate information and the lower right X and Y coordinate information of the detected field. Further, the field detection unit 17 associates the detected upper left X and Y coordinate information, the lower right X and Y coordinate information, and the information of the rectangular rectangle ID included in the field in association with the detected field. Write to the information storage unit 51.

  The field information storage unit 51 stores field information having the data configuration shown in FIG. 3. The field information includes “field ID”, “upper left X and Y coordinates”, “lower right X and Y coordinates”, It has an item “configuration rectangle”. The field ID is an ID that can uniquely identify each field in one form image data, and is generated every time the field detection unit 17 detects a field. In the item “Field ID”, the field ID is written by the field detection unit 17. In the “upper left X, Y coordinates” item, the upper left X, Y coordinate information of the leftmost rectangle constituting the field is written by the field detection unit 17, and in the “lower right X, Y coordinates” item. The X and Y coordinate information at the lower right of the rightmost rectangle constituting the field is written by the field detector 17. In the item “configuration rectangle”, the field detection unit 17 writes a rectangle ID indicating the rectangle forming the field.

  The rectangle erasure unit 20 uses the rectangle information stored in the rectangle information storage unit 50 to erase the rectangle included in the form image data output by the image composition unit 14. The ruled line erasure corresponding image dividing unit 21 divides the form image data from which the rectangle has been erased by the rectangle erasing unit 20 into four, and outputs it to the ruled line erasing unit 22. Here, the four divisions of the form image data by the ruled line erasure corresponding image dividing unit 21 are performed by the line segment connecting the midpoints of the upper and lower sides and the line segment connecting the midpoints of the left and right sides performed by the image dividing unit 11 described above. It is different from equal division. The four divisions by the ruled line erasure corresponding image dividing unit 21 are divisions including an extended region that extends the boundary region of the division by the length of the black pixel that the ruled line erasing unit 22 detects as a ruled line. The ruled line erasing unit 22 erases the ruled lines from each of the form image data divided for ruled line erasing by image processing. The ruled line erasure corresponding image composition unit 23 deletes the extended area from each of the divided form image data after the ruled line erasure by the ruled line erasure unit 22 and performs composition.

  The character detection unit 24 recognizes characters one by one by image processing from the form image data from which the rectangle and the ruled line have been deleted, which is synthesized by the ruled line erasure corresponding image synthesis unit 23, and detects a character. The information indicating the character finally detected by the character detection unit 24 is character information such as a character code, for example. Note that the conversion from the recognized character to the character code may be performed by the character detection unit 24 using a conversion table stored in advance, or the character detection unit 24 may refer to an external database or the like. Alternatively, conversion from a recognized character to a character code may be performed. The character detection unit 24 detects a circumscribed rectangle circumscribing the detected character, and detects upper left X and Y coordinate information and lower right X and Y coordinate information of the detected circumscribed rectangle. Further, the character detection unit 24 writes the character information of the detected character, the upper left X and Y coordinate information of the circumscribed rectangle of the character, and the lower right X and Y coordinate information in the character information storage unit 52.

  The character information storage unit 52 stores character information having the data structure shown in FIG. 4. The character information includes “character ID”, “recognized character”, “upper left X and Y coordinates”, and “lower right X, Y”. "Coordinate" item. The character ID is an ID that can uniquely identify each character recognized in one form image data, and is generated every time the character detection unit 24 detects a character. In the item “character ID”, the character ID is written by the character detection unit 24. In the item “recognized character”, character information of a character recognized and detected by the character detection unit 24 through image processing is written by the character detection unit 24. The items of “upper left X, Y coordinates” and “lower right X, Y coordinates” include the upper left X, Y coordinate information of the circumscribed rectangle of the character and the lower right X, Y detected by the character detection unit 24. Coordinate information is written by the character detection unit 24.

  The dictionary attribute information storage unit 53 stores dictionary attribute information shown in FIG. The dictionary attribute information has items of “keyword” and “attribute information”. In the “keyword” item, characters printed in advance on a form to be detected or character information made up of character strings are stored in advance. Remembered. The information on the item “keyword” is also used as dictionary information when the keyword detection unit 25 detects a keyword from characters stored in the character information storage unit 52. In the “attribute information” item, information indicating an attribute associated with the keyword is written in advance. The information indicating the attribute is, for example, information indicating the type of character to be entered in the character frame. If the keyword is “request date”, the date is written, so the attribute of numerical value is written. Further, in the field for the keyword “request date”, in more detail, from a total of six character frames including two character frames indicating the last two digits of the year, two character frames indicating the month, and two character frames indicating the day. If it is predetermined, the range of numbers entered in each character frame, for example, for two character frames indicating the day, the first character frame is one of 0-3, An attribute that any one of 0 to 9 is written in the second character frame may be written in advance.

In the case of the keyword “month”, it is pre-written as attribute information that 0 or 1 and 9 are written in the first character frame in two character frames. The numerical value written in the two character frames may be written in advance as attribute information that the numerical value written in the two character frames is an integer of 1 to 12. In this case, for example, an attribute of “numerical value (with range specification)” is written in the item of “attribute information”, and in the “with range specification” portion, specifically, any integer from 1 to 12 "Is written.
In addition, in the case of the keyword “prefecture”, a kanji indicating one of the prefectural names may be written, but it is an option when several prefectures are selected. A predetermined prefecture name is written in advance as attribute information.

  The keyword detection unit 25 detects characters constituting the keyword based on the character information stored in the character information storage unit 52, and among the detected keywords, the item “keyword” in the dictionary attribute information storage unit 53 A character string or a character that is written in advance in the character string or a character that completely matches or partially matches is detected as a keyword. Further, when the keyword detection unit 25 detects a keyword, the keyword detection unit 25 reads attribute information associated with the keyword from the dictionary attribute information storage unit 53. The keyword detection unit 25 detects a circumscribed rectangle of the detected keyword, and detects upper left X and Y coordinate information and lower right X and Y coordinate information of the detected circumscribed rectangle. In addition, the keyword detection unit 25 generates a keyword ID that is an ID that can uniquely identify the keyword detected by the keyword detection unit 25 in one form image data, and detects the detected keyword and corresponding attribute information. The upper left X and Y coordinate information and the lower right X and Y coordinate information of the circumscribed rectangle are written in the form definition information storage unit 54 in association with the generated keyword ID. Further, the keyword detection unit 25 outputs the keyword ID and the information of the circumscribed rectangle of the keyword, that is, the upper left X and Y coordinate information and the lower right X and Y coordinate information of the circumscribed rectangle to the keyword corresponding rectangle detection unit 26.

  The form definition information storage unit 54 stores form definition information having the data structure shown in FIG. 6, and the form definition information includes “keyword ID”, “content”, “attribute information”, “X and Y coordinates at the upper left”, It has items of “lower right X and Y coordinates”, “corresponding rectangle”, and “corresponding field”. In the “keyword ID” item, the keyword ID generated by the keyword detection unit 25 by the keyword detection unit 25 is written. In the “content” item, a character string or a character indicating the content of the keyword detected by the keyword detection unit 25 is written. In the “attribute information” item, the attribute information read from the dictionary attribute information storage unit 53 by the keyword detection unit 25 based on the detected keyword is written. The items of “upper left X, Y coordinates” and “lower right X, Y coordinates” include the upper left X, Y coordinate information and the lower right X, Y coordinates of the circumscribed rectangle of the keyword detected by the keyword detection unit 25. Information is written by the keyword detection unit 25. In the item “corresponding rectangle”, a rectangle ID of a rectangle surrounding the keyword detected by the keyword corresponding rectangle detection unit 26 is written by the keyword corresponding rectangle detection unit 26. In the item “corresponding field”, the field search unit 27 writes the field ID of the field corresponding to the keyword detected by the field search unit 27. For the items “corresponding rectangle” and “corresponding field”, when the keyword detection unit 25 writes a keyword ID and generates a record, a blank “-” is written by the keyword detection unit 25 as an initial value.

  The keyword-corresponding rectangle detection unit 26 includes the upper left X and Y coordinate information and the lower right X and Y coordinate information of the circumscribed rectangle of the keyword output by the keyword detection unit 25 and the character frame stored in the rectangle information storage unit 50. The rectangle surrounding the keyword is detected based on the upper left X and Y coordinate information and the lower right X and Y coordinate information of the other rectangles. In addition, the keyword-corresponding rectangle detection unit 26 associates the rectangle ID indicating the detected rectangle with the keyword ID and writes it in the item “corresponding rectangle” of the form definition information storage unit 54. If the rectangle surrounding the keyword cannot be detected, the keyword corresponding rectangle detection unit 26 writes “none” in the item.

  The field search unit 27 refers to the coordinate information of the keyword circumscribing rectangle stored in the form definition information storage unit 54 and the field coordinate information stored in the field information storage unit 51, and the positional relationship between the keyword and the field Based on, the field corresponding to the keyword is detected. In addition, when a rectangle ID is stored in the item “corresponding rectangle” of the form definition information storage unit 54, the field search unit 27 stores the rectangular coordinate information instead of the keyword circumscribed rectangle coordinate information. Read from unit 50. Further, the field search unit 27 refers to the coordinate information of the read rectangle and the coordinate information stored in the field information storage unit 51, and the field corresponding to the keyword based on the positional relationship between the rectangle surrounding the keyword and the field. Is detected. The field search unit 27 writes the field ID indicating the detected field in the item “corresponding field” of the form definition information storage unit 54 in association with the keyword ID indicating the keyword.

(Form definition processing)
A form definition process performed by the form definition apparatus 1 according to the first embodiment will be described. 7 and 8 are flowcharts showing the flow of the form definition process. The image reading unit 10 optically reads a form and outputs, for example, color form image data F composed of 24 bits shown in FIG. 9 (step Sa1). The image dividing unit 11 divides the color form image data F output from the image reading unit 10 into four equal parts (step Sa2). Here, the quarter is divided into a line segment connecting the midpoints of the upper and lower sides of the color form image data F and a line segment connecting the midpoints of the left and right sides. The upper left divided color form image data F-D1 is shown. As shown in FIG. 10, the binarizing unit 12 and the correcting unit 13 are composed of four threads, threads 1, 2, 3, and 4, and each of the threads 1, 2, 3, and 4 is an image dividing unit. 11 is performed on the divided color form image data F-D1, F-D2, F-D3, and F-D4 divided by 11. By performing multi-thread processing with four threads 1, 2, 3, and 4, binarization processing and correction processing of divided color form image data F-D1, F-D2, F-D3, and F-D4 can be performed. Performed in parallel, the speed of image processing can be improved, and binarization quality can be improved.

  The image dividing unit 11 outputs divided color form image data F-D1, F-D2, F-D3, and F-D4 divided into each of four threads of threads 1, 2, 3, and 4. The thread 1 of the binarizing unit 12 receives the upper left divided color form image data F-D1 divided by the image dividing unit 11, performs binarization, and generates divided form image data BF-D1 (step S1). Sa3-1). When the divided form image data BF-D1 is generated, the thread 1 of the correction unit 13 performs correction for rectangle detection and outputs corrected corrected divided form image data BF-D1-C (step Sa4-1). ).

  The process of detecting a rectangle by the rectangle detection unit 15 is a process of detecting a rectangular shape or a substantially rectangular shape as a rectangle. In order to detect a rectangle, it is necessary that there is no defect in the lines constituting the rectangle. It becomes. On the other hand, the binarization by the binarization unit 12 needs to perform binarization so that the character line remains thin and clear so that the character detection unit 24 can detect the character. In addition, a defect may occur in a line constituting the rectangle. Therefore, the correction process by the correction unit 13 is an image process in which a line is expanded to fill a defect generated by the binarization process. 11 to 17 are diagrams illustrating correction processing by the correction unit 13. As the expansion process performed by the correction unit 13, any one of the expansion process A, the expansion process B, and the expansion process C described below is selected in advance.

(Expansion treatment A)
The expansion process A is a general algorithm for expanding a line. In the expansion process A, when the correction unit 13 selects pixels one by one and, for example, selects “x” at the center as the target pixel as shown in FIG. If there is any black pixel, the target pixel is replaced with black. Specifically, when the target pixel is a white pixel, the correction unit 13 determines whether or not the pixel is expanded, that is, converted into black. For this purpose, first, it is determined whether or not there is black in the pixels around the pixel of interest, that is, the pixels marked with “◯” shown in FIG. In the example of FIG. 11, since the lower right pixel is black, the correction unit 13 replaces the target pixel with black as shown in FIG. Thereby, expansion is performed. The correction unit 13 performs this process on all the pixels to expand the lines included in the binarized divided form image data. In the examples of FIGS. 11 and 12, since the designated peripheral range is 1, the periphery of the target pixel is 8 neighboring pixels. However, the designated peripheral range is set to 2 or more to increase the number of peripheral pixels, and more. In this case, it may be inflated.

(Expansion treatment B)
The expansion process B is a process of replacing the target pixel with black based on the ratio of black pixels existing in the designated peripheral range of the target pixel. As described in the expansion process A above, if the designated peripheral range is 1, the designated peripheral range is eight neighboring pixels indicated by “◯”. Assume that the ratio predetermined in the correction unit 13 is 25%, and the correction unit 13 replaces the target pixel with black when the ratio is 25% or more. In the case of FIG. 11, there is one black pixel around. Since 25% of the eight is two, in the case of FIG. 11, the correction unit 13 does not replace the target pixel with black. On the other hand, in the case of FIG. 13, there are two black pixels in the vicinity. Therefore, the correction unit 13 replaces the target pixel with black as shown in FIG. The correction unit 13 can expand the lines included in the binarized divided form image data by performing this process for all the pixels. The designated peripheral range is not limited to 1, and the designated peripheral range may be set to 2 or more to increase the number of peripheral pixels.

(Expansion treatment C)
The expansion process C is an expansion process that is more specialized for rectangular correction by adding the weight of the black pixel distribution to the expansion determination element in addition to the method of the expansion process B described above. As illustrated in FIG. 15, when the center pixel indicated by “x” is selected as the target pixel, the correction unit 13 counts the number of black pixels existing in the left and right designated range pixels. When the designated range is 2, the pixel indicated by “◯” is the left and right designated range pixel. When the black pixel is counted to the left of the target pixel, the correction unit 13 increments the weight by “−1”, and when the black pixel exists to the right, the correction unit 13 increments the weight by “+1”. Note that the initial value of the weight is 0. In the case of FIG. 15, there are one black pixel on the left and two black pixels on the right, so the number of black pixels is three. In the expansion process B, it is determined whether or not to replace the pixel of interest with black based on the ratio of black pixels in the designated peripheral range. On the other hand, in the expansion process C, a predetermined threshold value is used. Here, for example, the threshold value is 2. Since the number 3 of black pixels corresponds to a case where the threshold value is 2 or more, the correction unit 13 determines to replace the target pixel with black. Next, the correction unit 13 calculates the weight. In the example of FIG. 15, since there are one black pixel on the left and two black pixels on the right, the correction unit 13 calculates the weight as −1 + 1 + 1 = + 1. When the weight is +1, the pixel is shifted to the right by one pixel from the target pixel, and the pixel indicated by the arrow in FIG. Then, the pixels on both sides of the starting pixel are replaced with black. When replaced, the result is as shown in FIG. 17, and as a result, the pixel of interest “x” that was white is replaced with black. The correction unit 13 performs the above-described process also on the pixels in the upper and lower specified range of the target pixel, and performs the process on the pixels in the left and right upper and lower specified range for all the pixels.

  Returning to FIG. 10, in the threads 2, 3, and 4, the binarization unit 12 binarizes the divided color form image data F-D2, F-D3, and F-D4 as in step Sa3-1. Thus, the binarization processing by the binarization unit 12 is completed (step Sa3). Further, in the threads 2, 3, and 4, similarly to step Sa4-1, a correction process for expanding the above-described line by the correction unit 13 is performed, thereby completing the correction process by the correction unit 13 (step Sa4). ).

Threads 1, 2, 3, and 4 of the binarizing unit 12 and the correcting unit 13 are binarized divided form image data BF-D1, BF-D2, BF-D3, and BF-D4, respectively, which are not corrected. Then, two types of image data of corrected binary document image data BF-D1-C, BF-D2-C, BF-D3-C, and BF-D4-C are output. At this time, a delay occurs with respect to the time required for the correction processing by the correction unit 13, so that the processing is synchronized, and these two types of image data are transmitted to the image synthesis unit 14 and the binarization unit 12 at the same timing. The correction unit 13 outputs. When the divided form image data BF-D1, BF-D2, BF-D3, and BF-D4 are output from the threads 1, 2, 3, and 4, the image composition unit 14 synthesizes them to detect keywords. The form image data BF is output, and the process proceeds to the connection point B (step Sa20).
In addition, the image composition unit 14 outputs corrected divided form image data BF-D1-C, BF-D2-C, BF-D3-C, and BF-D4-C from the threads 1, 2, 3, and 4. Then, these are combined and the corrected form image data BF-C for rectangle detection is output (step Sa5).

  The rectangle detection unit 15 detects a rectangle included in the corrected form image data BF-C by image processing, and for each detected rectangle, obtains the upper left X and Y coordinate information and the lower right X and Y coordinate information. To detect. The rectangle detection unit 15 generates a rectangle ID that can uniquely identify the rectangle, and associates the generated rectangle ID with the detected upper left X and Y coordinate information and the lower right X and Y coordinate information. Is output to the rectangle determination unit 16 (step Sa6). The rectangle determination unit 16 outputs the upper left X and Y coordinate information, the lower right X and Y coordinate information output from the rectangle detection unit 15, and information on the size of the character frame rectangle determined in advance inside Are compared to determine whether the rectangle output by the rectangle detector 15 is a character frame rectangle (step Sa7). Note that the rectangle detected by the rectangle detection unit 15 is not necessarily a rectangle as described above, and includes a substantially rectangular shape. Therefore, the size comparison is not based on whether or not they match, but the size of a predetermined character frame rectangle. On the basis of the thickness, it is determined whether it is within a certain range of whether it is not too large or not too small. Therefore, in addition to the condition of the size of the character frame rectangle, the condition of the range is also determined in advance inside the rectangle determination unit 16. If the rectangle determination unit 16 determines that the character frame is a rectangle, the rectangle ID, the upper left X and Y coordinate information, the lower right X and Y coordinate information, and the information indicating “character frame” are obtained. Corresponding data is written in the rectangular information storage unit 50. On the other hand, if the rectangle determination unit 16 determines that the character frame is not a rectangle, the rectangle ID, the upper left X and Y coordinate information, the lower right X and Y coordinate information, and the information indicating “other” are obtained. Corresponding data are written in the rectangular information storage unit 50 (step Sa8).

  When the rectangle determination unit 16 finishes writing the rectangle information to the rectangle information storage unit 50, the field detection process by the field detection unit 17 is started through the connection point A as shown in FIG. 8 (step Sa9). As a configuration for notifying the field detection unit 17 of the start of processing, for example, the rectangle determination unit 16 that has received notification from the rectangle detection unit 15 of completion of rectangle detection processing for one correction form image data, A configuration is employed in which when the last rectangular information is written in the rectangular information storage unit 50, a notification of processing completion is output as a processing start notification to the field detection unit 17.

(Field detection processing)
18 and 19 are flowcharts showing the field detection processing by the field detection unit 17. The field detection unit 17 refers to a rectangle in which the item “rectangle type” is “character frame” and the item “processing state” is not “completed” in the rectangle information storage unit 50, Among the frame rectangles, the character frame rectangle having the smallest X-axis coordinate value at the upper left vertex is selected as the reference character frame rectangle (step Sb1). The field detection unit 17 refers to the coordinate value of the upper left vertex of the selected reference character frame rectangle on the Y axis, and extracts another character frame rectangle whose coordinate value is approximately equal to the coordinate value of the upper left vertex of the Y axis. (Step Sb2). FIG. 20 is a diagram illustrating the process of step Sb2. When the character frame rectangle R1 is selected as the reference character frame rectangle, the field detection unit 17 extracts character frame rectangles R3 and R5 whose coordinate values on the upper left Y axis are the same or substantially equal as other character frame rectangles, The frame rectangles R2 and R4 are not extracted. In this process, the character frames that make up the field are horizontally written, and the top side is at the same or almost the same position on the Y axis. It is an object. In step Sb2, the field detection unit 17 determines whether another character frame rectangle has been extracted. If not, the field detection unit 17 proceeds to step Sb14 through the connection point C. On the other hand, if the field detection unit 17 has extracted another character frame rectangle, the process proceeds to step Sb4.

  When the field detection unit 17 extracts another character frame rectangle, the coordinate value of the upper left X axis of the rectangle selected as the reference character frame rectangle, the coordinate value of the upper left X axis of the extracted other character frame rectangle, Other character frame rectangles whose distance between the adjacent character frame rectangles is equal to or smaller than a predetermined value are selected as adjacent character frame rectangles. At this time, if there are a plurality of other character frame rectangles that satisfy the condition equal to or less than the predetermined value, the other character frame rectangle that is closest to the reference character frame rectangle is selected as the adjacent character frame rectangle (step Sb4). The field detection unit 17 determines whether there is an adjacent character frame rectangle in addition to the selected adjacent character frame rectangle (step Sb5). If it is determined that there is another adjacent character frame rectangle in the selected adjacent character frame rectangle, the field detection unit 17 replaces the selected adjacent character frame rectangle with the reference character frame rectangle and repeats the processing from step Sb4 (step Sb6). .

  FIG. 21 is a diagram illustrating the processing of steps Sb4 to Sb6. In the figure, for example, the character frame rectangles R1 to R4 have the same size, the widths of the character frame rectangles R1 to R4 in the X-axis direction are all 8 pixels, and the predetermined value range is 12 pixels. Suppose that First, the difference between the upper left X-axis coordinate value of the character frame rectangle R1 selected as the reference character frame rectangle and the upper left X-axis character frame rectangle of the character frame rectangles R2 to R4 is calculated. Differences between the reference character frame rectangle R1 and the character frame rectangles R2, R3, and R4 are (8 + 2 = 10) pixels, (10 + 8 + 3 = 21) pixels, and (21 + 8 + 10 = 39) pixels, respectively, and exist within a three-pixel range. What is to be done is the character frame rectangle R2. The field detection unit 17 selects the character frame rectangle R2 as the adjacent character frame rectangle, and further repeats the processing after step Sb4 with the character frame rectangle R2 as the reference character frame rectangle. Since the character frame rectangle R3 also satisfies the condition of step Sb4 in relation to the character frame rectangle R2, the field detection unit 17 selects it as the adjacent character frame rectangle. Regarding the character frame rectangle R4, the field detection unit 17 does not select the adjacent character frame rectangle because the condition of step Sb4 is not satisfied in relation to the character frame rectangle R3. Accordingly, in the case of FIG. 21, the field detection unit 17 selects the character frame rectangles R2 and R3 as the adjacent character frame rectangles.

  On the other hand, if it is determined in step Sb5 that there are no more adjacent character frame rectangles in the selected adjacent character frame rectangle, the field detection unit 17 determines the number of character frame rectangles selected as the adjacent character frame rectangles. If the number of selected adjacent character frame rectangles is 0, the field detection unit 17 proceeds to step Sb14 through the connection point C. If the number of selected adjacent character frame rectangles is 1, the field detection unit 17 proceeds to step Sb10 through the connection point E. When the number of selected adjacent character frame rectangles is plural, the field detection unit 17 proceeds to step Sb8 through the connection point D (step Sb7). In step Sb8 shown in FIG. 19, the field detection unit 17 resets the character frame rectangle initially selected as the reference character frame rectangle in step Sb1 as the reference character frame rectangle (step Sb8).

  The field detection unit 17 extracts character frame rectangles arranged at approximately equal intervals from the selected plurality of character frame rectangles with reference to the reference character frame rectangle (step Sb9). FIG. 22 is a diagram illustrating the processing in step Sb9. For example, when the character frame rectangle R1 is selected as the reference character frame rectangle, the field detection unit 17 sets the character frames arranged at equal intervals of 2 pixels. Rectangles R2 and R3 are extracted. On the other hand, since the distance between the character frame rectangle R3 and the character frame rectangle R4 is 5 pixels and is not equally spaced, the field detection unit 17 does not extract the character frame rectangle R4.

  The field detection unit 17 selects an adjacent character frame rectangle whose X-axis coordinate value at the upper left is the second smallest after the reference character frame rectangle based on the reference character frame rectangle, that is, a character frame rectangle adjacent to the reference character frame rectangle. Then, the field detection unit 17 determines whether or not the height of the selected adjacent character frame rectangle, that is, the difference between the upper left Y axis coordinate value and the lower left Y axis coordinate value is substantially equal to the height of the reference character frame rectangle. Is determined (step Sb10). If the field detection unit 17 determines that the height of the reference character frame rectangle is substantially equal to the height of the selected adjacent character frame rectangle, the process proceeds to step Sb11. On the other hand, if the field detection unit 17 determines that the height of the reference character frame rectangle is not substantially equal to the height of the selected adjacent character frame rectangle, the process proceeds to step Sb14.

  If the field detection unit 17 determines that the height of the reference character frame rectangle is substantially equal to the height of the selected adjacent character frame rectangle, the field detection unit 17 selects the selected adjacent character frame rectangle as a character frame rectangle constituting the field (step Sb11). The field detection unit 17 determines whether or not there is an adjacent character frame rectangle in the adjacent character frame rectangle selected as the character frame rectangle constituting the field (step Sb12). If it is determined that there is an adjacent character frame rectangle in the adjacent character frame rectangle selected as the character frame rectangle constituting the field, the field detection unit 17 uses the adjacent character frame rectangle selected as the character frame rectangle constituting the field as the reference character. It replaces with a frame rectangle and repeats the process from step Sb10 (step Sb13). FIG. 23 is a diagram illustrating the processing of steps Sb10 to Sb13. The field detection unit 17 forms the field of the character frame rectangle R2 because the character frame rectangle R1 and the character frame rectangle R2 are substantially equal in height. Select as a character frame rectangle. On the other hand, the field detection unit 17 does not select the character frame rectangle R3 as the character frame rectangle constituting the field because the character frame rectangle R2 and the character frame rectangle R3 have different heights.

  In the classification processing of the character frame rectangles so far, the classification including only one character frame rectangle selected as the reference character frame rectangle in step Sb1, or the character frame rectangle selected as the reference character frame rectangle and one in step Sb1 Alternatively, classification into a section including adjacent character frame rectangles forming a plurality of fields is completed. The field detection unit 17 generates a field ID, writes the rectangle ID of the character frame rectangle included in the classified section as one field in the field information storage unit 51 in association with the generated field ID (step Sb14). The field detection unit 17 writes the “completed” item in the “processing state” item of the rectangle information storage unit 50 corresponding to the rectangle IDs of all the rectangles constituting the field (step Sb15). The field detection unit 17 refers to the rectangular information storage unit 50 and determines whether or not there is a rectangle in which “rectangle type” is “character frame” and “processing” is not written in the item “processing state”. Determination is made (step Sb16). If the “rectangular type” is “character frame” and there is a rectangle in which “completed” is not written in the item of “processing state”, the field detection unit 17 repeats the processing from step Sb1 to obtain “rectangular type”. "Is a" character frame ", and if there is no character frame rectangle in which" completed "is not written in the item of" processing state ", the field detection process ends.

(Erase rectangles and ruled lines)
Returning to the connection point B in FIG. 8, the rectangle erasure unit 20 erases the rectangle from the form image data BF using the rectangle information stored in the rectangle information storage unit 50 (step Sa21). As a configuration for notifying the rectangle erasing unit 20 of the start of processing, for example, the rectangle determination unit 16 that has received a notification of completion of the rectangle detection processing for one correction form image data from the rectangle detection unit 15, A configuration is adopted in which, when the last rectangular information is written in the rectangular information storage unit 50, a notification of the completion of the processing is output as a processing start notification to the rectangular erasing unit 20. The ruled line erasure corresponding image dividing unit 21 receives the form image data BF-R from which the rectangle output from the rectangular erasing unit 20 is erased, and divides the image for ruled line erasure (step Sa22).

  Here, with reference to FIGS. 24 and 25, division of an image for erasing ruled lines will be described. As image processing for erasing ruled lines, for example, image processing in which a portion where black pixels of 30 pixels or more are continuous is determined as a ruled line and replaced with white pixels to erase the ruled line is applied. Note that the ruled line is not limited to a straight line but includes a curved line, and whether or not it is a ruled line may be determined based on the density of black pixels instead of the number of continuous black pixels. The image division for ruled line erasing is a slightly larger and slightly larger boundary so that the ruled line judgment process is performed well in the boundary area in consideration of the judgment process of the ruled line erasing unit 22 as to whether or not the ruled line is a ruled line. This is a process of dividing into areas. Specifically, as shown in FIG. 25, a line segment connecting the midpoints of the upper and lower sides of the form image data BF-R, ie, a line segment connecting DL1 and a midpoint of the left and right sides, ie, DL2 line. The image is divided so as to be a slightly larger area (1) obtained by adding the extension area EX1 and the extension area EX2 than the area divided by minutes.

  For example, when determining a ruled line with 30 consecutive pixels, there is only one pixel on the left side of the line segment DL1, and the remaining 29 pixels are straight lines perpendicular to the line segment D1 and on the right side of the line segment DL1. However, it is necessary to erase one pixel on the left side of the line segment DL1 as a ruled line. Therefore, by setting the width of the extended area EX1 to be expanded to 29 pixels, one pixel on the left side of the line segment DL1 becomes the leftmost pixel of the 30 pixels continuously present in the ruled line erasing process by the ruled line erasing unit 22. Will be erased. Similarly, the vertical extension area EX2 has 29 pixels. The sizes of the extended areas EX1 and EX2 depend on the image processing configuration of how many pixels are determined as ruled lines, and are appropriately changed depending on the resolution of the image read by the image reading unit 10. is there.

  When dividing as shown in FIG. 25, the area (1), (2), (3), (4) overlaps each other in the boundary area as shown in FIG. Will exist. In FIG. 24, the area (1) and the area (4) are indicated by bold lines, and the areas (2) and (3) are indicated by broken lines. When the image is synthesized by the ruled line erasure corresponding image synthesizing unit 23, for the area delimited by the line segment DL1 and the line segment DL2, that is, the area (1), after the ruled line erase, Synthesize the deleted area.

  As shown in FIG. 26, the ruled line erasure corresponding image dividing unit 21 divides the form image data BF-R from which the rectangle has been erased into four areas, and each of them is converted into threads 1, 2,. Output to 3 and 4. The process in the thread 1 will be described. The divided form image data BF-RD1W (corresponding to the area (1) described above) is erased by image processing by the thread 1 of the ruled line erasing unit 22 (step Sa23). -1). The thread 1 of the ruled line erasing unit 22 outputs the divided form image data BF-RLD1W from which the ruled lines have been erased. The same processing as that of the thread 1 is performed in the threads 2, 3, 4, whereby the ruled line erasing process is completed, and the divided form image data BF− in which each of the threads 1, 2, 3, 4 has erased the ruled line RLD2W, BF-RLD3W, and BF-RLD4W are output (step Sa23). By performing multi-thread processing with four threads 1, 2, 3, and 4, ruled line erasure processing is performed in parallel on the divided form image data BF-RD1W, BF-RD2W, BF-RD3W, and BF-RD4W. It is possible to reduce the processing load of the ruled line erasing process with a large amount of calculation and improve the image processing speed.

  The ruled line erasure corresponding image composition unit 23 extracts an image of an image area that does not include the extended areas EX1 and EX2 shown in FIG. 25 from the divided form image data BF-RLD1W. In other words, the ruled line erasure corresponding image composition unit 23 acquires an image excluding the extended areas EX1 and EX2 for the divided form image data BF-RLD1W. Similarly, the ruled line erasure corresponding image composition unit 23 extracts image areas that do not include extended areas corresponding to the extended areas EX1 and EX2 in the divided form image data BF-RLD2W, BF-RLD3W, and BF-RLD4W. To do. The ruled line erasure corresponding image composition unit 23 synthesizes the extracted divided form image data BF-RLD1W, BF-RLD2W, BF-RLD3W, and BF-RLD4W, and outputs the form image data BF-RLW from which the ruled lines and the rectangle are erased. (Step Sa24).

(Character detection)
The character detection unit 24 detects characters one by one by image processing from the form image data BF-RLW from which the ruled lines and rectangles have been deleted, and converts the detected characters into character information such as a character code. The character detection unit 24 detects the coordinate information of the detected character, that is, the upper left X and Y coordinates and the lower right X and Y coordinates. The character detection unit 24 generates a character ID every time a character is detected, and stores the generated character ID, character information such as the character code of the detected character, and coordinate information of the detected character in the character information storage unit 52. Writing is performed (step Sa25).

(Keyword detection)
The keyword detection unit 25 detects a keyword from the detected characters when all the character detection processing by the character detection unit 24 is completed in one form image data. As a configuration for notifying the keyword detection unit 25 of the start of processing, for example, when the character detection unit 24 completes the detection processing of all characters in one form image data, the process completion notification is sent to the keyword. A configuration in which the detection unit 25 outputs a process start notification is applied.

  In the keyword detection by the keyword detection unit 25, first, all possible keywords are detected based on information such as the positional relationship, size, and spacing between characters stored in the character information storage unit 52. I will do it. From the detected keywords, a keyword that completely or partially matches the character string or character stored in the “keyword” item of the dictionary attribute information storage unit 53 is detected (step Sa26). Here, when the keyword detected by the keyword detection unit 25 is, for example, “transfer fee”, the keyword detection unit 25 completely or partially matches the item “keyword” in the dictionary attribute information storage unit 53. Determine whether a keyword exists. Here, since the “transfer fee” that completely matches the detected keyword exists in the “keyword” item, the keyword detection unit 25 detects “transfer fee” as a keyword. When the keyword detection unit 25 detects a keyword, the keyword detection unit 25 reads attribute information corresponding to the detected keyword from the dictionary attribute information storage unit 53. In the example of FIG. 5, since “numeric value” is stored as attribute information in the keyword “transfer fee”, “numeric value” is read as attribute information (step Sa27).

  For example, it is assumed that the keyword detection unit 25 detects a keyword such as “request date” from characters stored in the character information storage unit 52. In this case, the keyword “request date” partially matches both “request date” and “date” stored in the “keyword” item of the dictionary attribute information storage unit 53. Therefore, the keyword detection unit 25 detects “request date” as a keyword because there is a partial match, but it cannot correctly read “attribute information” for this. For this reason, when there are a plurality of partial matches, it is determined that the match is the largest number of characters, and attribute information corresponding to the match is read out.

The keyword detection unit 25 detects the coordinate information of the keyword circumscribed rectangle. FIG. 27 is a diagram showing a keyword circumscribing rectangle KCR. For example, when the keyword “name” is detected, the keyword circumscribing rectangle is the minimum and maximum coordinates of the keyword “name” on the X axis and the Y axis. A rectangle surrounded by the minimum and maximum coordinates. Therefore, in the case of the keyword “name”, the keyword detection unit 25 stores the upper left X and Y coordinate information of “M” and “Name” stored in the character information storage unit 52, the lower right X, from Y coordinate information for the Y axis, to select the minimum and maximum coordinate information as y ₁ and y _2, respectively. Also, the keyword detector 25 for X-axis, the coordinate information of the upper left X-axis of "Mr." arranged on the left, and the minimum coordinate values x _1, it is located to the right of the "Name" the coordinate information of the X-axis of the lower right is selected as the maximum coordinate value x _2. As a result, the keyword detection unit 25 uses the upper-left X and Y coordinate information (x ₁ , y ₁ ) and the lower right X and Y coordinate information (x ₂ , y ₂₎ detecting the (step SA28). Each time a new keyword is detected, the keyword detection unit 25 generates a keyword ID. The generated keyword ID, the detected keyword, attribute information corresponding to the keyword, and X and Y at the upper left of the keyword circumscribed rectangle The coordinate information and the lower right X, Y coordinate information are associated and written to the form definition information storage unit 54 (step Sa29).

The keyword detection unit 25 outputs the keyword ID, the upper left X and Y coordinate information, and the lower right X and Y coordinate information to the keyword corresponding rectangle detection unit 26. The keyword-corresponding rectangle detection unit 26 receives the upper left X and Y coordinate information and the lower right X and Y coordinate information from the keyword detection unit 25, and has an appropriate size surrounding the keyword circumscribed rectangle defined by the two coordinates. Is detected from the rectangles whose “rectangle type” stored in the rectangle information storage unit 50 is “other”. An appropriately sized rectangle means, for example, a rectangle that is not too large as a rectangle surrounding the keyword. Specifically, as shown in FIG. 28, the keyword-corresponding rectangle detection unit 26 stores therein a predetermined threshold value in the X-axis direction and a threshold value in the Y-axis direction, and an upper left corner of the keyword circumscribed rectangle KCR. X, Y coordinates (x ₁ , y ₁ ), lower right X, Y coordinates (x ₂ , y ₂ ), upper left X, Y coordinates (X ₁ , Y ₁ ), and lower right X, rectangle KR Based on the Y coordinate (X ₂ , Y ₂ ), x ₁ −X ₁ , X ₂ −x ₂ , y ₁ −Y ₁ , Y ₂ −y ₂ are calculated. As described above, the coordinate system has the upper left as the origin, the right direction for the X-axis and the downward direction for the Y-axis, and the keyword “name” is surrounded by the rectangle KR. The values obtained by the above four formulas are all positive values. In addition, the keyword-corresponding rectangle detection unit 26 determines that the difference of x ₁ −X _{1 and} the difference of X ₂ −x ₂ are within the threshold in the X-axis direction, and the difference of y ₁ −Y ₁ and Y ₂ If each of the differences in −y ₂ is within the threshold value in the Y-axis direction, it is determined that the rectangle is an appropriately sized rectangle surrounding the keyword circumscribed rectangle. When the keyword-corresponding rectangle detection unit 26 detects a rectangle surrounding the keyword circumscribing rectangle by this determination process, the keyword-corresponding rectangle detection unit 26 reads the rectangle ID of the detected rectangle from the rectangle information storage unit 50, and associates the keyword ID with the read rectangle ID. At the same time, it is written in the item “corresponding rectangle” in the form definition information storage 54. When the rectangle surrounding the keyword cannot be detected, the keyword corresponding rectangle detection unit 26 writes “none” in the item “corresponding rectangle” (step Sa30).

(Field search process)
The field search unit 27 detects a field printed on the form corresponding to the keyword written in the form definition information storage unit 54 from the field information storage unit 51, and performs field search processing for associating the keyword with the field. This is performed (step Sa31). At this time, since the field detection process in step Sa9 by the field detection unit 17 needs to be completed, if the field detection process is not completed, the field search unit 27 waits until completion and synchronizes the processes. Do.

  FIG. 29 is a flowchart showing the flow of the field search process. The field search unit 27 selects a record in which the item “corresponding field” in the form definition information storage unit 54 is a blank column indicated by “−” (step Sc1). The field search unit 27 determines whether or not a rectangle ID is stored in the “corresponding rectangle” item of the selected record (step Sc2). When the rectangle ID is not stored in the item “corresponding rectangle”, that is, “none”, the field search unit 27 displays “X and Y coordinates on upper left” and “lower right” in the form definition information storage unit 54. The coordinate information of “X, Y coordinates”, that is, the coordinate information of the keyword circumscribed rectangle is read out as reference rectangle information (step Sc3). On the other hand, if the field search unit 27 determines that the rectangle ID is stored in the item “corresponding rectangle”, the field search unit 27 reads the rectangle ID and reads “X, The coordinate information of “Y coordinate” and “X, Y coordinates in the lower right” is read out and used as reference rectangle information (step Sc4). Thereby, the field search unit 27 displays the coordinate information of the rectangle KR1 surrounding the keyword as shown in FIG. 30 when the rectangle ID exists, and the keyword circumscribed rectangle KCR1 as shown in FIG. 31 when there is no rectangle ID. Get coordinate information of.

  The field search unit 27 determines whether the shape of the reference rectangle is vertically long or horizontally long based on the reference rectangle information. Longitudinal indicates that the keyword is vertically written, and horizontally long indicates that the keyword is horizontally written (step Sc5). In addition, when the content of the keyword is one character and the lengths of the vertical and horizontal sides are the same, either vertical or horizontal may be selected.

  When the field search unit 27 determines that the shape of the reference rectangle is vertically long, as shown in FIG. 30, in the horizontally written form, the corresponding field tends to exist in the right direction. Accordingly, when the reference rectangle is vertically long, the rectangle KR1 is set as the reference rectangle, and the following three parameters are set (step Sc6). The three parameters are: 1) reference vertex: upper right coordinate of the reference rectangle, 2) height: height of the reference rectangle ± predetermined error, 3) width: standard search range in the X-axis direction ± error . A field search range VSA1 as shown in FIG. 32 is defined by these three parameters. Here, the reference vertex 1) is a reference coordinate that determines the coordinate position of the field search range VSA1. The height of 2) is a value obtained by adding the height of the reference rectangle to the reference vertex in the positive direction of the Y axis and further adding a predetermined error in the positive and negative directions of the Y axis. The horizontal width of 3) is a value obtained by adding a standard search range in a predetermined X-axis direction to the reference vertex in the positive direction in the X-axis direction and further adding a predetermined error in the positive and negative directions of the X-axis. . The standard search range in the X-axis direction in the parameter 3) is a range in which a field that is determined empirically exists and is a value that varies depending on the resolution of the image. Similarly, the error is a value determined empirically, and is a value that varies depending on the resolution of the image. The value of the error in height and width may be the same or different. Also good.

  On the other hand, when the field search unit 27 determines that the shape of the reference rectangle is horizontally long, as shown in FIG. 31, in the horizontally written form, the corresponding field tends to exist in the right, lower left, or downward direction. is there. Therefore, if the reference rectangle is horizontally long, the keyword circumscribed rectangle KCR1 is set as the reference rectangle, and three field search ranges HSA1, HSB1, and HSC1 are set as shown in FIG. 33 (step Sc7). The field search range HSA1 existing to the right of the reference rectangle is 1) reference vertex: upper right coordinates of the reference rectangle, 2) height: reference rectangle height ± error, 3) width: standard search range in the X-axis direction. It is set by three parameters of ± error. The field search range HSB1 existing at the lower left of the reference rectangle is 1) reference vertex: lower left coordinates of the reference rectangle, 2) height: standard search range ± error in the Y-axis direction, 3) width: width of the reference rectangle / 2 is set by three parameters. The field search range HSC1 existing in the downward direction of the reference rectangle is as follows: 1) Reference vertex: lower left coordinates of the reference rectangle, 2) Height: standard search range ± error in the Y-axis direction, 3) Width: width of the reference rectangle These three parameters are set. Here, the standard search range in the X-axis direction and the standard search range in the Y-axis direction are ranges in which fields that are determined empirically exist, and are values that vary depending on the resolution of the image. The standard search range in the Y-axis direction may be the same value or a different value for the field search ranges HSB1 and HSC1. Similarly, the error is a value determined empirically and is a value that varies depending on the resolution of the image. The field search ranges HSA1, HSB1, and HSC1 may be the same value or different values. It may be. Further, the height and width error values of the field search range HSA1 may be the same value or different values.

  The field search unit 27 determines whether a field exists in the set field search range (step Sc8). Specifically, the field search unit 27 refers to the coordinate values of the items “upper left X, Y coordinates” and “lower right X, Y coordinates” of each field stored in the field information storage unit 51. The determination is made based on whether or not the region surrounded by these two coordinate values overlaps the set field search range. For example, in the case of the field search range VSA1 in FIG. 32, since the left portion of the field FE1 overlaps with the field search range VSA1 as shown in FIG. 34, the field search unit 27 determines that a field exists. In the case of the field search ranges HSA1, HSB1, and HSC1 in FIG. 33, the field FE2 overlaps the field search range HSB1 on the left side and the field search range HSC1 in the center, and the field search range HSA1. And the upper part of the right side overlaps. Therefore, the field search unit 27 determines that a field exists.

  When the field search unit 27 determines that a field exists, the field search unit 27 reads the field ID of the field from the field information storage unit 51. Then, the field search unit 27 writes the read field ID in the “corresponding field” item of the form definition information storage unit 54 in association with the keyword ID of the keyword to be processed, and associates the keyword with the field (step) Sc9). When the field search unit 27 detects a plurality of fields for a certain keyword, the field search unit 27 detects the field having the shortest distance between the upper left vertex of the field and the reference vertex as the field for the keyword. On the other hand, if the field search unit 27 determines that the field does not exist, the field search unit 27 writes “processed” in the “corresponding field” item of the form definition information storage unit 54 in association with the keyword ID of the keyword to be processed ( Step Sc10).

  The field search unit 27 determines whether or not there is a record in which the item “corresponding field” in the form definition information storage unit 54 is “−” indicating a blank (Step Sc11). If there is a blank record, the field search unit 27 repeats the process from step Sc1. On the other hand, if there is no blank record, the field search unit 27 ends the field search process.

  With the configuration of the first embodiment, the image reading unit 10 optically reads a form to generate form image data, and the binarization unit 12 generates binarized form image data. . The correcting unit 13 corrects the form image data so as to fill in the defects generated by the binarization process, and generates corrected form image data. The rectangle detection unit 15 detects a rectangle from the corrected form image data, and the field detection unit 17 detects a field based on the detected rectangle. On the other hand, the character detection unit 24 detects characters from the form image data from which the rectangles and ruled lines have been deleted by the rectangle erasing unit 20 and the ruled line erasing unit 22, and the keyword detection unit 25 selects a keyword based on the detected characters. The attribute information associated with the keyword detected and stored in the dictionary attribute information storage unit 53 is read out. The field search unit 27 associates the field detected by the field detection unit 17 with the keyword detected by the keyword detection unit 25, and associates the keyword, the field, and the attribute information as the form definition information in the form definition information storage unit 54. To remember. As a result, from the form image data generated by optically reading the form, a field for entering handwritten characters and a keyword associated with attribute information indicating the type of characters to be entered in the field are extracted, and the relationship is extracted. A certain field can be associated with a keyword. Therefore, when a handwritten form is subsequently read, it is determined whether the character type of the handwritten character entered in the field is correct based on the definition information in which the field, the keyword, and the attribute information are associated with each other. Processing is possible, and if it is wrong, it is possible to improve the accuracy of information handwritten on the form by prompting the writer to correct it.

  In the first embodiment, the binarization is performed on the form image data binarized so as to form a thin line in order to increase the detection rate of characters and the binarized form image data. In order to fill in the defects generated by the processing, the line is expanded and corrected to generate two types of image data including corrected form image data for rectangle detection. With this configuration, it is possible to improve the accuracy of both the character detection rate and the rectangle detection rate. In general, when image data is corrected for rectangle detection, the ratio of black increases as a whole. In particular, there is a tendency for black areas to be corrected to black, such as “No.” and “Rei”. The characters of are easy to collapse. In this case, since it becomes difficult to recognize characters, the accuracy of matching between the detected character string and dictionary information is lowered. In order to avoid this, two binarization configurations are required to perform binarization for character detection such that the character line is thin and sharp and binarization for rectangle detection that thickens the line. On the other hand, in the first embodiment, in the binarization process, first, image data in which a line for character detection is thinned is generated, and thereafter, a defect caused by binarization is filled for rectangle detection. Therefore, the correction processing for expanding the line is performed. Therefore, it is a configuration that does not need to have a double binarization processing configuration. This configuration makes it possible to make corrections that prevent characters from being crushed, so that item names can be specified with higher accuracy.

  In the correction unit 13, correction that increases the detection rate of the rectangle, that is, correction by any of the expansion processes A, B, and C described above is performed. These expansion processes A, B, and C are processes for expanding a line. However, if the line is expanded too much, it will come into contact with a nearby rectangle and cannot be correctly detected as a rectangle. In some cases, a new rectangle that does not originally exist, that is, a noise rectangle, is generated in contact with characters or noise. Therefore, how to set parameters such as the designated peripheral range, the designated range, and the threshold value is important in order not to make contact with other rectangles and to generate a noise rectangle. Differences in effects due to differences in parameter settings will be described with reference to FIGS.

  FIG. 36 shows color image data optically read by the image reading unit 10, and FIG. 37 shows a case where a defect occurs in a continuous line segment forming a rectangle by the binarization processing by the binarization unit 12. FIG. In the drawing, a portion painted with diagonal lines indicates a black pixel after binarization, and a portion painted with black indicates a pixel that has been painted black by expansion by expansion processing. FIGS. 38 to 40 show the processing results when the expansion process B is increased in stages in the designated peripheral range 1 and the threshold values are increased in stages 1, 2, and 3 for the image data shown in FIG. It is a figure which shows the determination result about three parameters | indexes. Here, the three indicators are rectangle correction, rectangle contact, and noise rectangle. For rectangle correction, when a rectangle defect is filled, “◯” is indicated, and the rectangle defect remains. When it is, “x” is shown. Regarding the contact of rectangles, “o” is shown when two rectangles are not in contact, and “x” is shown when they are in contact. The noise rectangle indicates the presence or absence of a white area surrounded by the line segment generated by performing the dilation process. If there is such a white area, “○” is indicated. Show.

  In the designated peripheral range 1 shown in FIG. 38, in the case of the threshold value 1, there is no rectangular defect, and there is no white portion generated by the expansion process, so the indices of “rectangle correction” and “noise rectangle” are Although it is “◯”, since the adjacent rectangles are in contact with each other by being inflated, the index of the rectangular contact is “x”. In the designated peripheral range 1 shown in FIG. 39, in the case of the threshold value 2, the rectangular defect is not lost, and the adjacent rectangles are in contact with each other by being expanded, and are surrounded by line segments. A white area has newly occurred. Therefore, in FIG. 39, all of the three indicators “rectangular correction”, “rectangular contact”, and “noise rectangle” are “x”. In the designated peripheral range 1 shown in FIG. 40, in the case of the threshold 3, the rectangular defect is not lost, and the adjacent rectangles are in contact with each other by being expanded, so that “rectangular correction”, The index “rectangular contact” is “x”. On the other hand, since the white region surrounded by the line segment is not newly generated, the “noise rectangle” is “◯”.

FIG. 41 to FIG. 43 show the processing results when the expansion processing C is increased in stages in the designated range 2 and the threshold values 1, 2, and 3, respectively, and the determination results for the three indexes. FIG. In the case of the designated range 2 and the threshold value 1 shown in FIG. 40, the missing rectangle is not lost, and the adjacent rectangles are in contact with each other by being expanded, and the white region surrounded by the line segment is It has newly occurred. Therefore, in FIG. 41, all of the three indicators “rectangular correction”, “rectangular contact”, and “noise rectangle” are “x”. FIG. 42 shows the case of the designated range 2 and the threshold value 2, where the rectangular defect has disappeared, and the adjacent rectangles do not come into contact with each other by being expanded, and the white region surrounded by the line segment is It has not occurred anew. Therefore, in FIG. 42, all three indicators “rectangular correction”, “rectangular contact”, and “noise rectangle” are “◯”. FIG. 43 shows a case where the specified range is 2 and the threshold value is 2, and a rectangular defect is not lost. However, the adjacent rectangles do not come into contact with each other by being expanded, and a white region surrounded by a line segment is not newly generated. Therefore, in FIG. 43, the “rectangular correction” index is “x”, but the “rectangular contact” and “noise rectangle” indices are “◯”.
Therefore, it can be seen that in the example of the processing of FIGS. 38 to 43 for the form image data as shown in FIG. 37, the expansion processing C with the designated range 2 and the threshold 2 of FIG. 42 is the best expansion processing. Since this result also depends on the missing state of the rectangle to be processed, different results may be obtained if the missing state of the rectangle is different.

  In the first embodiment, the ruled line erasure corresponding image dividing unit 21 divides the image data into four parts in order to erase the ruled lines by the ruled line erasing unit 22 by thread processing. At this time, the area to be divided is divided into four so that the area is slightly wider than when the area is divided into four equal parts so that the ruled lines in the boundary area disappear. This effect is shown using FIG. 44 to FIG. In the example shown in FIGS. 44 to 46, in order to show ruled line erasing performance, the rectangle erasing unit 20 does not erase the rectangle, but erases all the ruled lines including the rectangle. FIG. 44 is a diagram illustrating the binarized form image data BF output from the image composition unit 14. In FIG. 44, a line segment DL1 and a line segment DL2 are line segments that divide the form image data BF into four equal parts. When ruled line processing is performed by the ruled line erasing unit 22 on the divided divided form image data, the image is synthesized, as shown in FIG. 45, boundary areas WA1, which are division boundaries by the line segment DL1 and the line segment DL2, as shown in FIG. The ruled line in the area surrounded by WA2 cannot be erased. On the other hand, when the ruled line erasure is performed by performing the division with the extension area described with reference to FIGS. 24 and 25, the ruled lines in the boundary areas WA1 and WA2 can be erased as shown in FIG. Recognize.

In the first embodiment described above, the number of divisions of the form image data in the image dividing unit 11 and the ruled line erasure corresponding image dividing unit 21 is four and the number of threads is four. However, the configuration of the present invention is as follows. It is not limited to the embodiment. The number of divisions may be two divisions or four or more divisions. The number of threads will be arranged according to the number of divisions. However, since the number of divisions and the size of the divisions affect the processing of binarization and ruled line erasing by the threads, division is performed in consideration of the degree of influence. The number and division size will be determined. Further, the form image data may not be divided and the processing by the thread may not be performed.
Also, the shape of the division is not limited to the shape divided into four by the line segments DL1 and DL2 as shown in FIG. 44, and may be four divisions with only vertical lines or four divisions with only horizontal lines. . Further, the image dividing unit 11 may divide the image data into four by providing an extended area, like the ruled line erasure corresponding image dividing unit 21. As a result, when the image data after the image division by the image dividing unit 11 is binarized, each image data has an overlapping area, so that the binarization quality of each image data is prevented from being extremely different. can do.

  In the first embodiment, the binarization unit 12 performs the binarization process after the image data is divided by the image division unit 11, but the configuration of the present invention is the same as that of the embodiment. Not limited. For example, after performing binarization processing on the color form image data before division, the image data may be divided and only the correction processing by the correction unit 13 may be performed by the thread.

  In the first embodiment, the correction unit 13 expands the black pixel portion by any of the expansion processes A, B, and C described above, and fills the rectangular defect. Without being limited to the processing, any method may be used as long as it is a method of filling a continuous line defect.

Further, in the first embodiment, the rectangle ID is generated by the rectangle detection unit 15, but may be generated by the rectangle determination unit 16.
In the first embodiment, the rectangle erasure unit 20 erases the rectangle detected by the rectangle detection unit 15 to improve the accuracy of erasing the rectangle. However, the rectangle is composed of lines. The rectangle erasing unit 20 may not be provided, and the ruled line erasing unit 22 may erase the rectangle together with the ruled line.

  In the first embodiment, the character detection unit 24 detects the coordinate data of the detected character, that is, the upper left X and Y coordinates and the lower right X and Y coordinates. Although the size is specified, the configuration of the present invention is not limited to the embodiment. For example, instead of detecting the upper left X and Y coordinates and the lower right X and Y coordinates, the character detection unit 24 detects the position of the detected character or the center of gravity and the character size such as the font size. Then, it may be written in the character information storage unit 52.

  In the first embodiment, the keyword detection unit 25 detects a keyword that completely matches or matches a character string or character stored in the “keyword” item of the dictionary attribute information storage unit 53. However, the configuration of the present invention is not limited to the embodiment. In addition, by referring to external dictionary information connected to the form definition device 1, a keyword that detects meaning as Japanese is detected, or a keyword is detected by analyzing syntax from Japanese grammar. Such a technique may be applied. In addition, since keywords are usually printed on forms with no wording, “transfer fees” etc. are detected as keywords, while sentences such as “Please enter in katakana” are not detected as keywords. There may be. In these cases, there is a case where a keyword that does not completely match or partially matches a character string or a character stored in the “keyword” item of the dictionary attribute information storage unit 53 may be detected. In the “attribute information” in the storage unit 54, “-” indicating a blank is written.

  Further, in the first embodiment, when the keyword detection unit 25 detects a keyword such as “request date”, it partially matches the character string or character of the item “keyword” in the dictionary attribute information storage unit 53. The attribute information associated with the character with the largest number of matching characters is read out, but the configuration of the present invention is not limited to this embodiment. For example, with respect to a thing with a polite prefix such as “request date”, only the exact match may be searched after removing the prefix. In addition, all character strings (for example, “request date” and “request date”) in which prefixes assumed in advance are added to the “keyword” and “attribute information” items in the dictionary attribute information storage unit 53. ) May be stored in advance together with the corresponding attribute information, and only those that completely match may be searched. Further, when it is desired to limit the keywords to be detected by the keyword detection unit 25, only a character string or a part matching the character stored in the “keyword” item of the dictionary attribute information storage unit 53 is detected as a keyword. Also good. For example, when the keyword detection unit 25 detects a character string “Please enter in katakana” as a keyword, the keyword partially matches “Katakana” in the “Keyword” item of the dictionary attribute information storage unit 53. , “Katakana”, which is a matching part, is detected as a keyword. In this case, the keyword detection unit 25 also detects the coordinate information about the “Katakana” portion for the coordinate information of the keyword circumscribed rectangle.

  In the configuration of the first embodiment described above, the field detection processing performed by the field detection unit 17 is based on the premise that the form is horizontally written, but the vertical / horizontal relationship is maintained even when the form is vertically written. It is possible to replace the field detection process.

  In the configuration of the first embodiment described above, the field search unit 27 uses the rectangle surrounding the keyword instead of the keyword circumscribing rectangle when the rectangle surrounding the keyword exists, and searches the field corresponding to the keyword. Detected. The purpose of this is to detect the field with a higher probability than using the circumscribed rectangle of the keyword because the field is likely to exist in a region extending the rectangle surrounding the keyword vertically or horizontally. Although it is a process, the structure of this invention is not restricted to the said embodiment. For example, all the processes by the field search unit 27 may be performed using the keyword circumscribed rectangle. In this case, the form definition device 1 does not need to include the keyword corresponding rectangle detection unit 26.

  Further, in the first embodiment, when a limitation is made such that one field is associated with one keyword, the field search unit 27 has already detected that the field ID of the field detected as corresponding to the keyword is the form definition information storage unit 54. Is stored in the “corresponding field” item, and if already associated with another keyword, “processed” is written in the “corresponding field” item of the keyword to be processed. Further, when the field search unit 27 detects a plurality of corresponding fields for one keyword, first, except for the fields associated with other keywords, the upper left of the field among the remaining fields The field having the shortest distance between the vertex and the reference vertex may be detected as the field corresponding to the keyword.

(Second Embodiment)
FIG. 47 is a block diagram showing the configuration of the form definition device 1a according to the second embodiment of the present invention. The same configurations as those in the first embodiment are denoted by the same reference numerals, and different configurations will be described below. In the second embodiment, a field search unit 27a and a reference information storage unit 55 are provided.
Depending on the form format, a plurality of keywords may correspond to one field. For example, two keywords “name” and “please enter in katakana” may be printed near the field for entering the name in katakana. At this time, by associating two or more keywords with one field, it is possible to specify characters to be entered in the field by a plurality of attribute information, and further improve the accuracy of the character type of the characters to be handwritten. be able to.

  In the form definition device 1a, the reference information storage unit 55 has a data configuration shown in FIG. 48, and includes items of “field ID”, “corresponding keyword”, and “attribute information”. When the field search unit 27a detects a field corresponding to the keyword by the field search process, the field search unit 27a writes the field ID of the detected field in the item “field ID”. In the “corresponding keyword” item, a keyword ID corresponding to the field ID is written by the field search unit 27a. In the item “attribute information”, the attribute information of the keyword corresponding to the keyword ID is written by the field search unit 27a.

  The field search unit 27a detects a field corresponding to the keyword in the field search process of the first embodiment shown in FIG. 29, and detects it in the item “corresponding field” in the form definition information storage unit 54 in step Sc9. Write the field ID of the field. At this time, the field search unit 27a reads the attribute information associated with the processing target keyword ID from the form definition information storage unit 54, associates it with the field ID of the detected field, The read attribute information is written in the reference information storage unit 55. In the reference information storage unit 55, when the record of the field ID already exists, the keyword ID and the attribute information are added to the record. For example, as shown in the example of field 2 in FIG. 48, when “keyword 4” is already written in “corresponding keyword” and “character” is written in “attribute information”, the field search unit When the field 27 is detected as a field corresponding to the keyword 5, the 27a adds “keyword 5” and “katakana character” to the record in the field 2 and writes them.

With the configuration of the second embodiment described above, by providing the reference information storage unit 55, it is possible to detect a plurality of keyword IDs associated with a field and attribute information corresponding to the keyword ID, It is possible to further improve the accuracy of the character type and the like of the handwritten characters.
Even if the configuration as in the second embodiment is not provided, it is associated with one field based on the information stored in the “corresponding field” item of the form definition information storage unit 54 in the first embodiment. The purpose of the second embodiment can also be achieved by searching for all the keyword and attribute information.

  The form definition information storage unit 54 in the first and second embodiments and the reference information storage unit 55 in the second embodiment read outside the form definition device 1, for example, a handwritten form, It may be provided on the device side that performs analysis and accumulation of handwritten characters. In that case, the keyword detection unit 25, the keyword-corresponding rectangle detection unit 26, and the field search units 27 and 27a write and read data to and from the form definition information storage unit 54 and the reference information storage unit 55 via a communication line or the like. It will be.

  You may make it implement | achieve the form definition apparatuses 1 and 1a in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be a program for realizing a part of the above-described functions, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. You may implement | achieve using programmable logic devices, such as FPGA (Field Programmable Gate Array).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 Form definition apparatus 10 Image reading part 11 Image division part 12 Binarization part 13 Correction part 14 Image composition part 15 Rectangle detection part 16 Rectangle determination part 17 Field detection part 20 Rectangle deletion part 21 Ruled line deletion corresponding image division part 22 Ruled line deletion Unit 23 Ruled Line Erasing Corresponding Image Composition Unit 24 Character Detection Unit 25 Keyword Detection Unit 26 Keyword Corresponding Rectangle Detection Unit 27 Field Search Unit 50 Rectangular Information Storage Unit 51 Field Information Storage Unit 52 Character Information Storage Unit 53 Dictionary Attribute Information Storage Unit 54 Form Definition Information storage unit

Claims (9)

An image reading unit that optically reads a form and generates image data of the form;
A binarization unit that binarizes the image data to generate form image data;
A correction unit that generates correction form image data by correcting the form image data to fill in the defects generated by the binarization;
A rectangle detector for detecting a rectangular or substantially rectangular shape as a rectangle from the corrected form image data;
A rectangle determination unit for determining whether or not the rectangle is a character frame rectangle;
A field detection unit that classifies the character frame rectangles based on the adjacent relationship of the character frame rectangles, and detects each of the classified sections as a field;
A character detection unit for detecting characters included in the form image data;
A keyword detection unit that detects, from the characters, a keyword that is preliminarily associated with the attributes of the characters entered in the field;
A field search unit that detects the field corresponding to the keyword based on a positional relationship between the keyword and the field, and writes the detected field in the form definition information storage unit in association with the keyword;
A form definition device comprising: A character string or a character to be detected as the keyword is predetermined,
The keyword detection unit
2. When detecting the keyword from the character, only the keyword that completely or partially matches the predetermined character string or character to be detected is detected. Form definition device. A dictionary attribute information storage unit that stores in advance attribute information indicating the attribute corresponding to the keyword;
The keyword detection unit
When the keyword is detected from the character, the attribute information corresponding to the keyword is read from the dictionary attribute information storage unit, and the read attribute information is associated with the keyword and written to the form definition information storage unit. The form definition device according to claim 1 or 2. The positional relationship between the keyword and the field is a tendency in a direction in which the field is predetermined when the keyword is vertically written, or a direction in which the field is predetermined when the keyword is horizontally written. Indicated by the trend,
The field search unit
The form definition device according to any one of claims 1 to 3, wherein, when the field satisfies the tendency, the field is detected as the field corresponding to the keyword. A keyword-corresponding rectangle detection unit that detects the rectangle surrounding the keyword from the rectangle that the rectangle determination unit determines is not the character frame rectangle;
The field search unit
When the rectangle surrounding the keyword exists, the positional relationship between the rectangle surrounding the keyword and the field is set as the positional relationship between the keyword and the field, and the field corresponding to the keyword is detected. The form definition device according to any one of claims 1 to 4. A ruled line erasable image dividing unit for dividing the form image data;
A ruled line erasing unit for erasing ruled lines from each of the divided form image data;
A ruled line erasure-compatible image composition unit that synthesizes the divided form image data after the ruled line erasure,
The ruled line erasure compatible image dividing unit
When dividing the form image data, there is an overlapping region at each boundary of the divided form image data so that the ruled lines existing at the division boundaries are erased by the ruled line erasing unit. Set up an extended area and divide it,
The ruled line erasure compatible image composition unit
Combining after deleting the extended area of each of the divided form image data,
The character detection unit
The form definition device according to any one of claims 1 to 5, wherein the character is detected from the form image data from which the ruled line is output which is output by the ruled line erasure corresponding image composition unit. The field search unit
The form definition device according to any one of claims 1 to 6, wherein a plurality of keywords are associated with the field. Optically read the form and generate image data of the form,
The image data is binarized to generate form image data,
A correction form image data is generated by correcting the form image data so as to fill a deficiency caused by the binarization,
A rectangular or substantially rectangular shape is detected as a rectangle from the corrected form image data,
Determining whether the rectangle is a character frame rectangle;
Based on the adjacent relationship of the character frame rectangles, classify the character frame rectangles, detect each of the classified division as a field,
Detecting characters included in the form image data,
Detecting a keyword associated with the attribute of the character entered in the field in advance from the character;
Detecting the field corresponding to the keyword based on a positional relationship between the keyword and the field;
A form definition method characterized by storing the detected field in association with the keyword. On the computer,
A procedure for optically reading a form and generating image data of the form,
A procedure for binarizing the image data to generate form image data;
A procedure for generating corrected form image data by correcting the form image data so as to fill in the defects generated by the binarization;
A procedure for detecting a rectangular or substantially rectangular shape as a rectangle from the corrected form image data,
A procedure for determining whether or not the rectangle is a character frame rectangle;
A procedure for classifying the character frame rectangle based on the adjacency relationship of the character frame rectangle and detecting each of the classified sections as a field;
A procedure for detecting characters included in the form image data;
A procedure for detecting a keyword associated with a character attribute to be written in the field in advance from the character;
Detecting the field corresponding to the keyword based on the positional relationship between the keyword and the field;
A procedure for storing the detected field in association with the keyword;
Form definition program to execute.