JP2012008791A - Form recognition device and form recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a form recognition device for correctly recognizing characters of a seal existing in a form.SOLUTION: A form recognition device comprises: an image input unit for acquiring a form image; a seal image detection unit for detecting a seal image from the form image; a background color separation unit for separating a seal color from the other colors; a tilt correction unit for correcting the tile of the seal image; a character string search unit for cutting out a character string; a character cutout unit for cutting out each character constituting the character string; a character recognition unit for recognizing each character; a knowledge processing unit for selecting a character recognition result matching a character string format; a rejection determination unit for determining whether to reject the character recognition result based on the character recognition result and the reliability of the character recognition result; and a retry determination unit for determining whether to recognize the characters of the seal image again if the character recognition result is rejected.

Description

  The present invention relates to a form recognition apparatus for recognizing characters existing in a form, and more particularly, to a form recognition apparatus for recognizing stamp characters existing in a form.

  The form recognition technology is used for a user to digitize a form. In this case, the user is, for example, a general company, a local government, a financial institution, an insurance institution, a medical institution, an educational institution, etc., and the form is, for example, an accounting slip, a purchase order, a gift certificate, a paid notice, These include salary reports, order forms, insurance contracts, general transfer forms, withholding forms, health checkups, medical fees, answer sheets, and application forms.

  The form recognition device searches items (for example, an ID field, a financial institution name field, an amount field, a product name field, a number field, an address field, a name field, etc.) from the form, and uses the characters described in the searched items. By reading, the item is digitized.

  A form recognition process executed by the form recognition apparatus will be described with reference to FIG. FIG. 15 is an explanatory diagram of a conventional form recognition process.

  First, the form recognition apparatus extracts a character string of each item from form image data acquired by a scanner or the like. In FIG. 15, a case where the character string “7,890,123” of “payment amount” is extracted will be described as an example.

  Next, the form recognition apparatus cuts out each character from the extracted character string. In FIG. 15, the extracted character string is cut out for each character as “7” “,” “8” “9” “0” “,” “1” “2” “3”. Then, the form recognition device recognizes the characters described in the form item by recognizing each cut out character. In FIG. 15, the form recognition apparatus recognizes that “7,890,123” is described in the item “payment amount”.

  A technique for cutting out each character from a character string and a technique for recognizing each cut out character are described in Non-Patent Document 1 and Non-Patent Document 2.

  A form recognition device that identifies each item from form image data and extracts a character string described in the identified item is known (for example, see Patent Document 1).

  The form recognition apparatus described in Patent Document 1 will be described.

  First, the form recognition device searches for a frame in the form image data. Specifically, the form recognition apparatus detects vertical ruled lines and horizontal ruled lines, and takes out an area surrounded by the detected vertical ruled lines and horizontal ruled lines as a frame. Next, the form recognition apparatus extracts a character string by cutting out a minimum rectangle surrounding black pixels in the extracted frame. The form recognition apparatus recognizes the item name of the extracted character string by recognizing the characters in the upper frame or the left frame in the extracted frame. Next, the form recognition apparatus recognizes the characters described in the item by cutting out each character from the extracted character string and recognizing each cut out character.

  A technique is known in which it is determined whether or not a stamp is present in a form, and if a stamp is present, the position and contour of the stamp are specified (for example, see Patent Document 2). A technique for recognizing a stamp character in a form is also known (see, for example, Patent Document 3).

  A technique for detecting a stamp in a form and recognizing a character in the stamp is used for reading a date of receipt, an institution name, and the like in a payment slip. For example, it is used for the purpose of classifying a form according to the presence or absence of arrears by checking a date and a due date.

JP 2007-328820 A JP 2009-25856 A JP-A-6-111066 Mohammed Cheriet, Nawwaf Kharma, Cheng lin Liu, Ching Suen, "Character Recognition Systems: A Guide for Students and Practitioners", Wiley-Interscience, 2007 Kenichiro Ishii, Noriyoshi Ueda, Eisaku Maeda, Hiroshi Murase, “Easy-to-understand pattern recognition”, Ohm Publishing House, August 1998

  In the conventional form recognition apparatus, it is difficult to search for a character string during the stamp due to the characteristics of the stamp. The characteristics of the stamp include, for example, that the stamp in the form is deviated from the position to be stamped and overlaps with a pre-printed portion (pre-printed portion) of the form, or the stamp is inclined. In addition, the darkness and darkness of the ink may cause blurring and crushing on the stamped characters and the stamped characters, and the stamped and stamped characters may be deteriorated. And so on.

  Further, when the character being stamped is faded or crushed, the accuracy of the form recognition device recognizing the character being stamped is deteriorated.

  In addition, when the form recognition device recognizes the date being stamped, it is necessary to distinguish the date correctly and recognize it correctly. However, it may be difficult for the form recognizing device to recognize a period, which is a separator of year, month and day, due to ink fading and noise. For example, if the period in the date column being stamped disappears due to blurring, it becomes “02127”, and the form recognition device cannot distinguish between “02.1.27” and “02.2.12.7”. In addition, the form recognition apparatus may erroneously recognize characters due to the lack of some of the numbers due to fading. For example, when the horizontal line at the top of “7” disappears due to fading, the form recognition device may misread as “1”. As described above, in order to prevent the form recognition device from recognizing characters by mistake, it is necessary to reject (unread) recognition of characters with low reliability.

  Further, in a conventional form recognition technique (for example, Patent Document 1), a rectangular frame is searched using vertical and horizontal ruled lines in order to cut out a character string existing in the form. However, in the case of stamping, there are cases where the character area where the character being stamped is not a rectangular frame, the stamping is tilted, the stamping and pre-printing overlap, etc. It cannot be applied to character recognition of stamps.

  In addition, since the conventional stamp recognition technology (for example, Patent Document 2 and Patent Document 3) cannot be separated from the background color, if the stamp and pre-printing overlap, the character and background existing in the stamp are overlapped. Overlapping characters makes it difficult to recognize characters. In addition, in the conventional stamp recognition technology, a histogram projected in a certain direction is used to correct the inclination of the seal. There is a problem that processing time is required to calculate a histogram of directions.

  In addition, in date recognition such as a receipt, it is required to recognize characters indicating months and days more accurately than characters indicating years. This is because, in terms of application, there are more cases of recognizing forms of the same year than cases of recognizing forms of several years ago.

  As described above, an object of the present invention is to provide a form recognizing apparatus that accurately recognizes a stamp character existing in a form.

  To show a typical example of the present invention, an image input unit that acquires a form image obtained by optically scanning a form, and a stamp that detects a stamp image from the form image acquired by the image input unit. An image detecting unit; a background color separating unit that separates a stamp color indicating the color of the outline of the stamped image from other colors; and the stamp in which the stamp color is separated from the other colors by the background color separating unit. An inclination correction unit that corrects an inclination of an image, a character string search unit that searches for a character string from the stamped image whose inclination is corrected by the inclination correction unit, and extracts the searched character string, and the character string search unit A character cutout unit that cuts out each character that constitutes the character string from the character string cut out by the step, and recognizes each character that is cut out by the character cutout unit, and all the characters that constitute the character string A character recognition result indicating a recognition result is calculated, a character recognition unit that calculates the reliability of each recognized character, and a character recognition result calculated by the character recognition unit is converted into a character string format designated in advance. A knowledge processing unit that selects a matching character recognition result and calculates a reliability for the selected character recognition result based on a reliability of each character calculated by the character recognition unit; and selected by the knowledge processing unit And a rejection determination unit that determines whether or not to reject the character recognition result selected by the knowledge processing unit based on the character recognition result and the reliability of the character recognition result, and the character recognition by the rejection determination unit. A retry determination unit that determines whether or not to recognize the character of the stamped image again when the character recognition result by the unit is rejected.

  According to the present invention, it is possible to accurately recognize a stamp character existing in a form.

It is a block diagram of the form recognition apparatus of the 1st Embodiment of this invention. It is a figure for demonstrating each module which performs the form recognition process by the form recognition apparatus of the 1st Embodiment of this invention. It is explanatory drawing of the form image data of the 1st Embodiment of this invention. It is explanatory drawing of the binary image data of the 1st Embodiment of this invention. It is explanatory drawing of the contour image data of the 1st Embodiment of this invention. It is explanatory drawing of the outline tracking process of the 1st Embodiment of this invention. It is explanatory drawing of the black pixel point selected in order to calculate an intersection by the outline tracking process of the 1st Embodiment of this invention. It is explanatory drawing which calculates the center and radius of the outline of the round mark of the 1st Embodiment of this invention. It is explanatory drawing of the date separator line of the stamp of the 1st Embodiment of this invention. It is explanatory drawing of the character string extracted by the character string search part of the 1st Embodiment of this invention. It is explanatory drawing of the state by which the stamp of the 1st Embodiment of this invention was inclination-corrected. It is a figure for demonstrating each module which performs the form recognition process by the form recognition apparatus of the 2nd Embodiment of this invention. It is explanatory drawing of the dictionary for a stamp recognition of the 2nd Embodiment of this invention. It is explanatory drawing of each module for performing the stamp template production | generation process of the stamp template production | generation part of the 2nd Embodiment of this invention. It is explanatory drawing of the conventional form recognition process.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS.

(First embodiment)
A first embodiment of the present invention will be described with reference to FIGS.

  The form (or stamp) recognition apparatus of the present invention detects a stamp from an input form image, recognizes a character existing in the detected stamp, and classifies the form according to a character recognition result. Examples of the form having a seal include a payment form such as a utility bill with a receipt stamp. When the form recognition apparatus recognizes the receipt, for example, the date in the receipt is recognized, and the form is classified based on whether or not the payment has been delayed.

  FIG. 1 is a configuration diagram of a form recognition apparatus according to the first embodiment of the present invention.

  The form recognition device 101 includes an input device 102, a display device 103, an image acquisition device 104, a communication device 105, a calculation device 106, and an external storage device 107.

  The input device 102 is a device for inputting a command for controlling a program executed by the arithmetic device 106, a command for controlling an external device connected to the form recognition device 101, and the like. The input device 102 is, for example, a keyboard or a mouse.

  The display device 103 is a display or the like that displays processing contents and the like as appropriate.

  The image acquisition device 104 is an image acquisition device such as a scanner, and acquires a form image by optically scanning the form. The acquired form image or the like is stored in the external storage device 107.

  The communication device 105 communicates data with an external device (for example, a PC or a server) connected to the form recognition device 101. The communication device 105 receives an execution command and data such as an image and text input by a user from an external device. In addition, the communication device 105 transmits the processing content in the form recognition device 101, the form recognition result by the form recognition device 101, and the like to an external device.

  The arithmetic device 106 is a CPU, and is an arithmetic device that executes a form recognition process for recognizing a form.

  The external storage device 107 is an external storage device such as an HDD (Hard Disk Drive) and a memory. The external storage device 107 stores various data such as a form image, a seal image, and a seal recognition dictionary. Further, the external storage device 107 temporarily stores data and the like generated during the processing executed by the arithmetic device 106.

  The form recognition device 101 only needs to include at least the arithmetic device 106 and the external storage device 107, and may not include the input device 102, the display device 103, the image acquisition device 104, and the communication device 105.

  If the form recognition device 101 does not include the input device 102, the processing is started when an instruction is given from the external device via the communication device 105, or the processing is automatically performed at a pre-instructed time. To be started.

  When the form recognition device 101 does not include the display device 103, the processing result by the form recognition device 101 is transmitted to the external device via the communication device 105 or stored in the external storage device 107. To do.

  When a module that executes a certain process inputs a processing result to a module that executes another process, the certain module may input the other module via the processing result via the external storage device 107. . Specifically, when the processing unit 1 outputs the processing result to the processing unit 2 and the processing result of the processing unit 1 is input to the processing unit 2, the processing unit 1 outputs the processing result to the external storage device 107, The external storage device 107 stores the processing result, and the processing unit 2 acquires the processing result of the processing unit 1 stored in the external storage device 107 as an input.

  Further, the module for executing the processing is appropriately controlled by the user via the input device 102.

  FIG. 2 is a diagram illustrating each module that executes a form recognition process by the form recognition apparatus 101 according to the first embodiment of this invention.

  The form recognition process is a process for recognizing characters in a stamped area stamped on a form.

  The form recognition device, for example, detects the presence or absence of a seal on the form shown in FIG. 3, and recognizes the characters (bank name, date, etc.) written in the seal when there is a seal, and results of character recognition In order to categorize the form according to or to refer to it later, the form image is stored in the storage device together with the recognition result. For example, it is used for the purpose of classifying a form by recognizing the date of the receipt and comparing the recognized date with the payment deadline to determine whether or not the payment is overdue.

  The form recognition apparatus 101 includes, as modules for executing a form recognition process, an image input unit 211, a seal detection unit 212, a background color separation unit 213, an inclination correction unit 214, a character string search unit 216, a character cutout unit 217, A character recognition unit 218, a knowledge processing unit 219, a rejection determination unit 220, a retry determination unit 221, a recognition result recording unit 222, and a form classification unit 223, and a knowledge processing dictionary 202 as a database to be referred to by the knowledge processing unit 219; The recognition result recording unit 222 includes a recognition result DB 203 for recording the recognition result.

  Each module and database will be described.

  The image input unit 211 acquires form image data by optically scanning the form with the image acquisition device 104, and outputs the acquired form image data to the seal detection unit 212. Here, it is assumed that the image input unit 211 has acquired the form image data shown in FIG.

  FIG. 3 is an explanatory diagram of the form image data according to the first embodiment of this invention.

  As shown in FIG. 3, the form taken into the form recognition apparatus 101 of the present embodiment is a tax receipt, and is stamped at the lower right.

  Returning to FIG. 2, the seal detection unit 212 will be described.

  The stamp detection unit 212 determines whether or not a stamp exists in the form image data input from the image input unit 211. If a stamp exists, the stamp detection unit 212 detects a stamp area from the form image data.

  Specifically, if there is a seal, the seal detection unit 212 performs an image binarization process, a contour extraction process, a contour tracking process, and a center coordinate / radius estimation process to obtain the shape of the seal, and Estimate the size of the stamp (when the stamp is a circle, the center coordinates and radius of the stamp area, and when the stamp is a square mark, the length and width of the stamp area, etc.) Then, the process proceeds to the background color separation unit 213.

  On the other hand, if there is no seal, the seal detection unit 212 records the form recognition result in the recognition result DB 203, classifies the form, and if there is a form to be read next, the image input unit 211 performs processing. Move.

next,
A process of the seal detection unit 212 when it is determined that a seal exists will be described.

  First, the image binarization process will be described.

  In the image binarization process, the seal detection unit 212 converts pixels having a predetermined luminance value or more into black pixels among the pixels constituting the form image data input from the image input unit 211, and uses the predetermined luminance value. By converting the smaller pixels into white pixels, the form image data is converted into binary image data composed of white pixels and black pixels.

  By performing the image binarization process on the form image data shown in FIG. 3, the form image data shown in FIG. 3 is converted into the binary image data shown in FIG.

  FIG. 4 is an explanatory diagram of binary image data according to the first embodiment of this invention.

  In FIG. 4, among the pixels constituting the form image data shown in FIG. 3, pixels having a predetermined luminance value or more are converted into black pixels, and pixels smaller than the predetermined luminance value are converted into white pixels.

  Returning to FIG. 2, the contour extraction process of the seal detection unit 212 will be described.

  In the contour extraction process, the seal detection unit 212 extracts the contour of the black area composed of the black pixels of the binary image data converted by the image binarization process, and generates contour image data. The seal detection unit 212 converts, for example, a pixel at a boundary portion between a white region composed of white pixels and a black region composed of black pixels into a black pixel, and converts the other pixels into white pixels. Contour image data composed of black pixels is generated. In other words, the seal detection unit 212 keeps the black pixels adjacent to the white pixels out of the black pixels of the binary image data as black pixels, and converts the black pixels not adjacent to the white pixels into white pixels, thereby generating a contour. Generate image data.

  By performing the contour extraction process on the binary image data shown in FIG. 4, the contour image data shown in FIG. 5 is generated from the binary image data shown in FIG.

  FIG. 5 is an explanatory diagram of contour image data according to the first embodiment of this invention.

  In FIG. 5, in the binary image data shown in FIG. 4, pixels at the boundary between the white area and the black area are converted into black pixels, and the other pixels are converted into white pixels.

  In the contour extraction process, other than the contour that forms other than the straight line and the circumference may be removed based on the smoothness of the contour.

  Returning to FIG. 2, the contour tracking process of the seal detection unit 212 will be described.

  In the contour tracking process, the seal detection unit 212 is a process of tracking the contour and estimating the center coordinates of the circle when the contour is a circumference.

  Details of the contour tracking process will be described with reference to FIG.

  FIG. 6 is an explanatory diagram of the contour tracking process according to the first embodiment of this invention.

  First, the seal detection unit 212 selects one black pixel point X1 from the black pixel points of the contour image data converted by the contour extraction process, and is connected to the selected black pixel point X1 by another black pixel point. Another black pixel point X3 is selected. Then, the seal detection unit 212 selects one black pixel point X2 from the black pixel points connecting the black pixel points X1 and X3.

  Next, the seal detection unit 212 divides the straight vertical bisector connecting the black pixel point X1 and the black pixel point X2 and the vertical bisector of the straight line connecting the black pixel point X2 and the black pixel point X3. The intersection point C with the line is calculated.

  Here, when the intersection C does not exist in the form, it is determined that the line connecting the black pixel point X1 and the black pixel point X3 is a straight line. On the other hand, when the intersection C exists in the form, the calculated coordinates of the intersection C are estimated coordinates of the center point of the circle.

  Thus, in the contour tracking process, N (N is a natural number) groups of black pixel points X1, X2, and X3 are selected from the black pixel points constituting the contour image data, and the selected N groups. N intersection points C1, C2,..., CN are calculated. The coordinates of these intersections C1, C2,..., CN are estimated coordinates of the center of the circle.

  FIG. 7 shows a state in which the contour tracking process is executed on the contour image data shown in FIG. 5 and black pixel points constituting N groups are selected from the contour image data.

  FIG. 7 is an explanatory diagram of black pixel points selected for calculating intersections in the contour tracking process according to the first embodiment of this invention.

  In FIG. 7, the black pixel point selected in the contour tracking process is indicated by a cross.

  Returning to FIG. 2, the center coordinate / radius estimation processing of the seal detection unit 212 will be described.

  In the center coordinate / radius estimation process, the seal detection unit 212 estimates the center coordinates of the circle and the radius of the circle using the intersections C1, C2,..., CN calculated in the contour tracking process.

  Specifically, the seal detection unit 212 selects a set of intersections existing at positions close to each other from the intersections C1, C2,..., CN calculated by the contour tracking process, and sets the selected intersection set. The average coordinate is the center coordinate of the circle.

  Further, the seal detection unit 212 sets the average value of the distances from the calculated center coordinates to the coordinates of the black pixel points used to calculate the intersections constituting the selected intersection set as the radius of the circle.

  If the radius calculated in the center coordinate / radius estimation process is larger than a predetermined value, the seal detection unit 212 detects a circle as a seal from the radius.

  A method for calculating the radius of the circle will be described with reference to FIG.

  FIG. 8 is an explanatory diagram for calculating the center and radius of the outline of the circle according to the first embodiment of this invention.

  The center coordinate calculated by the center coordinate / radius process is indicated as a point C. The coordinates of the black pixel points used to calculate the intersection used to calculate this point C are shown as X1 to X25.

  The seal detection unit 212 calculates each distance from the point C to X1 to X25, and calculates the average of the calculated distance as the radius of the circle.

  As described above, the seal detection unit 212 can detect the seal and can calculate the center coordinates of the seal and the size (radius) of the seal. The outline of the stamp and the position of the stamp in the form can be specified from the center coordinates of the stamp and the size of the stamp.

  Returning to FIG. 2, the background color separation unit 213 will be described.

  The background color separation unit 213 extracts the seal area by separating the seal color from other colors (background colors).

  Specifically, the background color separation unit 213 determines the contour position of the stamp in the form image data input from the image input unit 211 based on the center coordinates of the stamp and the radius of the stamp calculated by the stamp detection unit 212. Is identified. Then, the background color separation unit 213 selects, as the stamp color, a color that exists within a predetermined range in the color space from the peak color among the colors near the contour position of the specified stamp. For the peak color, a histogram of a color space (for example, RGB, HCL, etc.) is calculated for the stamped image, and the color indicating the peak of the histogram is the peak color.

  The vicinity of the contour position of the stamp indicates a pixel existing within a predetermined distance from the contour position of the stamp. The predetermined distance is r1, the vertical length of the circumscribed rectangle of the stamp is L1, the horizontal length is L2, and the longer one of the vertical length and the horizontal length of the circumscribed rectangle of the stamp If L is L (L = max {L1, L2}), and k is a positive real number smaller than 1, the predetermined distance is expressed as r1 = kL.

  For example, when the stamp is a circle, the coordinates of pixels existing within a predetermined distance from the contour position of the stamp are r−r1 <if the center position of the circle is c and the radius of the circle is r. This is a pixel located at the coordinates of the point z where || z−c || <r + r1. Thus, even if the size (scale) of the stamp is different, pixels existing within a predetermined distance from the contour position of the stamp can be extracted.

  Next, the stamp color will be described.

  The peak color of the color information of pixels existing within a predetermined distance from the contour position of the stamp is set as p, and the color x such that the distance in the color space with respect to the peak color p is within a predetermined range is set as the stamp color. Extract. Here, when the distance D (x, p) in the color space between the stamp color x and the peak color p is set to r2, and a value indicating a predetermined range set in advance is set to r2, D (x, p) ≦ r2 is satisfied. All colors x are extracted as stamp colors. As the distance D (a, b) in the color space between the color a and the color b, for example, the Euclidean distance in the RGB color space, the HCL color distance in the HCL space, or the like may be used.

  By setting the pixel color not extracted as the stamp color x as the background color, the stamp color pixel is used as the stamp portion, the background color pixel is used as the background portion, and the stamp color and the background color are separated. This makes it possible to accurately detect the stamped portion even if the stamp existing in the form overlaps with the pre-printing of the form.

  Next, the inclination correction unit 214 will be described.

  The inclination correction unit 214 corrects the inclination of the stamp of the stamp image in which the stamp color is separated from the background color by the background color separation unit 213.

  Hereinafter, the inclination correction process of the stamp by the inclination correction unit 214 will be described.

  The inclination correction unit 214 detects the inclination of the stamp using a line segment (date separator line shown in FIG. 9) existing during the stamp. Specifically, the inclination correction unit 214 is located inside the outline of the seal detected by the seal detection unit 212 (for example, || z−c || <in the case of a circle whose center is c and radius is r). A line segment existing in all the pixels z) to be r is detected, and the inclination of the detected line segment is calculated with the x axis set in the inclination correction unit 214 as a reference. The inclination correcting unit 214 corrects the inclination of the stamp by rotating the entire stamping part so that the calculated line segment matches the x axis. Alternatively, a line segment having the smallest inclination with respect to the x-axis may be detected from line segments existing in the form, and the detected line segment may be used as a line segment serving as a reference for the inclination of the stamp.

  In addition, when there are a plurality of line segments (date separator lines) in the seal, the inclination correction unit 214 first selects an arbitrary line segment, and based on the selected line segment, Correct the tilt. When the character recognition is rejected by the rejection determination unit 220 described later, the retry determination unit 221 selects another line segment that has not yet been selected by the inclination correction unit 214, and based on the selected line segment. The inclination of the stamp may be corrected.

  Note that the inclination correction unit 214 may use the contour tracking method used in the contour tracking process of the seal detection unit 212.

  FIG. 9 is an explanatory diagram of a date separator line for stamping according to the first embodiment of this invention.

  As shown in FIG. 9, the first date separator 901 exists above the character string “20.5.31” indicating the date, and the second date separator 902 exists below the character string “20.5.31”.

  As described above, the inclination of the stamp is detected from the line segment existing in the stamp, and the inclination is corrected so that it matches the reference line, so that the inclination is corrected even if the stamp is inclined on the form. Thus, the stamped portion can be extracted. For this reason, the character recognition during a stamp can be performed correctly. Furthermore, since the inclination is detected using the line segment being stamped, the processing time can be shortened compared to the inventions described in Patent Documents 2 and 3 in which the inclination is corrected using a histogram in which ruled lines are projected in a certain direction.

  Returning to FIG. 2, the character string search unit 216 will be described.

  The character string search unit 216 is a character that is an area where characters exist based on the first date separator line 901, the second date separator line 902, and the outline of the seal from the stamp whose inclination is corrected by the inclination correction unit 214. An area is specified, a character string is searched from the specified character area, and the searched character string is extracted. Here, it is assumed that the character string shown in FIG. 10 has been extracted.

  FIG. 10 is an explanatory diagram of a character string extracted by the character string search unit 216 according to the first embodiment of this invention.

  The character string search unit 216 extracts a character string consisting of “20.5.31”. Here, the character “2” is extracted together with diagonal lines. This is due to the fact that the stamp and pre-printing overlap in the form image data shown in FIG.

  Returning to FIG. 2, the character cutout unit 217 will be described. Each character is cut out from the character string extracted by the character string search unit 216.

  Specifically, the character cutout unit 217 calculates, for example, a component that connects a black pixel and a black pixel, and cuts out the calculated component as a character.

  Note that characters existing in the seal may be connected to each other due to ink bleeding at the time of the seal. For this reason, the character cutout unit 217 separates the connected components using the smoothness of the contours of the components that connect the black pixels and the black pixels, thereby connecting a plurality of components that are connected due to ink bleeding or the like. You may cut out a character as a different character.

  The character recognition unit 218 recognizes individual characters cut out by the character cutout unit 217, and recognizes all characters constituting the character string extracted by the character string search unit 216. Is calculated.

  Note that there are various known methods for recognizing a single character. A recognition method used by the character recognition unit 218 can be selected from these recognition methods, and the selected recognition method can be used.

  Furthermore, the character recognition unit 218 calculates the reliability for each recognized character.

  The knowledge processing unit 219 selects a character recognition result that matches a format registered in a dictionary preset in the form recognition device 101 from among a plurality of character recognition results calculated by the character recognition unit 218. The dictionary set in advance in the form recognition apparatus 101 is a dictionary recorded in the knowledge processing dictionary 202 shown in FIG.

  Rejection determination unit 220 calculates the reliability for each character recognition result selected by knowledge processing unit 219, and rejects the character recognition result selected by knowledge processing unit 219 based on the calculated reliability. Determine whether or not.

  A process for calculating the reliability for each character recognition result will be described.

  For example, when the reliability of the recognition result of each character constituting the character string including the L characters of the character recognition result selected by the knowledge processing unit 219 is X1, X2,. In 220, the reliability (X) of the character recognition result is calculated by X = (X1 +... + XL) / L.

  Then, rejection determination unit 220 determines that the reliability of the character recognition result is low when the reliability of the character recognition result is smaller than a preset threshold, and rejects the one character recognition result. .

  Rejection determination unit 220 may calculate the reliability of the character recognition result by weighting the reliability of the recognition result of each character.

  Specifically, rejection determination section 220 sets the recognition result weights of L individual characters as a1, a2,..., AL (a1 +... + AL = 1), and the reliability (X) of one character recognition result. X = (a1 · X1 +... + AL · XL) / L.

  For example, when the character recognition result selected by the knowledge processing unit 219 indicates a character string indicating a date, the rejection determination unit 220 indicates a character corresponding to “day”, a character corresponding to “month”, and The weight may be decreased in the order of the characters corresponding to “year”. In other words, the reliability weight of the character corresponding to “Year” is set to the lowest, then the reliability weight of the character corresponding to “Month” is set to the lowest, and the reliability weight of the character corresponding to “Day” is set to the lowest. Make it high. As a result, it is possible to design so as to require a high degree of reliability for character recognition that is important to the user.

  As described above, the rejection determination unit 220 rejects the character recognition result whose reliability is smaller than the threshold value. For example, when the period in the date column being stamped disappears due to blurring, and some of the numbers are missing due to blurring. Since the reliability of the character recognition result in this case is lower than the normal character recognition result, the character recognition result in such a case can be rejected. Thereby, erroneous recognition of characters in the form recognition apparatus 101 can be prevented.

  The retry determination unit 221 shifts the process to the character string search unit 216 again, and determines whether to perform character recognition again (retry) or to end the process. For example, the retry determination unit 221 determines to retry when all the character recognition results selected by the knowledge processing unit 219 are rejected by the rejection determination unit 220.

  If the retry determination unit 221 determines to retry, the retry determination unit 221 may correct the inclination of the seal again.

  Specifically, there are a case where a plurality of line segments (date separator lines) exist in the seal, and a case where the seal is reversed. Since the processing of the retry determination unit 221 when there are a plurality of line segments in the seal has been described in the tilt correction unit 214, description thereof will be omitted.

  Processing of the retry determination unit 221 when the stamp is reversed will be described.

  Even if the inclination of the stamp is corrected by the inclination correction unit 214, the corrected stamp may be reversed as shown in FIG. FIG. 11 is an explanatory diagram showing a state in which the stamp of the first embodiment of the present invention is tilt-corrected. The retry determination unit 221 moves the process to the character string search unit 216 after rotating the stamp after the inclination correction by 180 degrees.

  Regarding the determination of whether or not the stamp is reversed, the retry determination unit 221, for example, when the average value of the reliability of the character recognition result selected by the knowledge processing unit 219 is smaller than a preset threshold value, It is determined that the stamp is reversed.

  The recognition result recording unit 222 recognizes the recognition result of the form including the character recognition result regardless of whether the character recognition result selected by the knowledge processing unit 219 is rejected or not. Record in the result database (DB) 203.

  The form recognition result recorded in the recognition result DB 203 may include form image data, stamp image data, stamp image data after skew correction, and information including stamp information. The stamp information includes a stamp shape (a stamp contour), a stamp size, a character string position and character string information in the stamp, a character position in the character string, a character size, a stamp color, and the like.

  The information registered in the recognition result DB 203 may be used for a search using character string information (bank name, date, etc.) being stamped as a key.

  The form classification unit 223 classifies the form based on the result of character recognition during stamping. For example, the form classification unit 223 may classify by bank name, and if the seal is a receipt, the form stamped before the payment due date and the form stamped after the payment deadline You may classify by.

  As described above, the form recognition apparatus 101 according to the present embodiment includes the background color separation unit 213, and thus separates the stamp color and the background color, and extracts a pixel having a color near the stamp color as the contour of the stamp unit. Therefore, even if the stamp overlaps with pre-printing or the like, the contour of the stamp portion can be accurately extracted.

  In addition, since the form recognition apparatus 101 of the present embodiment includes the tilt correction unit 214, when the stamp is tilted, the stamp is tilted in order to recognize the character being stamped after correcting the tilt of the stamp. Even if it is, it is possible to accurately recognize the character being stamped.

  In addition, since the inclination correction unit 214 of the present embodiment uses a line segment (date separator line) being stamped for correction of the inclination of the seal, Patent Documents 2 and 3 perform slope correction using a histogram in which ruled lines are projected in a certain direction. The processing time can be shortened as compared with the invention described in. Further, in the present embodiment, since the date separator line is extracted by the contour tracking process, even if the date separator line overlaps with the pre-printing or the date separator line is blurred, the date The separator line can be extracted accurately.

  In addition, since the form recognition apparatus 101 of the present embodiment includes the rejection determination unit 220, when the reliability of the character string that has been recognized is smaller than a predetermined threshold, the character string is rejected. It is possible to prevent erroneous recognition of characters in the recognition apparatus 101.

  Further, the rejection determination unit 220, when calculating the reliability of the character string that has been recognized by weighting each character, sets the weight of the reliability of the character corresponding to “year” to the lowest, The weight of the reliability of the character corresponding to “month” is lowered, and the weight of the reliability of the character corresponding to “day” is made highest. As a result, since it is rare for a form to be recognized by the form recognition apparatus 101 to have a form with different stamp years, the reliability of character recognition for “month” and “day” is set to “year”. By making the reliability higher than the character recognition reliability, it is possible to prevent the form recognition apparatus 101 from erroneously recognizing the characters “month” and “day”.

(Second Embodiment)
Next, a second embodiment of the present invention will be described with reference to FIGS.

  The form recognition apparatus 101 according to the second embodiment of the present invention refers to the seal recognition dictionary 301 shown in FIG. 13 to identify the type of the seal existing in the form, and stores the seal registered in the seal recognition dictionary 301. By referring to the information corresponding to the type, the character of the seal present in the form is recognized.

  As a result, the character string being stamped, the meaning of the character string (date, bank name, etc.), the character string position, the character position, and the character size are determined for each stamp type. By specifying, the position of the character is determined, and the character string search unit 316 and the character cutout unit 317 can use these pieces of information to reduce the cutout of the character and the recognition of the character by mistake. .

  The process executed by the form recognition apparatus 101 according to the second embodiment of the present invention includes a recognition phase (form recognition process) for recognizing a seal existing in a form, and a seal recognition dictionary 301 that is referred to for recognizing the seal. And a learning phase (dictionary generation processing for stamp recognition) to be generated.

  In the recognition phase, the form recognition apparatus 101 recognizes the seals existing in the form one after another and classifies the forms based on the seals. The form recognition apparatus 101 may refer to the stamp recognition dictionary 301 in the recognition phase. In the learning phase, the form recognition apparatus 101 generates the stamp recognition dictionary 301 based on the recognition result recorded in the recognition result DB 203 or the definition input by the user.

  FIG. 12 shows a module and a database for executing the recognition phase, and FIG. 14 shows a module and a database for executing the learning phase.

  In the present embodiment, the device that executes the processing in the recognition phase (recognition device) and the device that executes the processing in the learning phase (dictionary generation device for stamp recognition) are executed by the form recognition device 101. May be executed by the apparatus. When the recognition phase processing and the learning phase processing are executed by different devices, the recognition device that executes the recognition phase processing includes the module shown in FIG. 12, and the stamp recognition generated by the stamp recognition dictionary generation device. Recognize the seals present in the input form using the dictionary. The stamp recognition dictionary generating apparatus that executes the learning phase processing includes the module shown in FIG. 14, and creates the seal recognition dictionary 301 based on the recognition result recorded in the recognition result DB 203 and the definition input by the user. Generate. The processing of each module is executed by the arithmetic device 106.

  FIG. 12 is a diagram illustrating each module that executes a form recognition process by the form recognition apparatus 101 according to the second embodiment of this invention. Of the modules shown in FIG. 12, the same modules as those in the first embodiment of the present invention are given the same numbers, and descriptions thereof are omitted.

  The form recognition apparatus 101 of the present embodiment includes an image input unit 211, a seal detection unit 212, a background color separation unit 213, an inclination correction unit 214, a dictionary use determination unit 315, a character string search unit 316, a character cutout unit 317, a character A recognition unit 318, a knowledge processing unit 319, a rejection determination unit 320, a retry determination unit 321, a recognition result recording unit 222, a form classification unit 223, a stamp type identification unit 331, and a stamp deterioration determination unit 332, and the knowledge processing unit 219 As a database to be referred to, a knowledge processing dictionary 202, a recognition result DB 203 for the recognition result recording unit 222 to record a recognition result, and a stamp recognition dictionary 301 as a database to be referred to by the stamp type identification unit 331 are provided.

  Each module and each database will be described. Of the modules and databases shown in FIG. 12, the same modules and databases as those in the first embodiment of the present invention are assigned the same numbers, and descriptions thereof are omitted.

  The dictionary usage determination unit 315 determines whether or not to recognize a character using the stamp recognition dictionary 301. If the dictionary use determination unit 315 determines that the character is to be recognized using the seal recognition dictionary 301, the process proceeds to the seal type identification unit 331. If the dictionary use determination unit 315 determines that the character is to be recognized using the seal recognition dictionary 301, the process proceeds to the character string search unit 316. The details of the stamp recognition dictionary 301 will be described with reference to FIG.

  The determination process of the dictionary use determination unit 315 determines, for example, that a character is recognized without using the stamp recognition dictionary until the form recognition process is executed a predetermined number of times after the form recognition apparatus 101 is introduced. May be determined to be recognized using the stamp recognition dictionary. As a result, when the number of stamp information registered in the stamp recognition dictionary 301 reaches a predetermined number, the characters are recognized using the stamp recognition dictionary 301, so that the characters of the character string existing in the stamp can be accurately recognized.

  The determination process of the dictionary use determination unit 315 determines that the first character recognition recognizes the character without using the stamp recognition dictionary 301, and the character recognition is performed for the second time or later, that is, the retry determination unit 321. If it is determined that the character is to be recognized again, the stamp recognition dictionary 301 may be used to determine that the character is to be recognized.

  The processing of the character string search unit 316 when the dictionary use determination unit 315 determines that the character is recognized without using the seal recognition dictionary 301 is the same as the processing of the character string search unit 216 of the first embodiment. Therefore, explanation is omitted.

  When the dictionary use determination unit 315 determines that the character is to be recognized using the seal recognition dictionary 301, the character string search unit 316 includes the stamp type identification unit 331 among the stamp information registered in the seal recognition dictionary 301. The character string existing in the form is extracted with reference to the coordinates registered at the stamped character string position 1305 (see FIG. 13) of the stamp type specified by.

  The processing of the character extraction unit 317 when the dictionary usage determination unit 315 determines that the character is recognized without using the seal recognition dictionary 301 is the same as the processing of the character extraction unit 217 of the first embodiment. Therefore, explanation is omitted.

  When the dictionary use determination unit 315 determines that the character is to be recognized using the seal recognition dictionary 301, the character cutout unit 317 includes the seal type identification unit 331 among the stamp information registered in the seal recognition dictionary 301. The character is cut out with reference to the coordinates registered at the individual character position 1307 (see FIG. 13) of the stamp type specified by.

  The processing of the character recognition unit 318 when the dictionary use determination unit 315 determines that the character is recognized without using the seal recognition dictionary 301 is the same as the processing of the character recognition unit 218 of the first embodiment. Description is omitted.

  When the dictionary use determining unit 315 determines that the character is to be recognized using the seal recognition dictionary 301, the character recognizing unit 318 determines the degree of deterioration of the stamp (the blurring degree and the crushing degree) calculated by the stamp deterioration determining unit 332. ) To switch the character recognition processing method.

  For example, the character recognition unit 318 recognizes a character using the character recognition method that has learned the blurred character when the degree of blurring of the stamp is large, and the character that has learned the blurred character when the degree of collapse of the stamp is large. By recognizing characters using the recognition method, a character recognition method suitable for character deterioration due to stamping can be used, and the accuracy of character recognition can be improved.

  Since the processing of the knowledge processing unit 319 when the dictionary use determination unit 315 determines that the character is recognized without using the seal recognition dictionary 301 is the same as the processing of the knowledge processing unit 219 of the first embodiment, Description is omitted.

  When the dictionary use determination unit 315 determines that the character is to be recognized using the seal recognition dictionary 301, the knowledge processing unit 319 displays the format of the character recognition result and the seal information registered in the seal recognition dictionary 301. Among them, the character recognition result that matches the format registered in the stamp character string format 1306 of the stamp type specified by the stamp type specifying unit 331 is selected.

  In addition to the processing executed by the retry determination unit 221 according to the first embodiment of the present invention, the retry determination unit 321 determines that the dictionary use determination unit 315 recognizes a character without using the stamp recognition dictionary 301. If the character recognition in this case is determined to be a retry, the character may be recognized again using the stamp recognition dictionary 301.

  The stamp type identification unit 331 identifies the type of stamp using the stamp information registered in the stamp recognition dictionary 301.

  First, the stamp recognition dictionary 301 will be described with reference to FIG.

  FIG. 13 is an explanatory diagram of the stamp recognition dictionary 301 according to the second embodiment of this invention.

  The stamp recognition dictionary 301 includes a stamp ID 1301, a stamp template image 1302, a stamp shape 1303, a stamp size 1304, a stamp character string position 1305, a stamp character string format 1306, individual character positions 1307, and a stamp color 1308.

  In the seal ID 1301, a unique identifier of the seal type is registered. A template image for each type of stamp is registered in the stamp template image 1302. The template image registered in the stamp template image 1302 is a binary image or a multi-value image (gray image).

  In the stamp shape 1303, for each type of stamp, whether the shape of the stamp is a circle, a rectangle, an ellipse, or the like is registered. In the stamp size 1304, when the stamp shape is a circle, a radius is registered. When the stamp shape is a rectangle, the vertical length and the horizontal length are registered. When the stamp is an ellipse The minor axis and major axis are registered in.

  In the stamped character string position 1305, coordinates where the character string exists during the stamping are registered for each type of the character string present in the stamp. Examples of character string types include a character string type indicating a date and a character string type indicating a bank branch name. In the stamped character string position 1305, for example, coordinates where a character string exists based on the coordinates of the lower left corner of the circumscribed rectangle of the stamp are registered.

  In the stamp character string format 1306, the format of a character string for each type of character string existing in the same type of stamp is registered. For example, YYMMDD is registered in the stamp character string format 1306 of the character string type indicating the date. YYMMDD indicates a character string composed of 6-digit numbers, YY indicates the year and the numbers 00 to 99, MM indicates the month and the numbers 01 to 12, and DD indicates the day. It consists of a number from 01 to 31. Further, the XX branch is registered in the stamp character string format 1306 of the character string type indicating the branch name.

  In each character position 1307, coordinates indicating the position of each character constituting a character string existing in the same type of seal are registered. Each character position 1307 of the character string type indicating the year, month, day includes the coordinates of the position of each character indicating the year, the coordinates of the position of each character indicating the month, and the position of each character indicating the day. Coordinates are registered. As in the case of the stamp character string position 1305, for example, the coordinates at which the character string exists based on the coordinates of the lower left corner of the circumscribed rectangle of the stamp are registered in each character position 1307.

  In the stamp color 1308, the stamp color of the same type of stamp is registered. The stamp color is extracted by the background color separation unit 213.

  Returning to FIG. 12, the stamp type identification unit 331 will be described.

  The stamp type identification unit 331 identifies the type of the stamp image by comparing the stamp image after the tilt correction generated by the tilt correction unit 214 with the stamp template image 1302 of the stamp recognition dictionary 301.

  The type specifying process of the stamp image will be described.

  First, the stamp type identification unit 331 selects an entry that matches the shape registered in the stamp shape 1303 and the shape of the stamp image detected by the stamp detection unit 212 among the stamp types registered in the stamp recognition dictionary 301. To do. The stamp type specifying unit 313 then selects a stamp template image of an entry whose difference between the size registered in the stamp size 1304 of the selected entry and the size of the stamp image detected by the stamp detection unit 212 is equal to or less than a predetermined threshold. A template image registered in 1302 is selected. The stamp type specifying unit 331 compares the selected template image with the stamp image.

  Next, the stamp type specifying unit 331 determines which image has the larger outline by combining the center of the stamp image and the center of the template image. The stamp type specifying unit 331 adds a margin or the like to the image determined to have a large outline so that the outlines of both images match.

  Then, the stamp type specifying unit 331 compares the stamp image p and the template image q using the function K (p, q). Hereinafter, it is assumed that the stamp image and the template image are binary images or gray images, and are normalized so that black pixels are 1 and white pixels are 0. Further, the pixel value of the stamp image p located at the image position x is p (x), and the pixel value of the pixel q of the template image is q (x).

  Below, an example of a function used for comparing the stamp image p and the template image q will be described.

  As a function used for comparing the stamp image p and the template image q, K (p, q shown in Equation 1 which is a function that monotonically increases f (a, b) according to the value of | a−b |. ) In Equation 1, the sum is calculated over all pixels.

  As the function that increases monotonously in a broad sense, for example, the functions shown in Formulas 2 to 5 can be used by the stamp type identification unit 331 of the present embodiment.

  First, Formula 2 and Formula 3 will be described.

  In the function shown in Equation 1, the sum of absolute values of differences between the pixel value p (x) of the stamp image p and the pixel value q (x) of the template image q is calculated for all pixel positions.

  In the function shown in Equation 3, the sum of values obtained by squaring the absolute value of the difference between the pixel value p (x) of the stamp image p and the pixel value q (x) of the template image q is calculated for all pixel positions. Is done.

  In the functions shown in Equation 2 and Equation 3, the value of K (p, q) increases as the difference between the pixel value p (x) of the stamp image p and the pixel value q (x) of the template image q increases. For this reason, when the function shown in Formula 2 or Formula 3 is used for the comparison between the stamp image p and the template image q, the stamp type specifying unit 331 has a template image that minimizes the value of K (p, q). q is specified as an image indicating the type of stamp.

  Next, Formula 4 and Formula 5 will be described.

  In the function shown in Formula 4, the calculation is performed only for pixels in which the pixel value p (x) of the stamp image p is larger than the pixel value q (x) of the template image q. In the function shown in Formula 5, the calculation is performed only for pixels in which the pixel value q (x) of the template image q is larger than 0 and the pixel value p (x) of the stamped image p is larger than 0.

  The functions shown in Equation 4 and Equation 5 are asymmetric functions in which K (p, q) and K (q, p) are different. Note that the functions shown in Equation 2 and Equation 3 are symmetric functions in which K (p, q) and K (q, p) are the same.

  In the functions shown in Equation 4 and Equation 5, the more the stamp image p is crushed compared to the template image q, in other words, the more the template image q is faint compared to the stamp image p, the more K (p, q ) Value increases. In other words, the value of K (p, q) increases as the number of pixels close to black in the stamp image p is larger than that in the template image q. In Equation 4, the value of K (p, q) increases as the template image q becomes faint compared to the stamped image p because it is calculated only for pixels that satisfy p> q. It is.

  In Equation 5, the value of K (p, q) increases as the template image q becomes faint compared to the stamp image p because log (q (x) / p (x)) is p When (x)> q (x), the value is positive, and log ((q (x) + ε) / p (x)) in the sum of Equation 5 is when q (x) = 0. This is because log (q (x) / p (x)) is corrected with a positive real number ε> 0 so that can be calculated.

  When the type of the stamp indicated by the template image q is the same as the type of the stamp of the stamped image p, the degree of crushing of the stamped image p with the template image q as a reference can be calculated by K (p, q). The faintness of the stamped image p with reference to q can be calculated by K (q, p).

  K (p, q) indicates the degree of fading of the stamp image p with the template image q as a reference, and K (q, p) indicates the degree of collapse of the stamp image p with the template image q as a reference. You can also Specifically, in Expression 4, calculation is performed only for pixels in which the pixel value p (x) of the stamp image p is larger than the pixel value q (x) of the template image q. It suffices to perform calculation only for pixels whose value q (x) is larger than the pixel value p (x) of the stamped image p. Further, in Expression 5, calculation is performed only for pixels in which the pixel value p (x) of the stamped image p is p (x)> 0. However, in Expression 5, p and q are interchanged, and q (x) It suffices to perform computation only for pixels that are> 0.

  Here, when the stamp type identifying unit 331 identifies the stamp type of the stamp image p using the function shown in Formula 4 or Formula 5, αK (p, q) + βK (q , P), the template image q having the minimum value is specified as an image indicating the stamp type.

  Next, the stamp deterioration determination unit 332 will be described.

  The stamp deterioration determination unit 332 determines the degree of deterioration of the stamp image p by comparing the stamp image p with the template image q of the stamp type corresponding to the stamp image p, and corresponds to the degree of deterioration of the stamp image p. The character recognition method to be selected is selected, and the process proceeds to the character string search unit 316. If the degree of deterioration of the stamp image p is equal to or greater than a predetermined value, the stamp deterioration determination unit 332 does not shift the processing to the character string search unit 316 so that the character of the stamp image is not recognized. Good.

  The stamp deterioration determination unit 332 may use Formulas 1 to 5 to determine the degree of deterioration of the stamp image p. For example, the functions shown in Formula 2 and Formula 3 may be used. In this case, the stamp deterioration determination unit 332 determines that the deterioration of the stamp image p is larger as the value of K (p, q) of the function shown in Expression 2 and Expression 3 is larger.

  In addition, the seal deterioration determination unit 332 determines the degree of crushing and blurring of the stamp image p as the deterioration level using the functions (K (p, q) and K (q, p)) shown in Expression 4 and Expression 5. May be. When the functions shown in Formula 4 and Formula 5 are used, if the value of K (p, q) is large, the seal deterioration determination unit 332 determines that the degree of deterioration due to the seal of the stamp image p is large, and the collapse is large. Select the character recognition method. On the other hand, if the value of K (q, p) is large, it is determined that the degree of deterioration due to the blur of the stamped image p is large, and the character recognition method when the blur is large is selected.

  As described above, in the present embodiment, the character existing in the recognized seal is recognized using the same position information of the character as the recognized seal among the character position information registered in the seal recognition dictionary 301. Since the position of the character existing in the stamp is determined for each type of stamp, the form recognition apparatus 101 can perform character recognition accurately. For example, since the position of the year, month, and day of the date is determined by the belt position of the seal, even if the period that is the date break point disappears, the form recognition device 101 can accurately determine the date. Can be recognized.

  In this embodiment, the degree of deterioration of the stamp image is determined by comparing the stamp image with a stamp template image of the same type as the stamp image. For this reason, when the degree of deterioration of the stamped image is larger than a predetermined value, by preventing the character recognition during the stamping from being performed, the stamped image is severely deteriorated due to blurring or crushing, and all or part of the character is missing. It is possible to reduce misrecognition when the In the present embodiment, the character recognition processing method is changed according to the degree of deterioration of the stamped image. As a result, it is possible to select a character recognition process suitable for the degree of deterioration of the stamped image, so that the accuracy of character recognition can be improved.

  Next, the learning phase will be described.

  FIG. 14 is a diagram for explaining each module that executes a stamp template generation process according to the second embodiment of this invention.

  The stamp template generation process may be executed by an apparatus different from the form recognition apparatus 101 that executes the form recognition process, but is assumed to be executed by the form recognition apparatus 101 here.

  The form recognition apparatus 101 includes a stamp template generation unit 401, a user stamp registration unit 402, a stamp recognition dictionary 201, and a recognition result DB 203 as a module and database for executing a stamp template generation process.

  In the stamp template generation process, the stamp type specifying unit 331 in the form recognition process specifies the type of stamp of the stamp image, the character string searching unit 316 specifies the position of the character string being stamped, and the character cutout unit 317 performs the stamp. A seal recognition dictionary 301 is generated that is used in the process of specifying the position of each character in the middle character and the process of determining the degree of deterioration by the seal deterioration determination unit 332.

  The form recognition device 101 stores various information input directly by the user in the seal recognition dictionary 301, and the recognition result stored in the recognition result DB 203 in the form recognition process in the seal recognition dictionary 301. A stamp template generation unit 401 for registration is provided.

  The stamp template generation unit 401 classifies the form for each type of stamp based on information stored in the recognition result DB 203. Then, the stamp template generation unit 401 generates a stamp template image for each classified stamp type, and the generated stamp template image together with stamp information (such as the shape, size, and stamp color of the stamp) is a dictionary for stamp recognition. 301 is registered. The process of the stamp template generation unit 401 is divided into a stamp classification process, a stamp template generation process, and a stamp information registration process.

  First, the stamp classification process of the stamp template generation unit 401 will be described.

  In the stamp classification process, the stamp template generation unit 401 classifies the stamp image stored in the recognition result DB 203 into one or more types based on the shape, character recognition result, character position, size, and the like. Note that the stamp template generation unit 401 does not need to classify a stamp image in which character recognition has failed.

  In the stamp classification process, the character string such as the date of the seal is changed by rotating a belt provided in the seal, etc., so that the character strings that are not changed in the seal (such as “Received”, “Receipt”, etc.) The stamp image is classified as a target. However, even if the character string can be changed, the position of the character constituting the changeable character string and the size of the character are not changed. Get the size of the character.

  Here, the stamp classification process of the stamp template generation unit 401 will be specifically described.

  First, the stamp template generation unit 401 selects one stamp image as a classification target stamp image from the stamp images stored in the recognition result DB 203, and an immutable character string of stamp images having the same shape as the selected stamp image is displayed. A stamp image that matches the selected stamp image is selected as a type stamp image.

  Next, the stamp template generation unit 401 has the size of the classification target stamp image and the size of the type stamp image out of the type stamp images smaller than the threshold value h1 specified by the user, and the character position of the classification target stamp image. And a stamp image whose character position of the type stamp image is smaller than the threshold value h2 designated by the user are selected as the same type stamp image. However, the vertical displacement of the character positions of the characters constituting the character string indicating the date is not considered as a difference in character positions. This is because each character constituting the character string indicating the date is often changed by the rotation of the seal belt, and the vertical shift occurs even in the same seal.

  Next, the stamp template generation process of the stamp template generation unit 401 will be described.

  In the stamp template generation process, the stamp template generation unit 401 uses the threshold value h3 specified by the user to shift the center position and the contour position between the classification target stamp image selected in the stamp classification process and the same kind of stamp image. The image sizes of both are adapted so as to be smaller. Specifically, the stamp template generation unit 401 matches the center positions of both of the stamp images, and adds a margin to the stamp image having the smaller contour, thereby adapting the image sizes of both.

  Furthermore, the stamp template generation unit 401 removes a variable character string such as a date from both of the stamp images. Since the character positions in both of the stamped images are registered in the recognition result DB 203, the stamp template generating unit 401 refers to the recognition result DB 203 and specifies the character positions of the variable character strings in both of the stamped images. The pixel at the specified character position is converted into a white pixel.

  Next, the stamp template generation unit 401 converts both of the stamp images into a binary image. Specifically, the stamp template generation unit 401 converts a pixel whose luminance value is equal to or higher than the threshold value h4 specified by the user among the pixels constituting both of the stamp images to the black pixel, and the luminance value is specified by the user. A pixel smaller than the threshold value h4 is converted into a white pixel.

  Next, the stamp template generation unit 401 generates one gray-scale gray image from both of the stamp images converted into the binarized image, and uses the generated gray image as the stamp template image.

  A process of generating a gray image of the stamp template generation unit 401 will be described.

  Here, since N-1 stamp images are selected as the same type of stamp image and there is one classification target stamp image, it is assumed that the number of both stamp images is N. Also, the pixel value (1 for black, 0 for white, etc.) at the image position x of any one of the imprinted images converted into the binarized image is defined as pi (x).

  The pixel value p (x) at the image position x of the gray image is calculated by p (x) = (p1 (x) +... + PN (x)) / 255.

  By creating a stamp template image of a gray image, it is possible to improve robustness against a difference in the outline and character thickness of a stamp due to a difference in ink density at the time of stamping.

  Next, the stamp information registration process of the stamp template generation unit 401 will be described.

  In the stamp information registration process, the stamp template generation unit 401 newly generates an entry for registering the stamp template image generated by the stamp template generation process in the stamp recognition dictionary 301.

  Then, the stamp template generation unit 401 registers the stamp template image generated by the stamp template generation process in the stamp template image 1302 of the generated entry.

  Then, the stamp template generation unit 401 registers a unique identifier in the stamp ID 1301 of the entry, and registers the shape of the stamp template image in the stamp shape 1303. In addition, the stamp template generation unit 401 registers the size of the stamp image generated by the stamp template generation process in the stamp size 1304 of the entry.

  The stamp template generation unit 401 registers coordinates indicating the position of the character string existing in the stamp template image in the stamp character string position 1305 of the entry, and stores the coordinates in the stamp template string image 1306 of the entry in the stamp template image. Register the existing string format.

  In addition, the stamp template generation unit 401 registers the position of the character constituting the character string existing in the stamp template image in each character position 1307 of the entry, and registers the stamp color in the stamp color 1308 of the entry. To do.

  As described above, the stamp template generation unit 401 generates the stamp recognition dictionary 301 shown in FIG.

DESCRIPTION OF SYMBOLS 101 Form recognition apparatus 102 Input apparatus 103 Display apparatus 104 Image acquisition apparatus 105 Communication apparatus 106 Arithmetic apparatus 107 External storage device 201 Dictionary for imprint recognition 202 Knowledge processing dictionary 203 Recognition result DB
211 Image input unit 212 Impression detection unit 213 Background color separation unit 214 Inclination correction unit 216 Character string search unit 217 Character extraction unit 218 Character recognition unit 219 Knowledge processing unit 220 Rejection determination unit 221 Retry determination unit 222 Recognition result recording unit 223 Form Classification part 301 Dictionary for seal recognition

Claims (15)

An image input unit for acquiring a form image obtained by optically scanning the form;
A stamp image detection unit for detecting a stamp image from the form image acquired by the image input unit;
A background color separation unit that separates the stamp color indicating the color of the outline of the stamp image from other colors;
An inclination correction unit that corrects an inclination of the stamp image in which the stamp color is separated from the other colors by the background color separation unit;
A character string search unit that searches for a character string from the stamped image whose inclination is corrected by the inclination correction unit, and extracts the searched character string;
A character cutout unit that cuts out each character constituting the character string from the character string cut out by the character string search unit;
A character that recognizes each character cut out by the character cutout unit, calculates a character recognition result indicating a result of recognizing all characters constituting the character string, and calculates a reliability of the recognized character A recognition unit;
From the character recognition result calculated by the character recognition unit, select a character recognition result that conforms to a character string format specified in advance, and the reliability for the selected character recognition result is calculated by the character recognition unit A knowledge processing unit that calculates based on the reliability of the character;
A rejection determination unit that determines whether to reject the character recognition result selected by the knowledge processing unit based on the character recognition result selected by the knowledge processing unit and the reliability of the character recognition result;
A form recognition apparatus, comprising: a retry determination unit that determines whether or not to cause a character of a stamped image to be recognized again when the character recognition result by the character recognition unit is rejected by the rejection determination unit. A recognition result recording unit that records a character recognition result calculated by the character recognition unit and a recognition result of the form image acquired by the image input unit;
A form classification unit that classifies the form image input by the image input unit for each form image satisfying a predetermined condition based on information recorded in the recognition result database. The form recognition apparatus according to 1. The background color separation unit is
A vertical length and a horizontal length of a circumscribed rectangle of the stamp image detected by the stamp image detection unit are calculated, and the longer length of the calculated vertical length and horizontal length is calculated as L. And when k is a positive real number smaller than 1 set in advance, pixels existing within a distance kL from the contour position of the stamp image are extracted as pixels near the contour,
Extracting a predetermined range of colors set in advance in the color space from the peak color of the pixels in the vicinity of the extracted contour as a stamp color;
The form recognition apparatus according to claim 1, wherein the extracted pixel of the stamp color is used as a stamp unit, and the pixels of other colors are used as a background unit.   The inclination correction unit detects a line segment existing inside the outline of the stamp image in which the stamp color is separated from the other colors by the background color separation unit, and based on the detected line segment The form recognition apparatus according to claim 1, wherein an inclination of the stamp image is calculated and the calculated inclination of the stamp image is corrected. The rejection determination unit
By calculating the reliability of the character recognition result by dividing the sum of the reliability of each character included in the character recognition result selected by the knowledge processing unit by the number of characters included in the character recognition result,
The form recognition apparatus according to claim 1, wherein when the reliability of the calculated character recognition result is smaller than a predetermined value, the character recognition result selected by the knowledge processing unit is rejected. The rejection determination unit, when the character recognition result selected by the knowledge processing unit is a character string indicating the year, month, day, the reliability of the character indicating the year, the reliability of the character indicating the month, and the date Weighting the reliability of the indicated character to calculate the reliability of the character recognition result,
The reliability weight of the character indicating the year is lowest, the reliability weight of the character indicating the month is then reduced, and the reliability weight of the character indicating the day is highest. The form recognition device according to Item 5. A stamp recognition dictionary that stores a stamp template image that is a template image of the same kind of stamp image and position information of characters existing in the stamp template image;
Using the stamp recognition dictionary, a stamp type identifying unit that identifies the type of the stamp image whose tilt is corrected by the tilt correcting unit;
A character position specifying unit that acquires position information of characters existing in the stamp image whose type is specified by the stamp type specifying unit using the seal recognition dictionary;
A deterioration determining unit that determines the degree of deterioration of the stamped image by comparing the stamped image whose tilt is corrected by the tilt correcting unit with a stamped template image of the same type as the stamped image. The form recognition apparatus according to claim 1.   The stamp recognition dictionary records at least one of the stamp shape, the stamp size, the character string format, the size of each character constituting the character string, and the stamp color of the same type of stamp image, Further, as the position information of the character existing in the stamp template image, at least one of the position information of the character string existing in the stamp template image and the position information of each character constituting the character string is stored. The form recognition apparatus according to claim 7.   When the pixel value of the stamped image at the position X is p (x) and the pixel value of the stamped template image at the position X is q (x), the stamp type specifying unit | p (x) −q The type of a stamp template image that minimizes the value of a function K (p, q) that increases monotonically in accordance with the value of (x) | is specified as the type of the stamp image. The form recognition device described. The seal recognition dictionary records the position information of the character string existing in the stamp template image, the position information of each character constituting the character string, and the format of the character string,
When the type of the stamp image is specified by the stamp type specifying unit, the character string search unit is based on the position information of the character string stored in the seal recognition dictionary, Search for a character string from the imprinted image with corrected inclination,
The character cutout unit, when the type of the stamp image is specified by the stamp type specifying unit, based on the position information of each character stored in the seal recognition dictionary, the character string search unit Cut out the characters that make up the character string extracted by
The character recognition unit switches the character recognition method according to the degree of deterioration of the stamped image determined by the deterioration determining unit when the type of the stamped image is specified by the stamp type specifying unit,
The knowledge processing unit, when the type of the stamp image is specified by the stamp type specifying unit, out of the character recognition results calculated by the character recognition unit, stored in the stamp recognition dictionary 8. The form recognition apparatus according to claim 7, wherein a character recognition result conforming to a character string format is selected.   When the pixel value of the imprinted image at the position X is p (x) and the pixel value of the imprinted template image at the position X is q (x), the deterioration determining unit | p (x) −q ( 8. The form recognition apparatus according to claim 7, wherein a value of a function K (p, q) that monotonously increases in accordance with a value of x) | The function K (p, q) is an asymmetric function in which the value of the function K (p, q) is different from the value of the function K (q, p) obtained by exchanging p and q.
12. The degradation determination unit calculates a degree of crushing and blurring of the imprinted image with respect to the imprint template image using the function K (p, q) and the function K (q, p). The form recognition device described. In the form recognition method that makes the form recognition device recognize the form including the seal,
An image input step for acquiring a form image obtained by optically scanning the form;
A stamp image detection step of detecting a stamp image from the form image acquired by the image input step;
A background color separating step for separating a stamp color indicating the color of the outline of the stamp image and other colors;
An inclination correction step for correcting an inclination of the stamp image in which the stamp color is separated from the other colors by the background color separation step;
A character string search step of searching for a character string from the stamp image whose inclination is corrected by the inclination correction step, and cutting out the searched character string;
A character extraction step of extracting each character constituting the character string from the character string extracted by the character string search step;
A character that recognizes each character cut out by the character cutting step, calculates a character recognition result indicating a result of recognizing all characters constituting the character string, and calculates a reliability of the recognized character A recognition step;
Of the character recognition results calculated by the character recognition step, select a character recognition result that matches a character string format specified in advance, and the reliability for each selected character recognition result is calculated by the character recognition step. A knowledge processing step to calculate based on the reliability of each character,
A rejection determination step for determining whether to reject the character recognition result selected by the knowledge processing step based on the character recognition result selected by the knowledge processing step and the reliability of the character recognition result;
A form recognition method comprising: a retry determination step of determining whether or not to recognize a character of a stamped image again when the character recognition result of the character recognition step is rejected by the rejection determination step. The form recognition method is:
A recognition result recording step for recording a character recognition result calculated by the character recognition step and a recognition result of the form image acquired by the image input step;
The form classification step of classifying the form image input in the image input step for each form image satisfying a predetermined condition based on information recorded in the recognition result database. The form recognition method according to 13. The form recognition device includes a stamp recognition dictionary that stores a stamp template image of the same type of stamp image and position information of characters existing in the stamp template image,
The form recognition method is:
Using the stamp recognition dictionary, a stamp type specifying step for specifying the type of the stamp image detected by the stamp image detection step;
Using the stamp recognition dictionary, a character position specifying step for acquiring position information of characters existing in the stamp image whose type is specified by the stamp type specifying step;
A deterioration determining step of determining a degree of deterioration of the stamped image by comparing the stamped image detected by the stamped image detecting step with a stamp template image of the same type as the stamped image. The form recognition method according to claim 13.