JP5674615B2 - Character recognition device and character recognition method - Google Patents

Description

  The present invention relates to a character recognition technique, and more particularly to a technique for determining whether or not to reject a character recognition result.

  The character recognition device (OCR device) is used for reading characters of slips / forms and general documents. Thereby, characters in the document can be handled as electronic data. The slips and forms include accounting slips, purchase orders, gift certificates, paid notices, salary reports, paid notices, order forms used in general companies, local governments, finance, insurance, medical institutions, educational institutions, etc. , Insurance contracts, total transfers, withholding, health checkups, medical fee details, answer sheets, application forms for admission. General documents include newspapers, magazines, technical materials, handwritten memos and the like used in offices or individuals. The character recognition device searches items such as ID, financial institution name, amount, product name, number, address, name, etc. from these documents, and reads these characters to digitize these items.

  Character recognition processing mainly includes pre-processing such as imaging by document scanning, binarization (graying) and noise processing, layout analysis processing, character string extraction processing, character string recognition processing, retry processing, and results. It is divided into processing after recognition such as recording (FIG. 7).

  First, in the imaging of a document, the document is converted into an electronic image using a scanner or the like. This processing is not necessary when the document is imaged in advance.

  In the preprocessing, binarization of the document image, background removal, noise processing, inclination correction, and the like are performed. For example, non-patent document 4 describes binarization and non-patent document 5 describes tilt correction.

  Next, in the layout analysis, a text part of an image and a non-text part such as a figure are separated (Non-Patent Document 6), or the correspondence between tables is analyzed (Patent Document 1).

  Next, in the character string extraction, a character string is extracted from the text portion. In the case of a general document, it is a character string in line units. This is described in Non-Patent Document 7, for example.

  In the character string recognition, the character string extracted as described above is recognized. Thereby, the characters in the document are converted into electronic data. However, if it is determined that a character included in the extracted character string cannot be recognized based on a predetermined condition, the recognition result of the character is rejected. In this case, the recognition of the character fails, and the character is treated as an unrecognizable (unread) character at this point.

  In the retry process, it is determined whether the recognition is to be performed again. If it is determined that the recognition is to be performed again, the processing method is changed from an arbitrary process before that and the recognition is retried. For example, the processing is performed in a different method from the previous time, such as returning to the preprocessing and changing the binarization parameters. As a result of the retry processing, even if the first recognition fails, there are cases where the recognition succeeds by changing the parameters and processing again.

  In the processing after recognition, the recognition result is stored in a recognition result database, or the input document is sorted based on the recognition result. A scanned image may be recorded in the database together with the recognition result. Document sorting is performed based on, for example, the amount of money described in the document, an ID or an address, and the like. In addition, when recognition is rejected, a document including characters whose recognition has been rejected may be classified so as to be distinguished from other documents for later reprocessing.

JP 2008-21068 A JP 2007-328820 A

Mohammed Cheriet, Nawwaf Kharma, and Cheng lin Liu, and Ching Suen, "Character Recognition Systems: A Guide for Students and Practitioners.", Wiley-Interscience, 2007. T. M. Breuel, "Robust least square baseline finding using a branch and bound algorithm," in Document Recognition and Retrieval VIII, SPIE, San Jose, pp. 20-27, 2002. Breuel, T.M .: The OCRopus open source OCR system.In: Proceedings of SPIE Document Recognition and Retrieval XV, San Jose, CA, USA, pp. 0F1_0F15 (2008) F. Shafait, D. Keysers, and T. M. Breuel, "Efficient implementation of local adaptive thresholding techniques using integral images," in Document Recognition and Retrieval XV, (San Jose, USA), Jan. 2008. T. M. Breuel, "Robust least square baseline finding using a branch and bound algorithm," in Document Recognition and Retrieval VIII, SPIE, San Jose, pp. 20-27, 2002. D. Keysers, F. Shafait, and T. M. Breuel, "Document image zone classification − a simple high-performance approach,” in 2nd Int. Conf. On Computer Vision Theory and Applications, pp. 44-51, Mar. 2007. R. Smith, "An overview of the Tesseract OCR engine.," In Int. Conf. On Document Analysis and Recognition (ICDAR), Curitiba, Brazil, 2007.

  Degraded characters (for example, caused by blurring or crushing) are difficult to recognize, but need to be recognized as accurately as possible. Examples of fading and crushing will be described later (see FIG. 9).

  Further, non-characters may be input to the character recognition device. For this reason, it is required to reject them precisely so as not to misread them. Further, when it is difficult to recognize deteriorated characters, it is required to reject them more precisely than misreading.

A typical example of the present invention is as follows. That is, a character recognition device comprising a processor and a storage device connected to the processor, wherein the storage device calculates character image data to be recognized and the similarity between the character image and each character type. And a plurality of deterioration determination dictionaries including a function for calculating a deterioration degree of the character image for each type of deterioration, and a deterioration for each type of deterioration. The character recognition device extracts a numerical value indicating the characteristics of the character image for calculating the degree of similarity, and sets the degree of deterioration for each type of deterioration. A function for calculating the degree of deterioration of the character image for each type of deterioration is generated by performing regression learning using a variable and a numerical value indicating the characteristics of the character image of each character type in which the deterioration has occurred as an explanatory variable. Before Save the deterioration determination dictionary, a numerical value indicating the feature used to calculate the extracted similarity with and functions included in the recognition dictionary, calculating the similarity with respect to each character type of the character image For each type of deterioration, a numerical value indicating the characteristic of the character image for calculating the deterioration degree is extracted, a numerical value indicating the characteristic for calculating the extracted deterioration degree, and the deterioration determination The function included in the dictionary is used to calculate the degree of deterioration of the character image for each type of deterioration, and whether to reject the recognition result of the character image based on the calculated degree of deterioration. The determination result is recorded in the storage device, and when a document image is input, preprocessing including at least one of binarization or noise removal of the document image is performed, and the preprocessing is performed. Character string image from the document image The character image is cut out from the character string image, and it is determined whether or not to perform the character image recognition process again based on the degree of deterioration calculated for each type of deterioration. If it is determined that the recognition process is to be re-executed, the pre-processing, the character string image clipping, the character image clipping, the extraction of numerical values indicating the characteristics of the character image, or the similarity of the character image to each character type The calculation is re-executed using a parameter or a processing method changed according to the degree of deterioration .

  According to an embodiment of the present invention, the degree of deterioration is calculated by a method suitable for each type of deterioration, and it is determined whether to reject the recognition result based on the calculated degree. can do.

It is a block diagram which shows the hardware constitutions of the character recognition apparatus of embodiment of this invention. It is a functional block diagram which shows the flow of the process which the conventional character recognition apparatus performs. It is a functional block diagram which shows the flow of the process which the character recognition apparatus of embodiment of this invention performs. It is explanatory drawing of the specific example of the flow of the process of the character recognition in embodiment of this invention. It is explanatory drawing of the similarity calculation result by the similarity calculation part of embodiment of this invention. It is explanatory drawing of the information which shows the degradation state of a character image recorded on recognition result DB of embodiment of this invention. It is a flowchart which shows the procedure of the character recognition process which the character recognition apparatus of embodiment of this invention performs. It is explanatory drawing of the deteriorated character image handled in embodiment of this invention. It is explanatory drawing of the kind of degradation of the character image classified in embodiment of this invention. It is explanatory drawing of the division | segmentation of the character string which the character string recognition part of embodiment of this invention performs. It is explanatory drawing of the character string correct candidate network handled by the character string recognition part of embodiment of this invention. It is explanatory drawing of the character string dictionary of embodiment of this invention. It is explanatory drawing of the character feature extraction method by the character feature extraction part of embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a hardware configuration of a character recognition device 101 according to an embodiment of the present invention.

  The character recognition device 101 includes an input device 102, a display device 103, an image acquisition device 104, a communication device 105, an arithmetic device 106, and an external storage device 107.

  The input device 102 is a device for inputting a command for controlling a program executed by the arithmetic device 106, a command for controlling an external device (not shown) connected to the character recognition device 101, and the like. is there. The input device 102 is, for example, a keyboard or a mouse.

  The display device 103 is a display device or the like that displays processing contents and the like as appropriate.

  The image acquisition device 104 is an image acquisition device such as an optical scanner. The acquired image is stored in, for example, the external storage device 107.

  The communication device 105 communicates data with an external device (for example, a PC (Personal Computer) or a server, not shown) connected to the character recognition device 101. The communication device 105 receives, for example, an execution command, image data, text data, and the like input by a user from an external device. Further, the communication device 105 may transmit the processing content by the character recognition device 101 and the character recognition result to an external device.

  The arithmetic device 106 is a CPU (Central Processing Unit) that performs character recognition and the like according to a program stored in the external storage device 107.

  The external storage device 107 is an external storage device such as an HDD (Hard Disk Drive) and a memory. The external storage device 107 stores image data acquired by the image acquisition device 104, a program executed by the arithmetic device 106, data generated as a result of processing executed by the arithmetic device 106, and the like. Further, the external storage device 107 temporarily stores data generated during the processing executed by the arithmetic device 106.

  The character recognition device 101 only needs to include at least the arithmetic device 106 and the external storage device 107, and may not include all of the input device 102, the display device 103, the image acquisition device 104, and the communication device 105.

  If the character recognition device 101 does not include the input device 102, the character recognition device 101 may start processing according to an instruction received from an external device via the communication device 105, or may automatically start processing at a designated time. Good.

  If the character recognition device 101 does not include the display device 103, the processing result may be transmitted to an external device via the communication device 105, or the processing result may be stored in the external storage device 107.

  Input / output to a module (to be described later) for executing processing may be performed via the external storage device 107. For example, when the processing unit 1 (not shown) outputs a processing result, and the processing result is input to the processing unit 2 (not shown), the processing unit 1 outputs the processing result to the external storage device 107 and stores it in the external storage. The device 107 may store the processing result, and the processing unit 2 may acquire the processing result stored in the external storage device 107 as an input.

  A user can control a module that executes processing via the input device 102.

  FIG. 7 is a flowchart illustrating a procedure of character recognition processing executed by the character recognition device 101 according to the embodiment of this invention.

  The flowchart in FIG. 7 shows an outline of the character recognition processing executed by the character recognition device 101. Specific processing contents executed by each unit will be described later with reference to FIG.

  When a document such as a slip, a form, or a general document is input, the character recognition device 101 first images the document (step 701). This process is performed by the image acquisition device 104. Next, a preprocessing unit 201 (described later) of the character recognition apparatus 101 performs preprocessing (step 702). Next, the character string cutout unit 202 (described later) executes layout analysis (step 703) and character string extraction (step 704). Next, the character string recognition unit 300 (described later) executes character string recognition (step 705). Next, a retry determination unit 316 (described later) determines whether or not to retry the process. If it is determined to retry, the pre-processing unit 201, the character string cutout unit 202, and the character string recognition unit 300 At least one of the subsequent processes executes a retry process (step 706). Next, the recognition result recording unit 317 and the document classification unit 319 (described later) execute a process after recognition (step 707).

  The procedure of the process executed by the conventional character recognition device (see FIG. 2) is the same as that in FIG. However, as already described, the specific processing content of the character recognition apparatus 101 of this embodiment at least in steps 705 and 706 is different from the conventional processing content.

  FIG. 2 is a functional block diagram showing a flow of processing executed by the conventional character recognition apparatus.

  A conventional character recognition device includes an image input unit 200, a preprocessing unit 201, a character string cutout unit 202, a character string recognition unit 203, a retry determination unit 210, a recognition result recording unit 211, a recognition result database (DB) 212, a document A classification unit 213 and a recognition dictionary generation unit 214 are provided. The character string recognition unit 203 includes a character feature extraction unit 204, a similarity calculation unit 205, a character recognition dictionary 206, a rejection determination unit 207, a character string recognition unit 208, and a character string dictionary 209. The recognition dictionary generation unit 214 includes a character feature extraction unit 204, a recognition learning DB 216, and a recognition dictionary generation unit 217.

  When the character recognition device shown in FIG. 2 is realized by the same hardware (FIG. 1) as in the present embodiment, the dictionary and DB are stored in the external storage device 107. On the other hand, each unit other than the dictionary and the DB executes a program stored in the external storage device 107 by the arithmetic device 106, and the input device 102, the display device 103, the image acquisition device 104, and the communication device 105 as necessary. This is realized by controlling the external storage device 107 and an external device (not shown). Therefore, in the following description, the processing executed by each unit described above is actually executed by the arithmetic device 106.

  Hereinafter, each part of FIG. 2 will be described.

  The image input unit 200 acquires a document image image by optically scanning a document such as a general document, a form, or a slip using a scanner or the like provided as the image acquisition device 104. The acquired image is a color image, a gray image, a black and white image, or the like. Further, the image input unit 200 may execute processing for removing a specific color (background color) or the like.

  The preprocessing unit 201 performs binarization (monochrome image conversion) of a document image, noise removal, background processing, and the like. Since binarization and the like can be performed by a known method, detailed description is omitted. One example is described in Non-Patent Document 1.

  The character string cutout unit 202 searches for a character string in the document image and cuts out the character string image. A method for searching a character string from a general document is described in Non-Patent Document 2, for example. The search for a character string from a form is described in, for example, Patent Document 1, Patent Document 2, and Non-Patent Document 3. For example, in the case of a form, the character string cutout unit 202 may first search for a frame line and cut out a rectangular portion surrounded by the frame line as a character string.

  The character string recognizing unit 203 cuts out individual character images from the character string image and recognizes each character, thereby outputting the character string as a character code (or character representing code) string. However, as will be described later, when a predetermined condition is satisfied, the character string recognition unit 203 determines that it is difficult to recognize the character, and rejects the recognition result. In this case, the character is treated as “unread”. Such character recognition can be performed by a known method (for example, see Non-Patent Document 1). Detailed processing is performed by the character feature extraction unit 204, the similarity calculation unit 205, the rejection determination unit 207, and the character string recognition unit 208.

  A character string dictionary to be read is stored in the character string dictionary 209, and a recognition result is selected from character strings in the character string dictionary 209. An example of a reading target is an address, a date, an amount, or the like. For example, in the case of an address, a character string that displays all addresses to be read is stored in the character string dictionary 209, and the character string recognition unit 208 selects one of the character strings that display the stored address as a recognition result. To do.

  FIG. 4 is an explanatory diagram of a specific example of the flow of processing for character recognition in the embodiment of the present invention.

  For example, when a form image 401 such as a salary statement is input as a document image, the character string cutout unit 202 cuts out the character string image 402 in the amount column 401A to be read. For example, the character string cutout unit 202 searches for a frame line around the amount column 401A, and cuts out a character string image 402 of a rectangular portion surrounded by the frame line. Subsequently, the character string recognition unit 203 cuts out an image of each character (for example, a character image 403A) from the character string image 402. The method of cutting out (dividing) will be described later (see FIG. 10 and the like). Subsequently, the character string recognition unit 203 recognizes characters included in each character image, and an electronic character string 404, for example, a character string “7, 890, 123” representing a monetary amount is obtained. This recognition method will be described later (see FIGS. 13 and 5).

  Next, an example of processing of the character string recognition unit 208 will be described with reference to FIG.

  FIG. 10 is an explanatory diagram of character string division executed by the character string recognition unit according to the embodiment of this invention.

  In order to divide the character string image into character-by-character images, the character string recognition unit 208 first divides the character string image into a plurality of patterns.

  A character string 1001 in FIG. 10 is an example of a character string image divided by the character string cutout unit 202 and input to the character string recognition unit 203.

  The division result 1002 is an example of the division result of the character string 1001. Nodes 1002A to 1002E are division points of the character string 1001, and the division result 1002 shows an example in which the character string 1001 is divided into four parts (hereinafter also referred to as “patterns”) 1002F to 10002I. Each of these four patterns and a pattern obtained by synthesizing a plurality of adjacent patterns (hereinafter also referred to as “synthesis pattern”) are character image candidates.

Each pattern (including a composite pattern) can be represented by a set of a node corresponding to the beginning and a node corresponding to the end. In the example of FIG. 10, since there are ₅ C ₂ = 10 combinations of two nodes, there are 10 patterns (of which 6 are combined patterns). When a long character string is input, since the number of division points is large, the number of synthesis patterns becomes enormous. In this case, limiting the number of composite patterns may prevent the number from becoming enormous.

  A character candidate pattern 1003 indicates a character candidate network obtained by adding two synthetic patterns 1002J to 1002K to the above four patterns 1002F to 10002I. A composite pattern 1002J is a composite of patterns 1002F and 1002G, and a composite pattern 1002K is a composite of patterns 1002G and 1002H. In the example of FIG. 10, only two of the six synthetic patterns that can exist are shown, but other synthetic patterns may be further added.

  A character candidate pattern 1003 illustrated in FIG. 10 includes six character candidate patterns (that is, patterns 1002F to 1002K). The character string recognition unit 203 selects a character string that is a correct answer candidate by performing character recognition for each of these character candidate patterns. This character recognition is performed by the character feature extraction unit 204, the similarity calculation unit 205, and the rejection determination unit 207, and the character recognition dictionary 206 is referred to at that time.

  FIG. 11 is an explanatory diagram of a character string correct candidate network handled by the character string recognition unit according to the embodiment of this invention.

  The character string correct candidate network shown in FIG. 11 is obtained by adding the result of character recognition to the character candidate pattern 1003 shown in FIG. For example, the pattern 1002F is recognized as “t” or “l”.

  Each route from the leftmost node 1002A to the rightmost node 1002E of this character string correct answer candidate network becomes one correct candidate character string. For example, when the patterns 1002F, 1002G, 1002H, and 1002I are recognized as “t”, “l”, “r”, and “l”, respectively, the character string “tlrl”, the patterns 1002F, 1002K, and 1002I are respectively “t”, A character string “the” or the like when recognized as “h” and “e” is a correct candidate character string.

  The character string recognizing unit 208 selects the correct candidate character string from the correct candidate character strings that matches the character string format stored in the character string dictionary 209 based on the similarity of the character recognition results. Select as the final correct candidate. However, if the similarity of the character recognition result is smaller than a predetermined value, or if any of the correct candidate character strings does not match the character string format stored in the character string dictionary 209, the predetermined condition is satisfied. The character recognition result is rejected, and the character string is treated as unread.

  FIG. 12 is an explanatory diagram of the character string dictionary 209 according to the embodiment of this invention.

  The character string dictionary 209 stores information for identifying a character string to be read. However, for example, in an address-related dictionary, if an attempt is made to store character strings of all addresses that can appear, the amount of data becomes enormous, and therefore a tri-type dictionary structure is widely used. FIG. 12 shows a dictionary of seven words representing English days of the week as an example. A word obtained by sequentially tracing a node in the right direction from the leftmost node is a word stored in the dictionary. The capacity is reduced by sharing a part including the same character string (for example, “day”).

  Next, the character feature extraction unit 204, the similarity calculation unit 205, the character recognition dictionary 206, and the rejection determination unit 207, which are modules for recognizing the individual patterns 1002F to 1002K obtained as the division results, will be described.

  The character feature extraction unit 204 quantifies the character features included in the character image by converting them into vector values x = (x1, x2,..., Xn). As a method for extracting features from an image, a pixel feature extraction method, a contour feature extraction method, a gradient feature extraction method, or the like is widely used (see Non-Patent Document 1). In this embodiment, any character feature extraction method may be used. An example of simplest character feature extraction will be described with reference to FIG.

  FIG. 13 is an explanatory diagram of a character feature extraction method performed by the character feature extraction unit 204 according to the embodiment of this invention.

  First, the character feature extraction unit 204 normalizes the input character image 1301 to an image 1302 having a predetermined size. As a normalization method, there are a linear normalization method, a moment normalization method, a nonlinear normalization method, and the like (see Non-Patent Document 1), and any of them may be used. When the simplest linear normalization method is used, a normalized image 1302 having a predetermined size is generated by enlarging or reducing the input character image 1301.

  Next, the character feature extraction unit 204 converts the normalized image 1302 into a vector value. As the conversion method, there are an outline feature extraction method using a chain code, a gradient feature extraction method, a pixel feature extraction method, etc. (see Non-Patent Document 1), and any of them may be used. Here, the simplest pixel feature extraction method will be described. The character feature extraction unit 204 divides the normalized image 1302 into partial rectangular small areas 1303, and calculates the sum of the pixel values of each small area as a vector element. In the example of FIG. 13, the normalized image 1302 is divided into 8 × 8 = 64 partial rectangular small areas 1303, and 64 numerical values are calculated as vector elements. For this reason, the dimension number of the vector representing the character feature is 64.

  When the vector calculated as described above is input from the character feature extraction unit 204, the similarity calculation unit 205 refers to the character recognition dictionary 206 and is input for each character type to be read of the character candidate pattern. The similarity of a vector (hereinafter referred to as input vector x) is calculated. The character recognition dictionary 206 stores a function for calculating the similarity of the input vector x to the character type for each character type to be recognized.

For example, when 10 numbers from “0” to “9” are recognition targets, the character recognition dictionary 206 has 10 functions f0 (x) whose input is an n-dimensional vector and whose output is a real value.
f1 (x)
f2 (x)
...
f9 (x)
Is saved. The function fi (x) outputs the similarity of the vector x to the number i. The character recognition dictionary 206 is generated in advance by the recognition dictionary generation unit 214. An example of the similarity calculation result will be described with reference to FIG.

  FIG. 5 is an explanatory diagram of a similarity calculation result by the similarity calculation unit according to the embodiment of this invention.

  In the example of FIG. 5, each read target character type is displayed as a candidate character type in descending order of similarity to the input vector x. A column 501 displays a character image corresponding to the input vector x.

  For example, when the input vector x is a vector representing the characteristics of the character image 403A (see FIG. 4), the first candidate character type (column 502), the first candidate character similarity (column 503), the second candidate character type (column 504) ) And second-rank candidate character similarity (column 505) are “7”, “93”, “1”, and “68”, respectively. As a result of calculating the above f0 (x) to f9 (x) for the vector x representing the feature of the character image 403A, the value of f7 (x) (that is, the similarity between the character image 403A and the number “7”). Is the largest “93” and the value of f1 (x) is the second largest “68”.

  The similarity calculation unit 205 outputs the first candidate character type, the first candidate character similarity, the second candidate character type, and the second candidate character similarity calculated as described above. Although omitted in FIG. 5, the similarity calculation unit 205 similarly calculates and outputs candidate character strings ranked third and lower and their similarities.

  Rejection determination unit 207 determines whether to reject the recognition result based on the calculation result by similarity calculation unit 205. Here, a conventional rejection determination method will be described.

  As described above, the similarity calculation unit 205 calculates the similarity between each extracted character image and each character type to be read. For this reason, the character type with the highest similarity can be output as the recognition result. However, in practice, for example, a significantly deteriorated character image is input, a character image cut out at an incorrect position is input, or a non-character is cut out and input as a character by mistake. , Etc. can occur. It is difficult or impossible to correctly recognize such a character image. Even in such a case, since the similarity can be calculated, it can be forcibly recognized as any character based on the similarity. However, if the recognition accuracy (reliability) is predicted to be low to some extent (in other words, the possibility of misreading is high to some extent), the recognition result is rejected without forcibly recognizing the character as an unread character. Sometimes it is desirable to handle.

  For example, if the first candidate character similarity and the second candidate character similarity are substantially the same, the recognition result is ambiguous, so that the recognition accuracy is predicted to be low. Alternatively, it is predicted that the recognition accuracy is low even when the first candidate character similarity is extremely low. Rejection determination unit 207 can reject the recognition result in such a case.

  The rejection will be specifically described below. In the following description, the first candidate character type is described as m1, and the second candidate character type is described as m2.

When the difference between the first candidate character similarity and the second candidate character similarity is smaller than a predetermined threshold h1, that is,
fm1 (x) −fm2 (x) <h1
In this case, the rejection determination unit 207 determines that the recognition accuracy is low because the recognition result is ambiguous, and rejects the recognition result.

When the first candidate character similarity is smaller than a predetermined threshold h2, that is,
fm1 (x) <h2
In this case, rejection determination unit 207 determines that the first-ranked candidate character similarity is low (that is, recognition accuracy is low), and rejects the recognition result.

  The retry determination unit 210 determines whether to retry the process by changing parameters or the like when the recognition fails (that is, when the recognition result is rejected) or when the reliability of the recognition result is low. If it is determined to retry, any processing after the preprocessing unit 201 and subsequent processing are executed again.

  For example, when the recognition result is rejected by the rejection determination unit 207 of the character string recognition unit 203 and it is determined as unread, the retry determination unit 210 may determine to retry the process. When the pre-processing unit 201 and subsequent processing are retried, the pre-processing unit 201 may use, for example, different binarization parameters than the previous trial, or use a different noise removal method. May be. By changing parameters and the like in this way, an image different from the previous one can be obtained, and as a result, recognition may be successful.

  The recognition result recording unit 211 stores the recognition result in the recognition result DB 212. The input image may be saved together with the recognition result. Specifically, for example, the information shown in FIG. 5 may be stored in the recognition result DB 212 as it is. That is, the input image is stored in the column 501, and the candidate character types and their similarities in the respective ranks are stored in the column 502 and subsequent columns.

  The document classification unit 213 classifies the document based on the recognition result. For example, the document classification unit 213 may classify the document based on the amount of money obtained as a recognition result, a recognized ID, an address, or the like. In addition, a document including rejected characters may be classified so as to be distinguished from other documents for later retry.

  The recognition dictionary generation unit 214 generates a character recognition dictionary 206 used by the character string recognition unit 203 using the recognition learning DB 216.

  The recognition learning DB 216 stores character images with character type labels. An example of the stored character image will be described later (see FIG. 8). Generally, the more character images that are stored, the better. For example, it is desirable to store 1000 or more images for each character type to be read.

  The character feature extraction unit 204 in the recognition dictionary generation unit 214 performs the same processing as the character feature extraction unit 204 in the character string recognition unit 203. That is, the character feature extraction unit 204 in the recognition dictionary generation unit 214 converts each character image stored in the recognition learning DB 216 into a vector value. The set of the converted vector values and the corresponding character type labels is used by the recognition dictionary generation unit 217 to generate a function for calculating the similarity for each character type to be read.

The recognition dictionary generation unit 217 uses the set of vector values and character type labels calculated by the recognition dictionary generation unit 214 to generate a function for calculating the similarity for each character type. For example, when the number of character types is C, C functions f1 (x)
f2 (x)
...
fC (x)
Is generated. fi (x) is a function generated by learning so that a high value is output when a vector x extracted from a character image representing the i-th character type is input. This learning is performed by a known method.

  Next, the character recognition device 101 according to the embodiment of the present invention will be described.

  FIG. 3 is a functional block diagram illustrating a flow of processing executed by the character recognition device 101 according to the embodiment of this invention.

  The character recognition apparatus 101 according to the present embodiment includes an image input unit 200, a preprocessing unit 201, a character string cutting unit 202, a character string recognition unit 300, a retry determination unit 316, a recognition result recording unit 317, a recognition result DB 318, a document classification, and the like. A unit 319 and a dictionary generation unit 320.

  The character string recognition unit 300 includes a character feature extraction unit 301, a similarity calculation unit 302, a character recognition dictionary 303, a rejection determination unit 313, a category 1 feature extraction unit 304, a category 1 deterioration degree calculation unit 305, and a category 1 deterioration determination. Dictionary 306, category 2 feature extraction unit 307, category 2 degradation degree calculation unit 308, category 2 degradation determination dictionary 309, category N feature extraction unit 310, category N degradation degree calculation unit 311, category N degradation determination dictionary 312, character A column recognition unit 315 and a character string dictionary 209 are included.

  The dictionary generation unit 320 includes a character feature extraction unit 301, a recognition learning DB 321, a recognition dictionary generation unit 322, a category 1 deterioration learning DB 323, a category 1 feature extraction unit 304, a category 1 deterioration determination dictionary generation unit 324, and a category N. A degradation learning DB 325, a category N feature extraction unit 310, a category N degradation determination dictionary generation unit 326, and a rejection control dictionary generation unit 327 are included.

  Although not all of them are shown in FIG. 3, the character string recognition unit 300 actually includes a feature extraction unit, a deterioration degree calculation unit, and a deterioration determination dictionary for each classification from 1 to N. The dictionary generation unit 320 includes a deterioration learning DB, a feature extraction unit, and a deterioration determination dictionary generation unit for each classification from 1 to N. In the following description, a feature extraction unit, a deterioration degree calculation unit, a deterioration determination dictionary, a deterioration learning DB, and a deterioration determination dictionary generation unit related to a classification k (k is any one of 1 to N) are classified into k characteristics. The extraction unit, the category k degradation degree calculation unit, the category k degradation determination dictionary, the category k degradation learning DB, and the category k degradation determination dictionary generation unit are also described. The classification (that is, deterioration or non-character type) will be described later.

  The hardware configuration of the character recognition device 101 shown in FIG. 3 is as shown in FIG. Therefore, the above dictionary and DB are stored in the external storage device 107. On the other hand, each unit other than the dictionary and the DB executes a program stored in the external storage device 107 by the arithmetic device 106, and the input device 102, the display device 103, the image acquisition device 104, and the communication device 105 as necessary. This is realized by controlling the external storage device 107 and an external device (not shown). Therefore, in the following description, the processing executed by each unit described above is actually executed by the arithmetic device 106.

  Among the units shown in FIG. 3, the image input unit 200, the preprocessing unit 201, the character string cutout unit 202, and the character string dictionary 209 in the character string recognition unit 300 are the image input unit 200 shown in FIG. 2. Since it is the same as the pre-processing unit 201, the character string cutting unit 202, and the character string dictionary 209, description thereof will be omitted. Moreover, the description regarding the basic function of the conventional character string recognition part 203 with reference to FIGS. 10-13 etc. is applied also to the character string recognition part 300 of this embodiment. Hereinafter, in the processing executed by the character string recognition unit 300, description of parts common to the character string recognition unit 203 will be omitted, and parts different from the character string recognition unit 203 will be described.

  In the present embodiment, first, when deterioration that makes it difficult to recognize characters occurs in the input character image, or when the cut character string includes non-characters, the deterioration or Non-character types are classified. Examples of the type of deterioration include blurring and crushing of characters. On the other hand, examples of non-character types include the following.

  When an image of a document such as a form is input, a specific type of non-character frequently appears. For example, when an image including a ruled line is cut out as a character image because a character has contacted the ruled line, an image including an overlap of multiple characters is cut out as a character image, or an image of a character that is not to be read is cut out When a plurality of adjacent characters come into contact with each other and an image (or part thereof) of those characters is cut out as one character image, the character images are treated as non-characters. Here, the characters that are not to be read are, for example, kanji and the like entered in a column where numbers are supposed to be entered. An example of the contact between adjacent characters is the composite pattern 1002J of FIG. 10 that appears when the entire character “t” and a part of “h” contact each other.

  In the present embodiment, “blur” and “collapse” are particularly exemplified as the types of character deterioration, but these examples do not exclude non-characters. That is, the description regarding the type of deterioration in the present embodiment can also be applied to the types of non-characters described above (in other words, types of character deterioration caused by contact with ruled lines, etc.). The “category” of the category, the category k degradation degree calculation unit, the category k degradation determination dictionary, the category k degradation learning DB, and the category k degradation determination dictionary generation unit may include non-character types.

  FIG. 9 is an explanatory diagram of the types of deterioration of character images classified in the embodiment of the present invention.

  In the present embodiment, the deterioration of the character image is classified into a faint 901 and a crushed 902. For example, in a black and white character image in which black characters are displayed on a white background, the pixels that should originally be part of the characters become white, and deterioration that is assimilated with the background is “blurred” and should originally be part of the background Deterioration in which pixels become black and assimilate with characters is called “collapse”. Such deterioration occurs, for example, due to the influence of font, paper quality, scanner type, scanner setting, binarization method during preprocessing, or binarization parameters.

  In FIG. 9, blurred characters 901 </ b> A to 901 </ b> C are illustrated as examples of the blurred 901, and collapsed characters 902 </ b> A to 902 </ b> C are illustrated as examples of the collapsed 902. The blurred characters 901 </ b> A to 901 </ b> C are obtained by deteriorating “rock”, “U”, and “industry”, and line breaks or the like occur due to the blurred. The collapsed characters 902A to 902C are obtained by degrading “book”, “5”, and “3”, respectively, and disappearance of blank portions surrounded by lines, connection of lines, and the like occur.

  In FIG. 9, two types of blur 901 and collapse 902 are shown as examples of types of degradation. However, in this embodiment, degradation of a character image may be classified into other types. Hereinafter, an example in which the degradation of the character image is classified into N types (that is, category 1, category 2,..., Category N) will be described. One of them may be blurred 901 and the other may be collapsed 902.

  The character string recognition unit 300 cuts out individual character images from the character string image and recognizes each character, thereby outputting the character string as a string of character codes (or codes representing characters). If the predetermined condition is satisfied, the character string recognition unit 300 determines that it is difficult to recognize the character, and rejects the recognition result. In this case, the character is treated as “unread”.

  Furthermore, the character string recognition unit 300 of the present embodiment outputs a degree of deterioration for each type of deterioration for each cut out character image. This degree of deterioration is used for control of later processing.

  Hereinafter, the processing of the internal module of the character string recognition unit 300 will be described.

  The character string recognition unit 315 cuts out individual character images from the character string image and recognizes each character, thereby outputting the character string as a character code string. The character string dictionary 209 stores a character string dictionary to be read, and the character string recognition unit 315 selects one of the character strings included in the character string dictionary 209 as a recognition result. An example of a reading target is an address, a date, an amount, or the like. If the predetermined condition is satisfied, the character string recognition unit 300 determines that it is difficult to recognize the character, and rejects the recognition result. In this case, the character is treated as “unread”. Furthermore, the character string recognition unit 300 outputs a deterioration degree for each type of deterioration for each character image. Details of these processes will be described.

  The character string recognition unit 315 first divides the character string image as a character candidate pattern 1003 (see FIG. 10). The character feature extraction unit 301 calculates a vector representing the feature of each divided pattern, and the similarity calculation unit 302 refers to the character recognition dictionary 303 to determine the similarity of the calculated vector to each character type to be read. Is calculated. These processes may be the same as the processes performed by the character feature extraction unit 204 and the similarity calculation unit 205 in FIG. 2 with reference to the character recognition dictionary 206.

  Further, the category 1 feature extraction unit 304 calculates a vector representing the feature of each divided pattern, and the category 1 degradation degree calculation unit 305 refers to the category 1 degradation determination dictionary 306 to determine the first in each pattern. The degree of deterioration of each type (that is, category 1 deterioration degree) is calculated. Similarly, the category 2 feature extraction unit 307 calculates a vector representing the feature of each divided pattern, and the category 2 degradation degree calculation unit 308 refers to the category 2 degradation determination dictionary 309 to classify each pattern. 2 Calculate the degree of deterioration. The category N feature extraction unit 310 calculates a vector representing the feature of each divided pattern, and the category N deterioration degree calculation unit 311 refers to the category N deterioration determination dictionary 312 to determine the category N deterioration degree in each pattern. Is calculated.

  For example, the first type of deterioration may be due to “fading” and the second type of deterioration may be due to “crushing”. The category k feature extraction unit (k may be any of 1 to N, the same applies hereinafter) may calculate vectors representing the features of each pattern in the same manner as the character feature extraction unit 301. A method suitable for calculating the degree of deterioration (category k deterioration degree) may be used.

  After the recognition result of a certain pattern is rejected, the character string recognizing unit 315 determines the classification k degradation degree of the pattern and the similarity to each character type of the pattern when the retry determining unit 316 determines to retry the process. Based on this, it is possible to change the processing method and recalculate the degree of similarity and the degree of deterioration. This will be described with an example.

  When the recognition result of one of the character candidate patterns is rejected by the rejection determination unit 313, the character string recognition unit 315 may retry the character candidate pattern recognition process based on the degree of deterioration for each type of deterioration. Good.

  For example, when the degree of deterioration due to fading is large, pre-processing (binarization, noise removal, etc.) of the image of the character candidate pattern is performed in a method suitable for the faint characters, that is, it is more beautiful (that is, In some cases, it is possible to obtain an image of a character candidate pattern that is easy to recognize correctly. For example, in the case of fading, the pre-processing unit 201 may perform processing such as weakening (or eliminating) the isolated point noise removal processing or changing the binarization threshold so that it is easily determined to be black. . As a result, an image of a character candidate pattern that is less affected by blurring may be obtained.

  Similarly, when the degree of deterioration due to crushing is large, it is possible to perform preprocessing that makes it easy to obtain a character candidate pattern that is less affected by crushing.

  For the character candidate pattern image obtained in this way, the similarity calculation unit 302 and the category k deterioration degree calculation unit recalculate the similarity and the deterioration degree.

  Even if the recognition result is not rejected, if the degree of deterioration of a certain type is large, the preprocessing can be executed by a method that reduces the influence of the type of deterioration.

  Similarly to the above, when the recognition result of a character candidate pattern is rejected due to a certain type of deterioration, or when the degree of deterioration of a certain type is large even if it is not rejected, this method is suitable for that type of deterioration. The similarity calculation unit 302 calculates the similarity for each character type to be read. In this case, the character feature extraction unit 301, the similarity calculation unit 302, and the character recognition dictionary 303 include a feature extraction method, a similarity calculation method, and a recognition dictionary that are suitable for recognizing a character having the type of degradation.

  The character feature extraction unit 301 quantifies the character features included in the character image by converting them into vector values x = (x1, x2,..., Xn). As a method for extracting features from an image, a pixel feature extraction method, a contour feature extraction method, a gradient feature extraction method, a Gabor feature extraction method, or the like can be used. The feature amount extraction processing based on each method may be the same as that executed by the character feature extraction unit 204 (see FIG. 2).

  The character feature extraction unit 301 of the present embodiment may further select and apply a feature extraction method suitable for the type of deterioration. For example, when character candidate pattern recognition by the character string recognition unit 315 is retried, an appropriate feature extraction method is selected according to the determined type of degradation.

  For example, the Gabor feature extraction method is known to be suitable for recognizing blurred characters although it has a large amount of calculation. For this reason, the gradient feature extraction method or the contour feature extraction method with a small amount of calculation is normally used, and the character string recognition unit 315 determines that the deterioration of the character candidate pattern due to blurring is large, and the recognition is retried. In some cases, a Gabor feature extraction method may be used.

  When the vector calculated as described above is input from the character feature extraction unit 301, the similarity calculation unit 302 refers to the character recognition dictionary 303 and inputs the character candidate pattern for each character type to be read. The similarity of the vector (input vector x) is calculated. The character recognition dictionary 303 stores a function for calculating the similarity of the input vector x to the character type for each character type to be recognized. Such similarity calculation processing may be the same as that executed by the similarity calculation unit 205 (see FIG. 2).

  The similarity calculation unit 302 of the present embodiment may further select and apply a similarity calculation method suitable for the type of deterioration. For example, when recognition of a character candidate pattern by the character string recognition unit 315 is retried, an appropriate similarity calculation method is selected according to the determined type of deterioration.

  An example of similarity calculation according to the type of deterioration will be described. Using a similarity calculation function learned using a high-quality character pattern stored in the recognition learning DB and a deteriorated character (for example, a blurred character or a collapsed character) in the character recognition dictionary 303. The learned similarity calculation function is stored. For example, a function for calculating the similarity between the blurred character and the input vector x is stored for each character type to be recognized. The similarity calculation unit 302 normally calculates the similarity using a function learned using a high-quality character pattern, and when it is determined that deterioration due to blurring is large and recognition is retried. Calculates the degree of similarity using a function learned using the blurred characters.

  The category k feature extraction unit (that is, the category 1 feature extraction unit 304, the category 2 feature extraction unit 307, the category N feature extraction unit 310, etc.) converts the feature of the character included in the character image into a vector value x = (x1, x2, .., Xn) are converted into numerical values. As a method for extracting features from an image, a pixel feature extraction method, a contour feature extraction method, a gradient feature extraction method, a Gabor feature extraction method, or the like can be used. This method may be the same as that used by the character feature extraction unit 301. However, since the feature calculated here is used to calculate the degree of deterioration as described later, it is desirable not to execute blurring processing or the like for reducing deterioration.

  The category k deterioration degree calculation unit (that is, the category 1 deterioration degree calculation unit 305, the category 2 deterioration degree calculation unit 308, the category N deterioration degree calculation unit 311 and the like) is a category k deterioration determination dictionary (that is, for category 1 deterioration determination). Referring to the dictionary 306, the category 2 degradation determination dictionary 309, the category N degradation determination dictionary 312 and the like), the category k degradation degree of the character candidate pattern is calculated. The category k deterioration determination dictionary stores a function for calculating the category k deterioration degree of the input vector x.

  A function for calculating the deterioration degree for each character type to be read may be prepared, or a function common to all the character types to be read may be used.

For example, when 10 types of numbers “0” to “9” are to be recognized and a function for calculating the degree of deterioration is prepared for each character type to be read, the input to the category k deterioration determination dictionary is n-dimensional. 10 functions g0 (x) whose vectors and outputs are real values
g1 (x)
g2 (x)
...
g9 (x)
Is saved. The function gi (x) outputs the classification k deterioration degree for the number i of the vector x.

On the other hand, when a function common to all the character types to be read is used, the function k (x) having a n-dimensional vector as input and a real value as output in the category k deterioration determination dictionary.
Is saved.

  Here, calculation of the category k deterioration degree will be described. When a function for calculating the degree of deterioration is prepared for each character type to be read, the category k deterioration degree calculation unit calculates the category k by calculating gc (x) for the character type c having the highest similarity to the vector x. The degree of deterioration is calculated. For example, as illustrated in FIG. 5, when the character type having the highest degree of similarity with the vector x representing the feature of the image 403A is “7”, the category k degradation degree calculation unit converts g7 (x) to the category k degradation. Calculate as degrees. On the other hand, when a common function is used for all character types to be read, the category k deterioration degree calculation unit calculates the category k deterioration degree by calculating g (x).

  Rejection determination unit 313 determines whether or not to reject the recognition result based on the degree of similarity for each character type to be read and the degree of deterioration for each type of deterioration. The rejection control dictionary 314 stores information that is referred to in order to perform rejection determination based on the similarity and the deterioration level. Specifically, when the similarity for each character type to be read and the deterioration level for each type of deterioration are input to the rejection control dictionary 314, a function p that outputs a rejection score (real value) and a rejection score are used. And a threshold value for determining whether or not to reject based on. The threshold value may be different for each character type, or may be common to all character types.

  The rejection score is obtained by calculating the value of p with the similarity and the deterioration degree as arguments. The determination of rejection is performed as follows, for example. When a different threshold value is used for each character type, if the threshold value for character type i is hi and the first candidate character type is m, rejection determination unit 313 rejects the recognition result when p> hm, and otherwise recognizes the recognition result. It is determined that the result is not rejected. On the other hand, when a common threshold value h is used for all character types, rejection determination unit 313 determines that the recognition result is rejected when p> h, and the recognition result is not rejected otherwise.

  For example, if a function p that is easily affected by the degree of deterioration due to blurring (that is, the output tends to increase according to the degree of deterioration due to blurring) is used, faint characters are likely to be rejected. Can prevent misreading due to blurred characters.

  The retry determination unit 316 changes the parameter or the like when the recognition fails (that is, the recognition result is rejected), or the recognition result is not reliable even if the recognition result is not reliable. Determine whether to retry. If it is determined to retry, any one of the preprocessing unit 201, the character string cutting unit 202, and the character string recognition unit 300, and the subsequent processing are executed again.

  Similar to the retry determination unit 210 (see FIG. 2), the retry determination unit 316 may determine whether to retry the process. However, the retry determination unit 316 further performs a retry method (specifically, which process is retried and parameters used for the retry, etc.) according to the type of deterioration of the character image. To control.

  For example, when the recognition result is rejected by the rejection determination unit 313 of the character string recognition unit 300, the retry determination unit 316 may determine to retry the processing after the preprocessing unit 201. At this time, the retry determination unit 316 may control the processing method based on the degree of deterioration for each type of deterioration. For example, when the degree of deterioration due to blurring is large, the preprocessing unit 201 may be controlled to perform character image preprocessing (binarization, noise removal, etc.) by a method suitable for blurring.

  For example, the pre-processing unit 201 weakens (or does not execute) the isolated point noise removal processing when the degree of deterioration due to blurring is large (eg, greater than a predetermined threshold), or determines that the binarization threshold is black. Pre-processing is executed after changing parameters such as changing to a direction that is easy to be performed. As a result, a binary image with little blurring effect is obtained, and as a result, recognition may be successful. On the other hand, when the degree of deterioration due to crushing is large, the preprocessing unit 201 increases the isolated point noise removal processing or changes the binarization threshold to a direction in which it is easily determined to be white, etc. The pre-processing may be executed after making the change.

  Similarly, even when the recognition result is not rejected, if any type of deterioration degree is large, preprocessing is performed by a method suitable for the type of deterioration.

  The recognition result recording unit 317 records the recognition result in the recognition result DB 318. However, when the recognition result is rejected, information indicating that the recognition result is rejected and information indicating the cause of the rejection (specifically, for example, the degree of degradation of category k) may be recorded. .

  For example, the recognition result recording unit 317 may record information similar to that shown in FIG. 5 as the recognition result, and may further record information shown in FIG.

  FIG. 6 is an explanatory diagram of character recognition results recorded in the recognition result DB 318 according to the embodiment of this invention.

  The column 601 is the same as the column 501 in FIG. 5, and the input character image is recorded.

  A column 602 records a value indicating the degree of ambiguity of the recognition result of each character image. For example, a value calculated so as to increase as the difference between the first candidate character similarity (column 503 in FIG. 5) and the second candidate character similarity (column 505 in FIG. 5) decreases is shown in column 602 as the ambiguity. It may be recorded.

  In column 603, the rejection score calculated by the above-described function p based on the similarity of each character image and the category 1 deterioration is recorded. In column 604, the classification 1 deterioration degree of each character image is recorded. Although omitted in FIG. 6, the category 2 degradation degree to the category N degradation degree are similarly recorded in the recognition result DB 318.

  Similar to the document classification unit 213 illustrated in FIG. 2, the document classification unit 319 classifies the document according to the recognition result. However, the document classification unit 319 may classify a document including a rejected character according to a rejection factor. Specifically, the document classification unit 319 may classify the document according to the type of deterioration of characters included in the document, or may be classified according to the rank of the degree of deterioration based on the degree of deterioration.

  Note that the character recognition device 101 of the present embodiment may output the calculated degree of deterioration or the determination result of rejection made based on the degree of deterioration via the display device 103. For example, the character recognition device 101 may display a symbol or the like indicating that the character has been rejected instead of the character type of the recognition result for the character determined to be rejected. The type of deterioration and the degree of deterioration may be displayed. The type of deterioration that caused the rejection may be, for example, the type of deterioration corresponding to a category 1 degradation level to a category N degradation level that exceeds a predetermined threshold, or classification from category 1 degradation level. The type of deterioration corresponding to the maximum degree of N deterioration may be used.

  Next, generation of a dictionary used in the present embodiment will be described.

  The dictionary generation unit 320 uses the recognition learning DB 321 and the category k deterioration learning DB (that is, the category 1 deterioration learning DB 323, the category N deterioration learning DB 325, etc.), and the character recognition dictionary 303, the category k deterioration determination dictionary. (That is, the category 1 deterioration determination dictionary 306, the category 2 deterioration determination dictionary 309, the category N deterioration determination dictionary 312 and the like) and the rejection control dictionary 314 are generated. The generated dictionary is referred to by the character string recognition unit 300 for character string recognition and rejection determination.

  The recognition dictionary generation unit 322 generates a character recognition dictionary 303 used by the character string recognition unit 300 using the recognition learning DB 321. The generation method may be the same as that executed by the recognition dictionary generation unit 217 of FIG.

  The recognition learning DB 321 stores character images with character type labels, similarly to the recognition learning DB 216 of FIG. However, as will be described later, in the recognition learning DB 321 of this embodiment, a deteriorated character image may be stored in addition to a high-quality character image.

  The character feature extraction unit 301 in the dictionary generation unit 320 performs the same processing as the character feature extraction unit 301 in the character string recognition unit 300. That is, the character feature extraction unit 301 in the dictionary generation unit 320 converts each character image stored in the recognition learning DB 321 into a vector value. The set of sets of the converted vector values and the corresponding character type labels is used by the recognition dictionary generation unit 322 to generate a function for calculating the similarity for each character type to be read.

  Furthermore, the recognition dictionary generation unit 322 may generate a function for calculating the similarity that is suitable for all or some types of deterioration. This function is used by the character string recognition unit 315 to retry the recognition of the character candidate pattern according to the type of deterioration. In that case, the recognition dictionary generation unit 322 generates a function for calculating the similarity of the input vector x to the character type for each character type to be recognized for the deteriorated character.

  For example, the recognition learning DB 321 stores a high-quality (that is, no deterioration or little deterioration) character image and a deteriorated character image, and the recognition dictionary generation unit 322 performs normal character recognition. When generating a function that is used to calculate the degree of similarity at the time, only high-quality character images are used for learning, and when recognizing degraded character images (for example, when recognition is retried) When generating a function used in the above, only a degraded character image may be used for learning.

  Specifically, for example, even if the sample as shown in the character image 802 in FIG. 8 is stored as a high-quality character image and the samples as shown in the character images 803 and 804 are stored in the recognition learning DB 321 as a deteriorated character image. Good. In this case, the character feature extraction unit 301 extracts a vector x representing the feature from each of the high-quality character images such as the character image 802, and the recognition dictionary generation unit 322 determines the similarity for each character type regarding the high-quality character image. A function fi (x) for calculating the degree is generated and stored in the character recognition dictionary 303. The function fi is generated by learning by an algorithm such as a support vector machine so that a high value is output when a vector x extracted from a high-quality character image representing the i-th character type is input. Further, the character feature extraction unit 301 extracts a vector x representing the feature from each deteriorated character image such as the character image 803 or 804, and the recognition dictionary generation unit 322 determines the similarity for each character type with respect to the deteriorated character image. A function fi (x) for calculation is generated and stored in the character recognition dictionary 303.

  For example, the similarity calculation unit 302 normally calculates the similarity with each character type of the input character image using a function fi (x) for calculating the similarity regarding a high-quality character image, If the retry determination unit 316 determines to retry the process, and it is determined that the degree of deterioration due to blur exceeds a predetermined threshold, the degree of similarity regarding the character image 803 or 804 with the degree of deterioration due to blur being 1 or 2 Using the function fi (x) for calculating, the similarity with each character type of the input character image is calculated.

  The category k deterioration determination dictionary generation unit (that is, the category 1 deterioration determination dictionary generation unit 324, the category N deterioration determination dictionary generation unit 326, and the like) includes a category k deterioration learning DB (that is, a category 1 deterioration learning DB 323 and a category N). Using the deterioration learning DB 325 and the like), a category k deterioration determination dictionary (that is, a category 1 deterioration determination dictionary 306 and a category N deterioration determination dictionary 312) used by the character string recognition unit 300 is generated. The category k deterioration determination dictionary stores a function for calculating the category k deterioration degree of the input vector x. The function for calculating the degree of deterioration may be prepared for each character type to be read, or one function common to all the character types to be read may be used.

  Hereinafter, a method of generating a function for calculating the degree of deterioration when one function common to all the character types to be read is used will be described. When generating a function for each character type to be read, the following procedure may be performed for each character type.

  In the category k deterioration learning DB, a deteriorated character image is stored together with a numerical value indicating the degree of deterioration. For example, when the category 1 is “blur”, the character image deteriorated by the blur is stored in the category 1 deterioration learning DB 323.

  FIG. 8 is an explanatory diagram of a deteriorated character image handled in the embodiment of the present invention.

  The character image 801 is an example of a deteriorated character image. For example, in a character string of “Delivery date Heisei”, the character image of “Delivery date” has disappeared and some lines disappeared due to blurring. There are interruptions. On the other hand, the character image of “Heisei” did not deteriorate such as blurring or crushing.

  Character images 802 to 804 show examples of character images stored in the deterioration learning DB.

  The character image 802 is an example of a character image having a small degree of deterioration (or no deterioration). For example, the character image 802A is an image of the character type “7”, and the character type label 802B indicates that the character image 802A is an image of the character type “7”.

  Character images 803 and 804 are examples of character images in which deterioration due to blurring has occurred. The deterioration degree of the character image 803 is larger than the deterioration degree of the character image 802, and the deterioration degree of the character image 804 is further larger. Here, the deterioration degrees of the character images 802, 803, and 804 are described as 0, 1, and 2, respectively. Information indicating the degree of deterioration is also stored in the category k deterioration learning DB.

  Note that the character image and the information indicating the degree of deterioration similar to the character images 802, 803, and 804 may be stored in the recognition learning DB 321. If the character image 802 is a character image that has not deteriorated, the classification k deterioration learning DB may not include the character image 802.

  Similarly, character images in which other types of deterioration such as crushing have occurred are also stored in the category k deterioration learning DB corresponding to each type.

  The function g (x) for calculating the degree of deterioration of the input vector x uses the degree of deterioration as an objective variable and the vector x extracted by the category k feature extraction unit as an explanatory variable, and is regressed using the category k deterioration learning DB. Generated by doing learning. As a result, g (x) is a function having a tendency to output a larger value as deterioration of the same type as the character image used for learning increases.

  For example, when the category 1 is “blur”, the category 1 feature extraction unit 304 extracts a vector x representing the feature of the character image 803 (or 804). The category 1 degradation determination dictionary generation unit 324 performs regression learning using the category 1 degradation level as an objective variable and the extracted vector x as an explanatory variable, thereby calculating a function g (x ) And stored in the category 1 deterioration determination dictionary 306.

  Similarly, for example, when the category 2 is “collapsed”, the category 2 feature extraction unit 307 extracts a vector x representing the feature of the character image (not shown) that has deteriorated due to the collapse. The category 2 deterioration determination dictionary generation unit (not shown) is a function for calculating a category 2 deterioration degree by performing regression learning using the category 2 deterioration degree as an objective variable and the extracted vector x as an explanatory variable. g (x) is generated and stored in the category 2 deterioration determination dictionary 309.

  Thereafter, the function g (x) is calculated for the character image cut out from the document input to the image input unit 200 (specifically, for example, for the first candidate character type of the character image). When the inputted character image is faint, the function g (x) for calculating the category 1 deterioration degree is larger than the value of the function g (x) for calculating the category 2 deterioration degree. When the character image is crushed, the function g (x) for calculating the category 2 deterioration degree is larger than the value of the function g (x) for calculating the category 1 deterioration degree. As a result, it is possible to specify the type of deterioration occurring in the input character image and the degree of the deterioration.

  Rejection control dictionary generation unit 327 generates rejection control dictionary 314 used by rejection determination unit 313. The rejection control dictionary 314 determines whether to reject the recognition result and the function p that outputs a rejection score (real value) when the similarity for each character type to be read and the degradation level for each classification are input. And a threshold value used for the purpose. The threshold value may be different for each character type, or may be common to all character types. The calculation of the rejection score and the method of determination of rejection are as described for the description of the rejection determination unit 313.

  The function p and the threshold value used for determination of rejection determine in which case the recognition result is rejected, and are generated according to the setting of the user. For example, if some misreading is allowed, a function p that does not output a very large value even if the degree of deterioration increases may be generated, or if it is desired to prevent misreading due to blurring rather than misreading due to crushing The function p may be generated so that the output is likely to increase when the degradation level due to blurring increases, and the output does not increase too much when the degradation level due to collapse increases, or misreading due to squashing rather than misreading due to blurring When it is desired to prevent this, the function p may be generated such that the output tends to increase when the degree of deterioration due to crushing increases, and the output does not increase too much when the degree of deterioration due to blurring increases.

  According to the above-described embodiment of the present invention, the recognition result can be rejected with high accuracy, and the rejection result can be used for improving the accuracy of character recognition.

  For example, when a document image is subjected to the same preprocessing (binarization processing or the like) as in a part 801 of a form shown in FIG. 8, a difference in font, a difference in color, a difference in character size, or a difference in paper quality There are cases in which various deteriorated characters (such as faint or crushed) are mixed due to reasons such as. The character image 801 includes a character string image “deadline” in which fading has occurred and a character string image “Heisei” in which no fading has occurred. Therefore, it is necessary to change the preprocessing method depending on the position of the form. Even in such a case, since the preprocessing method and the feature extraction process can be changed for each character based on the degree of deterioration, the deteriorated character can be recognized with high accuracy. Further, since the processing method at the time of retry can be controlled by the degree of character deterioration, the deteriorated character can be recognized with high accuracy.

  Furthermore, since the rejection method specialized in the rejection is provided for each type of non-character and deteriorated character, it is possible to perform more precise rejection than in the conventional method. Further, the rejection strength can be adjusted for each category of non-characters and deteriorated characters.

  Furthermore, by providing a deterioration determination method for each type of non-character and deteriorated character, the cause of rejection can be identified and used for subsequent control (for example, at the time of retry).

DESCRIPTION OF SYMBOLS 101 Character recognition apparatus 102 Input apparatus 103 Display apparatus 104 Image acquisition apparatus 105 Communication apparatus 106 Arithmetic apparatus 107 External storage apparatus 200 Image input part 201 Preprocessing part 202 Character string extraction part 203,208,300,315 Character string recognition part 210 316 Retry determination unit 211, 317 Recognition result recording unit 212, 318 Recognition result DB
213, 319 Document classification unit 214 Recognition dictionary generation unit 320 Dictionary generation unit

Claims (10)

A character recognition device comprising a processor and a storage device connected to the processor,
The storage device
A character image for recognition, a recognition dictionary including a function for calculating the similarity between the character image and each character type, and a function for calculating the degree of deterioration of the character image for each type of deterioration. A plurality of deterioration determination dictionaries including,
For each type of degradation, hold a character image of each character type where degradation has occurred,
The character recognition device includes:
Extracting a numerical value indicating the characteristics of the character image for calculating the similarity,
Using the numerical value indicating the feature for calculating the extracted similarity and a function included in the recognition dictionary, the similarity for each character type of the character image is calculated,
For each type of deterioration, extract a numerical value indicating the characteristics of the character image for calculating the degree of deterioration,
For each type of deterioration, the deterioration degree of the character image is reduced by performing regression learning using the deterioration degree as an objective variable and a numerical value indicating the character image characteristic of each character type in which the deterioration has occurred as an explanatory variable. Generate a function to calculate for each type and save it in the deterioration determination dictionary,
Using the numerical value indicating the feature for calculating the extracted deterioration degree and a function included in the deterioration determination dictionary, the deterioration degree of the character image is calculated for each type of deterioration,
Based on the calculated degree of deterioration, determine whether to reject the recognition result of the character image,
The determination result is recorded in the storage device ,
When a document image is input, preprocessing including at least one of binarization or noise removal of the document image is performed,
A character string image is cut out from the preprocessed document image,
Cut out the character image from the character string image,
Based on the degree of deterioration calculated for each type of deterioration, determine whether to re-execute the character image recognition process,
When it is determined that the character image recognition process is to be re-executed, the pre-processing, the character string image cut-out, the character image cut-out, the extraction of numerical values indicating the characteristics of the character image, or the character types of the character image A character recognition apparatus characterized by re-calculating the similarity with respect to a parameter or a processing method changed according to the degree of deterioration . The kinds of deterioration include at least two kinds of deterioration due to blurring and deterioration due to crushing,
  The character recognition device is changed so that the binarization threshold is easily determined to be black or the isolated point noise removal processing is weakened when the degree of deterioration due to blurring is larger than a predetermined value. If the pre-processing is re-executed using at least one of the parameters and the degree of deterioration due to crushing is greater than a predetermined value, the binarization threshold is changed to be easily determined as white, or an isolated parameter The character recognition device according to claim 1, wherein the preprocessing is re-executed using at least one of the parameters changed so as to enhance the point noise removal processing. The character recognition device classifies a document including the character image according to at least one of the recognition result of the character image, the determination result of whether to reject the recognition result of the character image, or the degree of deterioration. The character recognition device according to claim 2. The character recognition device includes:
  For each type of deterioration, by performing regression learning with the similarity as an objective variable and numerical values indicating the characteristics of the character image of each character type in which the deterioration has occurred as an explanatory variable, according to the type of deterioration, Generating a function for calculating the similarity of the character image to the character type;
  If the degree of deterioration calculated for any of the types of deterioration is greater than a predetermined value, using a function for calculating the similarity to the character type of the character image generated according to the type of deterioration, the The character recognition apparatus according to claim 1, wherein similarity calculation for the character types of the character image is re-executed. The storage device holds a rejection control dictionary including a function for calculating a rejection score based on the similarity and the degree of deterioration for each type of deterioration,
  The character recognition device calculates the rejection score corresponding to the calculated similarity and the calculated deterioration using a function included in the rejection control dictionary,
  The character recognition device according to claim 1, wherein when the rejection score exceeds a predetermined value, it is determined to reject the recognition result of the character image. A character recognition method executed by a character recognition device comprising a processor and a storage device connected to the processor,
  The storage device
  A character image for recognition, a recognition dictionary including a function for calculating the similarity between the character image and each character type, and a function for calculating the degree of deterioration of the character image for each type of deterioration. A plurality of deterioration determination dictionaries including,
  For each type of degradation, hold a character image of each character type where degradation has occurred,
  The character recognition method is:
  A first procedure for extracting a numerical value indicating the feature of the character image for calculating the similarity;
  A second procedure for calculating a similarity for each character type of the character image using a numerical value indicating a feature for calculating the extracted similarity and a function included in the recognition dictionary;
  A third procedure for extracting a numerical value indicating characteristics of the character image for calculating the degree of deterioration for each type of deterioration;
  For each type of deterioration, the deterioration degree of the character image is reduced by performing regression learning using the deterioration degree as an objective variable and a numerical value indicating the character image characteristic of each character type in which the deterioration has occurred as an explanatory variable. A fourth procedure for generating a function for calculating each type and storing the function in the deterioration determination dictionary;
  A fifth procedure for calculating the degree of deterioration of the character image for each type of deterioration using a numerical value indicating a characteristic for calculating the extracted degree of deterioration and a function included in the deterioration determination dictionary. When,
  A sixth procedure for determining whether or not to reject the recognition result of the character image based on the calculated deterioration degree;
  A seventh procedure for recording the determined result in the storage device;
  When a document image is input, an eighth procedure for performing preprocessing including at least one of binarization or noise removal of the document image;
  A ninth procedure for cutting out a character string image from the preprocessed document image;
  A tenth procedure for cutting out the character image from the character string image;
  An eleventh procedure for determining whether to re-execute the character image recognition process based on the degree of deterioration calculated for each type of deterioration,
  In the eleventh procedure, when it is determined that the character image recognition process is to be re-executed, the first procedure, the second procedure, the eighth procedure, the ninth procedure, or the tenth procedure is changed to the deterioration. A character recognition method which is re-executed using a parameter or a processing method changed according to the degree. The kinds of deterioration include at least two kinds of deterioration due to blurring and deterioration due to crushing,
  In the eleventh procedure, when it is determined that the character image recognition process is to be re-executed, the eighth procedure to be re-executed is that the binarization threshold is set to black when the degree of deterioration due to blurring is larger than a predetermined value. The pre-processing is re-executed using at least one of a parameter that has been changed so that it can be easily determined, or a parameter that has been changed to weaken the isolated point noise removal processing, and the degree of deterioration due to crushing is greater than a predetermined value. If larger, the pre-processing is re-executed using at least one of a parameter changed so that the binarization threshold is easily determined to be white and a parameter changed so as to enhance the isolated point noise removal processing. The character recognition method according to claim 6, further comprising a procedure. The character recognition method further includes a document including the character image according to at least one of the recognition result of the character image, the determination result whether to reject the recognition result of the character image, or the degree of deterioration. The character recognition method according to claim 7, further comprising a twelfth procedure of classifying. The character recognition method further performs, for each type of deterioration, regression learning using the similarity as an objective variable, and numerical values indicating characteristics of the character images of the character types in which the deterioration has occurred as explanatory variables. Generating a function for calculating the similarity of the character image to the character type according to the type of deterioration,
  When it is determined in the eleventh procedure that the character image recognition process is to be re-executed, the second procedure to be re-executed is when the degree of deterioration calculated for any of the types of deterioration is greater than a predetermined value. And re-calculating the similarity of the character image for each character type using a function for calculating the similarity of the character image to the character type generated according to the type of deterioration. The character recognition method according to claim 6. The storage device holds a rejection control dictionary including a function for calculating a rejection score based on the similarity and the degree of deterioration for each type of deterioration,
  The sixth procedure includes
  A procedure for calculating the rejection score corresponding to the calculated similarity and the calculated deterioration level using a function included in the rejection control dictionary;
  The character recognition method according to claim 6, further comprising: a step of determining that the recognition result of the character image is rejected when the rejection score exceeds a predetermined value.