KR20090055087A - Method and system for evaluating document image automatically for optical character recognition - Google Patents

Abstract

A method for automatically evaluating a document image for the OCR and a system thereof are provided to obtain a clear image by extracting features of the document image and then offering an evaluation score and feedback to a user. A system(100) comprises a feature data extractor(110), a feature data evaluating unit(120), a scanner control unit(130), an interface unit(140), a communication unit(150) and a control unit(160). The feature data extractor extracts feature data which shows the attribute of a document image related to a character recognition rate of the OCR(Optical Character Recognition) from a document image. A feature data evaluating unit evaluates the extracted feature data. So as to re-set a scanner according to the evaluated feature data, the scanner control unit feeds back a scanner control method to a user.

Description

TECHNICAL AND SYSTEM FOR EVALUATING DOCUMENT IMAGE AUTOMATICALLY FOR OPTICAL CHARACTER RECOGNITION}

The present invention relates to a method and system for automatically evaluating a document image for OCR (Optical Character Recognition). Specifically, the document image is quickly extracted by quantifying and evaluating the feature of the document image before recognizing the document image. The character recognition rate of can be predicted, and the user can adjust the factors of the scanner or proceed to OCR preprocessing process by informing the user with the feedback of the score and the expected recognition rate for each item so that the character recognition rate of the document image can be easily performed without trial and error. The present invention relates to a method and system for improving the system.

In general, in order to digitize a document recorded on paper, an OCR (Optical Character Recognition) technique that scans a document and recognizes a document image is used. However, even when OCR is used, it is difficult to obtain a perfect image due to the preservation state of the document itself such as discoloration and noise generated during scanning, so that it is difficult to expect a high character recognition rate for the document image.

In Korea, the document recognition system that has been commercialized in recent years tends to focus on the recognition of Hangeul, and it is common to attempt to recognize the phoneme units after separating Hangeul characters into initial, neutral, and final characters using structural characteristics. to be. Such a system has a problem that it can function properly without relatively high resolution and noise when scanning an original document. Therefore, if the original document is not clean or the size of the character and the resolution of the scanner are not appropriate when scanning, it may be impossible to recognize.

Accordingly, prior to performing the recognition of the characters included in the document image, efforts have been made to remove unnecessary elements such as noise of the document image through the preprocessing process. However, even after the preprocessing process, there may be a disturbing factor in the recognition, and as a result, the final recognition rate of the document may be lowered due to such an obstacle. As a result, efficient processes have not been achieved.

Accordingly, an object of the present invention is to solve the problems of the prior art and to quickly predict the character recognition rate of a document image by extracting the feature of the document image and evaluating the numerical value by item before recognizing the document image generated by scanning the original document. By giving the feedback of such item-specific score to the user, the user can reset the resolution, brightness of the scanner, correct the skew of the document, or proceed to the actual OCR preprocessing process, so that the document image can be easily performed without trial and error. This is to improve the character recognition rate.

In order to achieve the object of the present invention as described above, and to perform the characteristic functions of the present invention described below, the characteristic configuration of the present invention is as follows.

According to an aspect of the present invention, a method for obtaining an expected recognition rate for a character before performing a recognition process using an optical character recognition (OCR) technique for a document image, which is an image file containing characters, comprising: (a) the Extract at least one feature data representing an attribute of the document image related to a character recognition rate of an OCR from the document image, and determine how suitable the feature data is for character recognition by the OCR for each of the extracted feature data. Obtaining a numerical score for each item, (b) a weight for each score for each item, wherein the weight is determined to be larger as the influence of the feature data on the document image is greater in the character recognition. (C) adding the weighted item-specific scores to the expected It provides a method comprising providing a recognition rate.

According to the present invention, the feature of the document image is extracted and the evaluation score and feedback are provided to the user, thereby directly controlling the scanner to obtain a clear image.

In addition, according to the present invention, since the feedback is provided by expressing the extracted feature as a numerical value for each item, it is helpful for the actual OCR preprocessing process of the document image, and the evaluation result is displayed as a score, so that the recognition rate is explicitly recognized even before the image is recognized. The effect is that estimation is possible.

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating extracting feature data indicating a property of a document image related to a character recognition rate of an OCR from a document image, and extracting the feature data from the document image before performing a character recognition process. It is a block diagram of the entire system 100 for predicting the character recognition rate of the document image in a short time by digitizing and evaluating each item.

Referring to FIG. 1, the entire system 100 includes a feature data extractor 110, a feature data value calculator 120, a scanner controller 130, an interface unit 140, a communication unit 150, and a controller 160. ) May be included.

According to the exemplary embodiment of the present invention, the feature data extractor 110, the feature data numerical calculator 120, the scanner controller 130, the interface 140, the communicator 150, and the controller 160 are At least some of them may be program modules included in the user terminal device or in communication with the user terminal device. However, in FIG. 1, the feature data extractor 110, the feature data numerical calculator 120, and the scanner controller 130 may be used. The interface unit 140, the communication unit 150, and the controller 160 are all included in the user terminal device. Such program modules may be included in the user terminal device in the form of an operating system, an application module, and other program modules, and may be physically stored on various known storage devices. Also, such program modules may be stored in a remote storage device that can communicate with a user terminal device. Such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

The feature data extractor 110 extracts feature data representing a property of the document image related to the character recognition rate of the OCR from the document image. Here, the feature data is the degree of tilt of the image, the degree of clustering repetition in the binarization of the image, the ratio of noise included in the image, the degree of setting the text area of the image, the degree of character division in the text area, the degree of brightness contrast in the character division area, The amount of repetition in the character thinning process and the size of the loop character such as 'ㅇ'.

In addition, the feature data numerical calculator 120 calculates and evaluates the feature data extracted by the feature data extractor 110, and generates a weighted sum by combining the feature data with a predetermined weight.

Meanwhile, the scanner controller 130 feeds back a scanner control method to the user to reset the scanner according to the digitized feature data.

The interface unit 140 notifies the user of the digitized feature data value and the weighted sum, etc. on the screen of the digital device, and receives a response from the user about whether to proceed with the OCR process.

The communication unit 150 is responsible for transmitting and receiving signals between each component module in the system 100 or transmitting and receiving data with various external devices.

The controller 160 according to the present invention controls the flow of data between the feature data extractor 110, the feature data numerical calculator 120, the scanner controller 130, the interface unit 140, and the communication unit 150. Perform the function. That is, the control unit 160 according to the present invention by controlling the signals transmitted and received between each component through the communication unit 150, the feature data extraction unit 110, feature data numerical calculation unit 120, scanner adjusting unit 130 ), The interface unit 140 controls to perform each unique function.

2 is a flow diagram illustrating the overall time series flow for operating a system for resetting a scanner using a pre-evaluation value of a document image, in accordance with one embodiment of the present invention.

First, a digital device such as a scanner generates a document image by scanning an original document (S210).

Skew of the image from the generated document image, Clustering Iteration (BI) when binarizing the image, Noise Ratio (NR) included in the image, and setting of the text area of the image (Zone) Detection: ZD), Segmentation Difficulty (SD) in Text Areas, Segmentation Contrast (SC) in Text Areas, Thinning Iteration (TI), ' ㅇ 'character is extracted using the size of loop character (SLC) such as character as an evaluation item, and the score is calculated by digitizing each item (S220).

The calculated scores for each item are provided as feedback to the user as a total score, that is, the expected recognition rate, that is summed by applying respective weights that are optimally determined according to the type of document image (S230).

Thereafter, the system 100 detects whether a response to the OCR from the user is input (S240). In inputting the response, the user may refer to the expected recognition rate value calculated in step S230.

If it is determined that the expected recognition rate is appropriate and the user inputs an OCR to proceed, the system 100 proceeds to the OCR process (S250). The OCR process consists of (1) a preprocessing process including processes such as gray conversion, black and white conversion, and skew correction, (2) a structural analysis process including processes such as layout analysis, classification and recognition of text / pictures / tables, ( 3) a character segmentation process, (4) a character recognition process including a feature extraction process, and (5) a post-processing. The OCR technology itself is a generally known technology, so the detailed description thereof will be omitted here. do. Through the process of (1) to (5) as described above, it is possible to know how much the final recognition rate is based on the result of the recognition of the characters.

When the information on the final recognition rate is obtained, the correlation between the expected recognition rate calculated in step S230 and the final recognition rate may be examined, and the expected recognition rate and the final recognition rate may be substantially the same according to the weight applied to each evaluation item. will be.

On the other hand, in step S240, the user may think that the expected recognition rate is low, in this case, in order to further increase the character recognition rate for the document image in the OCR process, adjust the factors of the scanner without starting the OCR process Can be. In this case, the controller 160 proposes a scanner control method based on the score for each evaluation item calculated in step S220 (S270). For example, the controller 160 suggests a specific scanner control method such as allowing the user to control the automatic document feeder of the scanner, adjust the brightness of the scanner, or increase the resolution of the scanner by referring to the score of each evaluation item. can do. Of course, the present invention is not limited thereto, and may be adjusted to be in an appropriate state automatically without presenting the above information to the user.

After the various factors of the scanner are adjusted according to the information provided in step S260, the process may return to step S210 and scan the original document again to obtain an expected recognition rate.

3 is a detailed flowchart illustrating a process of performing evaluation of a document image for each item according to an embodiment of the present invention.

The degree of tilt of the image (S302: angular characteristic extraction) for the document image generated by scanning the original document, the clustering repetition degree (S303: BI characteristic extraction) when binarizing the image, and the ratio of noise included in the image (S304: NR characteristic) Extraction), setting degree of the text area of the image (S305: ZD characteristic extraction), character division degree in the text area (S306: SD characteristic extraction), brightness contrast degree in the character division area (S307: TI characteristic extraction), character The value of the repetition in the thinning process (S308: SC characteristic extraction) and the size of the loop character such as the 'ㅇ' character (S309: SLC characteristic extraction) are calculated using the evaluation items, respectively (S310), and each evaluation item. When the scores are derived, the weights are assigned according to the order in which they have a significant influence on the recognition, and then the sums are added to calculate the final evaluation score (that is, the expected recognition rate) of 100 points (S311).

Specifically, the degree of inclination (S302) of the document image is a problem to be considered in the scanning process of the image. If the distortion is severe due to the document image tilt, the document recognition itself is impossible, so it is necessary to correct the tilt. FIG. 4A illustrates a skew image, and for this reason, the expected recognition rate may be calculated to be low, and in this case, the feedback may be adjusted to a correct image corrected for tilt as shown in FIG. 4B.

The left region of FIG. 5 represents a tilt allowable range of the document image, and the right region of FIG. 5 shows an example of extracting an angle from four portions of the image.

In order to recognize the document image, the skew image must be correctly returned. In order to return the document image correctly, it is necessary to know the angle of tilt of the document image. This angle is usually judged as the inclination of the upper part of the document, which is insufficient to be used as the inclination of the entire document, and thus the upper, lower, left, and right parts of the document as shown in the right area of FIG. It is preferable to detect the angle at and then take the average of these angles. However, at this time, an angle having a large difference from the remaining angles of the four parts may be determined as an error angle and excluded, and even if there is an angle that is out of the ± 5 degree range, the average angle may be calculated. Can be. In general, a scanned image of a document has an inclination close to zero degrees. Therefore, even if the angle is inclined by only 1 degree, the distortion of the character is severe, so that the document image outside the range of ± 5 degrees can be judged to be a wrong scan and treat the score as 0. By default, the angle causes the clockwise direction to show a positive value. In the tilt evaluation of the image, the score may be calculated differentially every 0.5 degrees in the range of ± 5 degrees with respect to 0 degrees.

Step S303 of Figure 3 refers to the degree of clustering iteration (Binarization Iteration, BI) during the binarization of the image. This process converts a color image into a grayscale image to recognize a document image, and then converts the image into a binary image composed of black representing letters and white representing backgrounds rather than letters. to be.

The method of converting gray image composed of 0 ~ 255 steps into binary image basically has Global Thresholding which uses the average of the whole image as threshold.But binary image has to be distinguished from text color and background color. A fixed threshold of thresholding will not always yield good results. To remedy this problem, the author of Andreas E. Savakis, published in IEEE Proceedings of international Conference on Image Processing (ICIP'98), published in 1998, published in "Adaptive Document Image Thresholding Using Foreground and Background Clustering," This can be done using Foreground and Background Clustering (FBC) technology, which finds a fluid threshold while clustering text and background colors.

It is difficult to obtain a good binary image in the case of a discoloration of a document or a dark background color that makes it difficult to distinguish between a letter and a background. According to the FBC technique described above, the average value is found while clustering letters and background pixels. When the image of discoloration or dark background color is converted by FBC method, the clustering process is more than that of general image. The binarization evaluation of an image calculates a score based on the number of iterations of the clustering process.

6 shows a pseudo code for returning the number of clustering repetitions. First, the average value of the entire image is obtained, and black and white pixels are classified based on the average value. The black and white pixels are further classified into average values of pixels classified as black pixels and average values of pixels classified as white pixels. This process is repeated until each mean value does not change. Accordingly, the evaluation is inversely proportional to the number of FBC repetitions.

In step S304 of FIG. 3, a noise ratio (NR) included in an image is represented.

When noise is mixed in the text portion of a document image, it may be recognized as a completely different character. Determine if this noise is part of a letter and remove the noise if it is not a letter part. In order to remove noise, a low pass filter, a median filter, and a smoothing may be applied. The median filter may be effective to preserve strong edges and details of the original image, but the present invention is not limited thereto.

However, even if the median filter is used, noise cannot be completely removed. In the noise evaluation item, a score is calculated by determining the ratio of pixels determined as noise in the median filter applied image. The ratio of noise can be expressed as follows.

In operation S305 of FIG. 3, a setting degree (Zone Detection, ZD) of the text area of the image is represented.

When classifying a document image using the projection profile, the picture area should be set to one area, and the text area may be set to one area of more than one paragraph. However, there are cases where the set area is too small or too large due to irregular letter spacing and line spacing. The left region of FIG. 7 shows an example of a text area that is set too small as a result of region classification, and the right region of FIG. 7 shows an example that is correctly set.

That is, according to step S305, after the inclination of the document image is corrected through the preprocessing of the document image, the document image may be divided into meaningful regions, each of which is text or graphic information (pictures, tables, lines, etc.). It may include. To this end, step S305 may include extracting a geometric structure from the document image to divide the document image into a set of various regions, and classifying the region into a text and non-text region by examining the characteristics of each region. Area classification generally generates a profile by projecting in the horizontal and vertical directions of an image and dividing the area based on this.

In the evaluation based on the area classification, the ratio of the area in which the width and height of the area set as a result of the classification of the document image is less than the reference in the whole area is converted into a score. That is, the ratio of the incorrectly set area is evaluated.

In step S306 of FIG. 3, the degree of character division (SD) in the text area is represented.

The text area extracted through document structure analysis is divided into character units that are subject to recognition. Generally, the projection profile method is applied or written by Yu Seok, Kim Soo Hyung, and two others, 2002 Journal of Information Science, Vol. 20, No. 20 You can apply the method of connected component analysis published in "Document Image Processing Technology and Digital Library" published in No. 8, pp.24-34. This will be described in detail with reference to FIGS. 8 and 9 below.

8 illustrates a division result of a character unit in a text area.

In the area division, the area divided into the character areas must go through the process of character division so that the recognizer can recognize it. Character segmentation is the final process of the preprocessing process, and thus has the most influence on recognition. Therefore, abnormal character segmentation may result in completely different recognition results. In the case of Korean character division, the divided rectangle should have a similar ratio of width and height.

9 shows an abnormal character division in which letters are so narrow that the characters are combined. When the 's' and' p's are combined, it can be seen that the height and width of the divided rectangles show a significant difference.

In the character division evaluation, character division is attempted for areas set as text in the area classification to identify abnormally divided characters. The score is calculated by calculating the ratio of abnormal character division to total character division.

In step S307 of FIG. 3, the brightness contrast degree (Segment Contrast, SC) in the character segmentation region. A value using the following equation is calculated to evaluate the degree of contrast in the divided characters.

As the calculated value of the following equation, that is, the brightness contrast value is closer to 1, it means that the text and the background are clearly separated, and the character recognition rate of the document image is increased in proportion to the brightness contrast value.

Step S308 of FIG. 3 refers to the degree of repetition (Tinning Iteration, TI) in the character thinning process, which measures and evaluates the number of thinning repetitions for making the thickness of the character 1 while performing the thinning process. One of the methods for measuring the thickness of. 10 shows the character thinning process. The left image of FIG. 10 shows that one character is extracted from a document image. The middle image of FIG. 10 standardizes the extracted character, and the right image of FIG. 10 shows that the thickness of the character is thinned to 1. FIG. .

Step S309 of FIG. 3 refers to a process of measuring the size of a character having a loop shape such as 'ㅇ' (Size of Loop Character: SLC), and FIG. 11 shows the SLC process.

In the upper region of FIG. 11, a process of dividing into character units through analysis of a connection element of a character is illustrated, and in the lower region of FIG. 11, loop characters such as ㅁ, ㅂ, ㅇ (ㅎ) and ‰ are extracted. The result is shown. The SLC characteristic value can be determined based on the measured loop size.

FIG. 12 is an exemplary diagram illustrating a screen of a result of evaluating one document image by using an automatic evaluation method applying an evaluation item corresponding to steps S302 to S309 of FIG. 3.

The left area of FIG. 12 illustrates a state in which the document image is divided into two text areas and one picture area.

Embodiments according to the present invention can be implemented in the form of program instructions that can be executed by various computer means can be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In the present invention as described above has been described by the specific embodiments, such as specific components and limited embodiments and drawings, but this is provided to help a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations are possible from these descriptions.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and all the things that are equivalent to or equivalent to the claims as well as the following claims will belong to the scope of the present invention. .

1 is a block diagram illustrating an automatic evaluation system of a document image for OCR.

2 is a flowchart illustrating a process of performing evaluation of a document image.

3 is a flowchart illustrating a process for obtaining an expected recognition rate using a plurality of characteristic data reflecting attributes of a document image.

4A illustrates a skew image, and FIG. 4B illustrates a correct image through calibration.

5 is a diagram illustrating an example of determining an inclination degree with respect to an inclined image.

6 shows a pseudo code for returning the number of clustering repetitions.

7 is a diagram illustrating a region classification result for a document image.

8 illustrates a result of character unit division in a text area of a document image.

9 is a diagram illustrating an example in which character division is abnormally divided due to too narrow a letter spacing.

10 is a diagram illustrating a character thinning process.

11 is a diagram illustrating an example of detecting a loop portion through an SLC process.

12 is an exemplary view illustrating a result screen of evaluating one document image by using an automatic evaluation method to which an evaluation item is applied.

<Description of the symbols for the main parts of the drawings>

110: feature data extraction unit

120: characteristic data numerical calculation unit

130: scanner control

140: interface unit

150: communication unit

160: control unit

Claims (18)

As a method for obtaining an expected recognition rate for a character before performing a recognition process using an optical character recognition (OCR) technique for a document image, which is an image file containing characters, (a) extracting from the document image at least one feature data indicating an attribute of the document image related to the character recognition rate of the OCR, wherein the feature data is to some extent in character recognition by the OCR for each extracted feature data; Obtaining a score for each item quantifying whether it is in a suitable state; (b) applying a weight for each score for each item, wherein the weight is determined as a larger value as the attribute itself of the feature data has greater influence in performing character recognition on the document image; and (c) adding the scores for each item to which the weight is applied to provide the expected recognition rate. The method of claim 1, In step (c), And providing the item-specific score. The method of claim 2, The document image, Method characterized in that it is generated by scanning the original document rather than digital data. The method of claim 2, and (d) adjusting factors of a scanner for reproducing the document image with reference to the expected recognition rate and the item-specific score. The method of claim 2, (d) presenting a scanner control method to the user to reset the scanner according to the expected recognition rate and the item-specific score. The method of claim 5, The method of resetting the scanner comprises at least one of controlling the automatic document feeder of the scanner, adjusting the brightness of the scanner or increasing the resolution of the scanner. The method of claim 2, (d) if the expected recognition rate is greater than or equal to a predetermined threshold, further comprising performing an OCR process. The method of claim 2, and (d) receiving a determination as to whether to perform an OCR process from the user after the expected recognition rate is provided to the user. The method of claim 1, In the step (a), The feature data may include skew of an image, clustering repetition degree (BI) of binarization of an image, ratio of noise included in an image (NR), setting degree of a text area of an image (ZD), and characters in a text area. Include at least one of the degree of division (SD), the degree of brightness contrast in the character division region (SC), the degree of repetition in the character thinning process (TI), and the size of the loop character (SLC) such as 'ㅇ'. How to feature. The method of claim 9, The skew of the document image is determined by detecting angles of four portions (Left, Right, Top, Bottom) of the document image and referring to the average of these angles. The method of claim 9, The clustering repetition degree (BI) during binarization of the document image is determined by referring to the number of clustering repetitions in the binarization process using FBC. The method of claim 9, And a ratio (NR) of noise included in the document image is determined by referring to a ratio of noise among all images of the document image. The method of claim 9, And a setting degree (ZD) of a text area of the document image is determined by referring to a ratio of areas in which the width and height of the area set as a result of the classification of the document image are less than a reference value in the entire area. The method of claim 9, Character division degree (SD) in the text area of the document image, After the character segmentation is performed in the text region, if the width or height of the segmented character region shows a difference greater than or equal to a predetermined threshold compared to the average character width or height of the characters included in the document image, it is determined to be an abnormal character segmentation region. If it is said, it is determined by referring to the ratio of the abnormal character segmentation area. The method of claim 9, The brightness contrast degree SC in the text division region of the document image is And is determined with reference to the calculated value of the equation. The method of claim 9, The repetition degree (TI) in the character thinning process of the document image is And determining the number of repeated thinnings to make the thickness of the character 1. The method of claim 9, The size of the loop character (SLC) of the document image is And extracting the loop part by dividing a character into a phoneme unit through a connected component analysis. A computer-readable medium for recording a computer program for executing the method according to any one of claims 1 to 17.

Images