JP5337563B2 - Form recognition method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To leave only the pixels of characters behind while dropping out pixels of other colors in a situation where the colors of the characters and ruled lines are unstable. <P>SOLUTION: Color clustering of the pixels of an input image is performed and the colors of the background and characters are determined based on the number of pixels in a cluster. Colors close to the colors of the characters in the image are left behind to create binary images of the characters. By dropping out the colors other than the colors of the characters, it is possible to drop out regardless of the colors of noises such as ruled lines or the hues of false colors. A determination is made as to whether or not pixels where colors are mixed because of the crossing of the characters and the ruled lines, and the false colors in the characters, should be dropped out in relation to surrounding pixels. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an image processing technique such as an OCR (Optical Character Reader), and more particularly to a technique for dropping out a background color or the like in a situation where a color to be removed is indefinite.

  In order to recognize a character on a form by OCR, it is necessary to extract only a character component from an image read by an image scanner or the like. In order to extract a character component, it is necessary to separate a noise component such as a ruled line, a background, and a character component. In general, a binarization process is used to separate a character and a background. By binarization, a binary image in which a character component with low luminance is black and a background portion with high luminance is white is obtained. When the color of a noise component such as a character component and a ruled line is different, dropout processing is used before binarization. The dropout processing is processing for preventing a specific color from appearing as an image in a printed or printed form or document image.

There are the following three types of typical conventional techniques for dropout processing.
(1) The specified red or blue color is optically specified as either a red or blue dropout noise component, and an image captured with a light source having the same color as the noise component is used for character recognition. For example, if a form with black characters and red ruled lines is imaged with a red light source, the ruled line portions do not appear in the image. The same applies to the blue system. Many existing hardware OCR products use this method.
(2) Dynamically selecting an optimal dropout color in RGB After picking up a color image and selecting a color having the highest luminance of the noise component among the three RBG colors, the gray value of the pixel in that color There is a method of binarization using. There is Patent Document 1 as a conventional example of this method.
(3) Separation by density of gray image This method uses a gray image without using color information. There is a method of taking a histogram of luminance values in a grayscale image and separating them on the assumption that the brightest area is a background, the darkest area is a character, and the middle is a noise component. There exists patent document 2 as a prior art example of this system.

JP 2003-196292 A JP 2004-94427 A JP 2007-156664 A

An object of the image processing apparatus of the present invention is to separate a character component and other components using color information and shading information in an image. Furthermore, the object is to drop out correctly even when the combination of the color of the character component and the noise component is different for each image. Problems to be solved in realizing this processing are as follows.
(A) Character components and noise components vary in color, and combinations of these colors differ from image to image When processing forms issued by different organizations at one time, the colors of characters and noise components such as ruled lines There is no guarantee that they are the same. Therefore, it is necessary to drop out components other than characters in a state where forms with different characters and noise components are mixed. In this case, since the color to be dropped out cannot be designated in advance, the method of the conventional method (1) cannot be applied.
(B) Colors that are difficult to separate with RGB Conventional method (2) cannot be applied to forms that include colors that are difficult to separate with RBG information, such as brown and purple, and forms that have high gray values.
(C) A plurality of noise component colors As in the conventional method (2), in the method of separating with RGB, there are cases where dropout cannot be performed when noise components of a plurality of colors are included. For example, when red and blue noise components are included, only one of the colors can be dropped out.
(D) High-density noise component color The conventional methods (1), (2), and (3) use binarization processing based on the gray value of a pixel. Therefore, when the noise component is as high as the character component, the noise component cannot be dropped out because they cannot be separated.
(E) Noise component of similar color When the character color and the noise component color are similar colors, the conventional methods (1) and (2) focusing on the color difference cannot drop out the noise component. In some cases, it is possible to drop out by focusing on the concentration. However, with this method, when noise of the same color as the character and noise of the same color and different color are mixed, the latter noise component cannot be dropped out.
(F) False color Due to the characteristics of the image sensor of the scanner, a color (false color) different from the original color may occur near the boundary of characters, ruled lines, and the like. In the conventional methods (1) and (2), dropout processing is performed by paying attention to the color to be removed, so that there is a problem that a false color portion that is different from the color to be removed is not removed. Further, when a false color occurs in the character, there is a problem that the pixel is dropped out.

  In one example of the present invention, there is provided an image processing method for dropping out pixels other than those constituting a character in a processed image, the step of inputting the processed image, and the pixels of the processed image in the first color space. Clustering, and identifying a background color and a character color based on the color clustering. In the step of identifying the background color and the character color, for example, the background color can be identified from the pixels belonging to the cluster having the largest number of pixels or the cluster having the highest brightness. In addition, a cluster including a character color is selected from clusters identified as not having a background color. For example, the character color can be identified from the pixel belonging to the cluster having the second largest number of pixels or the cluster having the lowest brightness.

  Furthermore, in addition to the background color and character color identified above, each pixel of the processed image is used as a background candidate, a character candidate, and a removal candidate using an element other than RGB (for example, hue and saturation) of the pixel. And so on.

  As the first color space, various color spaces such as RGB, HSV, HLS, HSB, CMY, and CMYK can be used. When using the hue and saturation of a pixel, the pixel of the image is converted from the first color space to a second color space including dimensions such as hue and saturation other than RGB, such as HSV, HLS, and HSB. Can be processed. As another example, hue, saturation, and the like can be calculated from numerical values in the first color space each time processing is performed without performing color space conversion.

  According to one embodiment of the present invention, a noise component can be automatically dropped out of a form whose character component color and noise component color are indefinite. Furthermore, when the noise component is a similar color, or when there are a plurality of noise components, it can be dropped out even when the density is high.

  Embodiments of the present invention will be described with reference to the drawings. In addition, this invention is not limited by the following description.

  FIG. 1 is a diagram showing a configuration of a form recognition system according to an embodiment of the present invention.

  The form recognition system includes an input device 10, an image input device 20, an image processing device 30, a dictionary 40, a display device 50, and an image database (DB) 60.

  The input device 10 is a device such as a keyboard and a mouse for inputting commands and code data to the image processing device 30.

  The image input device 20 is a device such as a scanner for inputting a form as image data to the image processing device 30.

  The image processing apparatus 30 is a computer that detects a reading area of a form input by the image input apparatus 20 and performs a dropout process, and includes a CPU, a memory, and a storage device (not shown). The image processing apparatus can also execute processing such as character recognition of the dropout image.

  The dictionary 40 is a dictionary database that is referred to when the image processing apparatus 30 recognizes a form. Specifically, the dictionary 40 stores a character recognition dictionary that is referred to when the image processing apparatus 30 recognizes characters, form information that is referred to when a form reading area is detected, and the like.

  The display device 50 is a device such as a display that displays the result of the form recognized by the image processing device 30.

  The image DB 60 stores image data input to the image processing device 30 by the image input device 20. The image DB 60 may store in advance image data to be recognized by the image processing apparatus 30 by the image input apparatus 20.

  The present invention may be implemented in a normal computer by software having the same function as the image processing apparatus 30.

Before showing a specific example of the dropout method realized in the present invention, an outline of the processing result of the present invention will be described using an example of a date stamp shown in FIG.
In FIG. 2A, 2000 represents a region to be processed, 2010 represents a stamp, 2020 represents a ruled line on the form, and 2030 represents noise. The dropout process is a process for generating the dropout image of FIG. 2B from the input image of FIG. In the image (b) that has been dropped out, only the pixel (2040) of the color component of the seal remains in the area. Since FIG. 2 is an example of a stamp, the outline of the stamp is the same color as the character, so that a circular outline remains together.
In order to realize such processing, the present invention adopts the following method. First, characters, backgrounds, and other colors are clustered using information in the RGB color space. Next, it is determined whether or not it is a character color, and components of colors other than characters are dropped out. When determining whether or not the character color is used, not only the RGB color space information but also the HSV color space information is used. By using the information in the HSV color space, it is possible to separate high density noise components by hue.
Next, the outline | summary about the solution of the subject in this invention is demonstrated. Details will be described with reference to FIG.
(A) There are various colors of character components and noise components, and combinations of these colors are different for each form. After estimating the background color and character color, binarization processing is performed in which the character color is black and the others are white. Do. Since the text color is estimated for each document, dropout is possible even if the text color is not constant.
(B) Colors that are difficult to separate with RGB In order to drop out colors that are difficult to drop out even with simple RBG separation, such as brown and purple, not only RGB color information but also the brightness (V) of the HSV color space Or using hue (H).
(C) Multiple Noise Component Colors In the conventional method, there are many methods of selecting and dropping out the color to be removed, so there is a method that is difficult to cope with when there are a plurality of colors to be removed. In the present invention, a method for removing colors other than the character color is adopted. For this reason, even when there are a plurality of colors to be dropped out, determination processing for each color to be removed is unnecessary and does not depend on the number of colors to be removed.
(D) High-density noise component color In the conventional method, pixels with high density tend to become black by binarization processing. Since the present invention focuses on the difference from the character color, a pixel having a high density can be dropped out by removing pixels having a hue different from the character color.
(E) Noise component colors of similar colors When the character color and noise components are similar colors, dropout can be realized by separating the relatively lighter as the character color and the higher one as the noise color.
(F) False color Since the character color is left and other colors are removed, the false color is rarely left. When the pixel of the character becomes false color, it is possible to leave the false color in the character by determining whether it is the character color or not by considering the color of the surrounding pixels.

  Hereinafter, an embodiment of an image processing method and an image processing apparatus to which the present invention is applied will be described.

  FIG. 3 is a diagram showing a processing flow of an image processing method to which the present invention is applied. This is executed by an image processing device (CPU) 30. Usually, it is realized as a program executed by the CPU, and such a program can be stored in various recording media, stored in a memory and executed by the CPU.

  The processing area selection process (3000) is a process for selecting an area including characters to be read from an image. This processing may select a part of the image or the entire image.

  The color clustering process (3010) is a process for color clustering each pixel in the processing area. In color clustering, each pixel in an image is mapped onto a color space, and then close colors are made the same cluster based on a predetermined criterion.

  FIG. 4 shows an example of color clustering. FIG. 4 shows an example of mapping to the RGB color space, but other types of color spaces may be used. The RGB color space is a three-dimensional space having RGB values as axes. Usually, the value of each axis of RGB ranges from 0 to 255. The origin (0, 0, 0) represents black and (255, 255, 255) represents white. Each pixel in the processing target area can be mapped to one point on the RGB color space. Since characters, ruled lines, and background colors are never the same, a plurality of distributions centered on representative colors of each category can be created. In this way, clustering focusing on red, green, and blue colors can be performed in the RGB color space. In the example of FIG. 2, if the background is white, the color of the character (2010) is red, and the color of the ruled line (2020) is blue, in FIG. 4, 4000 is the background pixel distribution, 4010 is the character pixel distribution, 4020 is the distribution of ruled lines. In addition, other colors such as 4030 to 4050 may exist as noise and false colors. In color clustering, a cluster centered on the center color of each distribution or a color with the highest frequency is divided using Voronoi division, a threshold of Euclidean distance, or the like. Thereby, when dividing into three clusters, for example, it can be divided into three clusters centered on 4000, 4010, and 4020.

  The background color selection process (3020) is a process for selecting a background color from the result of color clustering. In the example of FIG. 4, after 4000 clusters are selected, the color at the center of the cluster distribution or the color with the largest number of pixels (frequency) is used as the background color. As an example of a selection criterion for a cluster including a background color, a cluster having the largest number of pixels is employed. This is because the area of the background is generally the largest among the characters, ruled lines, noise, and background in the processing target area. As another example of the selection criterion for the cluster including the background color, the cluster having the highest brightness can be selected. This is because the paper color has a higher brightness than the character color. As the brightness of the cluster, the average brightness of the pixels included in the cluster or the brightness of the peak or center of the distribution can be used. Note that when the brightness is used as a reference, an HSV color space or the like may be used instead of the RGB color space. The HSV color space will be described later with reference to FIG.

  The character color selection process (3030) is a process for selecting a character color from the result of color clustering. In the example of FIG. 4, after selecting 4010 clusters, the color at the center of the cluster distribution or the color with the largest number of pixels (frequency) is set as the background color. As an example of a selection criterion for a cluster including a character color, a cluster having the next largest number of pixels is employed after a cluster including a background color. The reason for this is due to the area occupied in the processing region, similar to the background color. Alternatively, there is a method of adopting a cluster having the lowest brightness.

  When the background and the character color are designated in advance, the cluster may be selected using the designated color information. For example, since the background is often white or yellow and the characters are black or blue in many cases, such information may be used.

  Note that the processing areas for color clustering in the background color selection process (3020) and the character color selection process (3030) may be the same or different. In the case of the same case, all the pixels in the processing area designated by 3000 may be used. As an example of a different case, the character color selection can be limited to a region where a character exists. This can be used when a character position such as a character on a form can be specified in advance. Further, when selecting a background color, it can be limited to an area where a background tends to exist.

The HSV conversion process (3040) is a process of converting the pixel to be processed from the RGB color space obtained from the input device to the HSV color space for use in the pixel classification process (3050).
The outline of the HSV color space will be described with reference to FIG. The HSV color space is a model that expresses colors by hue (H), saturation (S), and brightness (V), and can be visualized as a cone. Hue changes along the circumference of the cone. The vertical axis represents lightness, and the horizontal axis represents saturation. Conversion from the RGB color space to the HSV color space can be performed using mathematical formulas. The reason why the HSV color space is used in the present invention is that the expression of colors using the HSV color space is generally similar to a human color perception method using hue and brightness. Note that the conversion of the color space is not limited to HSV, but the HLS color space (represented by hue (H), luminance (L), saturation (S)) or HSB color space (hue (H), saturation (S)). , Expressed in brightness (B)) or the like. In addition, a CMY color space or a CMYK color space that is a subtractive color expression method used in the printing process may be used.

  By using dimensions other than RGB in this way, highly accurate pixel classification processing can be performed. In the above example, the following processing is performed after batch conversion of the color space. However, H, S, etc. are calculated from RGB each time without performing color space conversion. It is also possible.

  The pixel classification process (3050) is a process of classifying each pixel in the area for the purpose of dropout.

  The outline of this process will be described with reference to FIG. FIG. 6 is an enlarged view around the number “2” in FIG. 2. Each square represents a pixel. FIG. 2A shows a part of the input image. There are character pixels (such as 6000), background pixels (such as 6010), ruled line pixels (such as 6020), and pixels such as noise. Regarding noise, in addition to noise (such as 6030) that existed on paper, there is noise (such as 6040) that occurs during scanning, such as false colors. The false color is a color that is different from the actual color due to the characteristics of the image sensor or the optical system. Since false colors are likely to occur at locations where the colors are significantly different, they are often generated near the borders of ruled lines and characters. The image of FIG. 6A is input, and each pixel is classified as shown in FIG. The classification is first made by character (eg 6050) and background (eg 6060). This determination can be made using the background color and character color selected in 3020 and 3030 of FIG. If it cannot be clearly classified into these two, depending on the color and brightness of the pixel, it mainly means a character candidate (6070, etc.) that means false color and a ruled line, etc., because it is neither a character nor a background. It is determined as a removal candidate to be removed (6080 or the like). Details of the pixel classification processing will be described later with reference to FIG.

  The grayscale image generation process (3060) is a process for generating a grayscale image in which the luminance value of each pixel is corrected so that the character pixels remain in the subsequent binarization process (3070) and the other pixels are easily removed. It is. In the correction process, for each pixel divided into a character, background, character candidate, or removal candidate in the pixel classification process (3050), the luminance value of the pixel is determined based on the determination result and brightness of the pixel and surrounding pixels. To change. An outline of a method for correcting the luminance values of the background, characters, character candidates, and removal candidates is shown below. Background pixels are converted to white (luminance value 255) to ensure dropout. The luminance value of the character pixel is left as it is. Pixels for character candidates and removal candidates are set to reliability based on determination results and brightness of pixels around the pixel, and converted to white if the reliability is low. Increase the brightness value. If it is above the second reference, the luminance value is left as it is. Details of this processing will be described later with reference to FIGS.

  The binarization process (3070) is a process for binarizing the grayscale image generated by the grayscale image generation process (3060) into white and black. As a result, a binary image is generated in which the pixels of the background, ruled lines, noise, and the like are dropped out, leaving the character pixels. As a binarization method, many methods such as a method using a fixed threshold and a method of dynamically changing the threshold have been proposed. As a representative method, there is Otsu's binarization method. This binary image is used for processing such as character recognition.

  Hereinafter, the processing flow of the pixel classification (3050) in FIG. 3 will be described in detail with reference to FIG.

  First, in step 7000, the first pixel in the region is selected. The following determination process is performed on this pixel. Step 7010 is processing for determining a background pixel. Since the paper color is often white, assuming that the background color is white and the color of the pixel is close to white, in step 7020 the pixel is determined to be the background. Further, instead of white, a color close to the background color selected in 3020 of FIG. 3 can be determined as the background. In this embodiment, the determination result is recorded in the flag. As an example of determining whether or not it is close to white (or background color), there is a method of determining whether or not the Euclidean distance between white (255, 255, 255) and the pixel is equal to or less than a reference in the RGB space. . In the HSV color space, if the lightness is equal to or higher than the reference and the saturation is lower than the reference, it is determined to be white. If the determination condition in step 7010 is not satisfied, the process in step 7030 is performed. Step 7030 is a process for determining a background pixel other than white or a background color. Although the background is not limited to white, since the brightness is higher than that of characters, if the brightness of the pixel is equal to or higher than the reference, in step 7020, the pixel is determined as the background. As an example of the brightness scale, the luminance and brightness obtained from the RGB color space and HSV color space can be used. The background pixels can be determined by the processing so far.

Step 7040 is processing for determining whether or not the pixel is a character. If the pixel is a pixel close to the character color obtained in step 3030 of FIG. 3, the pixel is determined to be a character in step 7050. An example of whether or not the character color is close is determined as a character if the Euclidean distance between the pixel color and the character color in the RGB space is equal to or less than the reference.
If the determination condition in step 7040 is not satisfied, the process is divided in step 7060 depending on whether the character color obtained in step 3030 is an achromatic color (gray or black) or a chromatic color. This determination can use the saturation (S) in the HSV color space. When the saturation is low, the character color is close to black, and when the saturation is high, it can be determined that the character color has a color (a color).

  If it is determined in step 7060 that the character color is black, it is determined in step 7070 whether the pixel is a character. If the saturation of the pixel is the reference, it is close to a character color with low saturation, so that the pixel is determined to be a character in step 7050. If the determination condition in step 7070 is not satisfied, the pixel is determined as a character candidate in step 7080. A pixel determined as a character candidate is subjected to brightness correction processing described later with reference to FIG. This process will be described later with reference to FIGS.

  If it is determined in step 7060 that the character color is a color, it is determined in step 7090 whether the pixel is a character. If the character color is color, if the difference in hue between the pixel and the character color is below the reference, the pixel is determined to be a character in step 7050. By judging only the hue, there is an advantage that the difference in shading due to fading can be absorbed.

  If the condition in step 7090 is not satisfied, the pixel is determined to be a removal candidate in step 7100, assuming that the pixel has a color different from that of the character and noise such as a ruled line. A pixel determined as a character candidate is subjected to brightness correction processing described later with reference to FIG. This process will be described later with reference to FIGS.

  After the pixel has undergone any one of steps 7020, 7050, 7080, and 7100, it is determined in step 7110 whether all the pixels in the region have been processed. If not all have been processed, the next pixel is selected in step 7120 and the process returns to step 7010. If all are processed, the pixel classification process is terminated.

In FIG. 7, in order to drop out colors other than characters, the hue is determined in Step 7090 while paying attention only to the color of characters. However, when the color of a noise component such as a ruled line can be detected, a determination as to the hue of the noise component may be added.
In FIG. 7, images are classified into characters, backgrounds, character candidates, and removal candidates. However, the character candidates and removal candidates may be combined into one. That is, Step 7080 and Step 7100 are the same processing. In this case, in the grayscale image generation described with reference to FIGS. 8 to 11, the processing is performed assuming that the character candidate and the removal candidate are the same.

  Hereinafter, the processing flow of the grayscale image generation process (3060) in FIG. 3 will be described in detail with reference to FIGS.

  FIG. 8 is an overview of the entire gray image generation process. Since the result of classification in FIG. 7 is not necessarily correct, the reliability of the classification result is calculated, and a grayscale image is generated using the reliability. First, the reliability is set for the pixel determined as a character candidate in step 8000. Details of this processing will be described later with reference to FIG. Next, in step 8010, the reliability is set for the pixel determined as the removal candidate. Details of this processing will be described later with reference to FIG. Finally, in step 8020, the luminance value is corrected in accordance with the reliability of the pixels determined as the character candidate and the removal candidate to generate a grayscale image. Details of this processing will be described later with reference to FIG.

  FIG. 9 is a diagram showing a processing flow for assigning reliability to pixels determined to be character candidates shown in Step 8000 of FIG. The purpose of this processing is to appropriately remove or leave the false color generated mainly near the boundary. This process determines whether there is a character pixel around the pixel of the character candidate. If there is a character pixel, the reliability of the pixel is increased according to the number and luminance. The reason why the pixels around the pixel are determined is that when there are many character pixels in the periphery, there is a high possibility that the color of a part of the character has changed due to a false color or the like. Otherwise, it can be determined that the noise component is close to the character color. The character candidate pixels having high reliability in the processing of FIG. 9 are more likely to be determined as character pixels in the binarization processing (3080) of FIG.

  In FIG. 9, first, in step 9000, the first pixel in the region is selected. The following determination process is performed on this pixel. Step 9010 determines whether or not the pixel is a character candidate. In the case of a character candidate, in step 9020, it is determined whether or not there is a pixel determined to be a character around the pixel. The perimeter may be 8 or 4 neighbors adjacent to each other with the pixel as a center. When there are character pixels around, the reliability is set using the number of surrounding character pixels and the luminance information of the pixels and character pixels. As an example of reliability, there is a method of multiplying the number of surrounding character pixels by a constant. In addition, there is a method of multiplying the reciprocal of the difference in luminance between the pixel and the character pixel by a constant. The reason for using the luminance difference is that the smaller the luminance difference from the pixel of the character, the higher the possibility of the character.

  If the condition is not satisfied in step 9020, or after step 9030, it is determined in step 9040 whether all the pixels in the region have been processed. If not all are processed, the next pixel is selected at step 9050 and the process returns to step 9010. If all the processes have been processed, the character candidate pixel reliability setting process ends.

  FIG. 10 is a diagram showing a processing flow for assigning reliability to a pixel determined as a removal candidate shown in step 8010 of FIG. The purpose of this process is to remove pixels of noise components such as ruled lines having colors that are neither characters nor background. In this process, it is determined whether there is a character pixel or a character candidate pixel with high reliability around the removal candidate pixel. If there is a character pixel or character candidate pixel, the number or luminance is determined. Accordingly, the reliability of the pixel is increased. The reason why the pixels around the pixel are determined is that when there are many character pixels around the pixel, there is a high possibility that a noise component such as a ruled line intersects the character. The reason why the character candidate pixel having high reliability in the processing of FIG. 9 is also determined is because it is highly likely that it will eventually become a character because it is adjacent to the character.

  In FIG. 10, first, in step 10000, the first pixel in the region is selected. The following determination process is performed on this pixel. Step 10010 determines whether or not the pixel is a removal candidate. In the case of a removal candidate, in step 10020, it is determined whether or not there is a pixel determined to be a character around the pixel. When there are character pixels around, the reliability is set using the number of surrounding character pixels and the luminance information of the pixels and character pixels. This reliability setting method may be the same as or different from step 9030 in FIG. If the condition of step 10020 is not satisfied, the presence / absence of a character candidate having high reliability in step 8000 of FIG. 8 is determined around the pixel. If this condition is met, the process proceeds to step 10030.

  If the condition is not satisfied in step 10040, or after step 10030, it is determined in step 10050 whether all the pixels in the region have been processed. If not all are processed, the next pixel is selected in step 10060 and the process returns to step 10010. If all the processes have been processed, the reliability setting process for the removal candidate pixel is terminated.

  FIG. 11 is a diagram showing the flow of the grayscale image generation process based on the determination result and the reliability shown in step 8020 of FIG. In this process, the gray level image is generated by correcting the luminance value of the pixel according to the classification and reliability of the pixel. In the correction of the luminance value, the background is set to white, and the luminance value is increased for pixels that are not characters. The purpose of this processing is to generate a binary image in which only characters remain in the subsequent binarization processing.

  First, in step 11000, the first pixel in the region is selected. The following determination process is performed on this pixel. Step 11010 is processing for determining whether or not the pixel is a character. In the case of a character pixel, in step 11010, the luminance value of the pixel is set. If the condition of step 11010 is not satisfied, it is determined in step 11020 whether the pixel is the background. In the case of a background pixel, a white luminance value (maximum luminance value 255) is set so as to ensure dropout.

  Pixels that do not satisfy the condition of step 11020 are character candidates or removal candidates. In step 11040, it is determined whether or not the reliability is zero. A reliability of 0 means that there are no character pixels around. In this case, it is determined as noise, and a white luminance value is set in step 11030. If the reliability is greater than 0, it is determined in step 11050 whether the reliability exceeds a predetermined reference value. If the reliability is less than or equal to the reference value, a value obtained by increasing the luminance value is set in step 11060. A pixel that satisfies the condition of step 11050 has a low possibility of being a character, but its possibility is not zero. Therefore, in order to make it easy to drop out by binarization, a value with an increased luminance value is set. Even if the luminance value is increased, the pixel may be dropped out depending on the luminance value of the pixel or the surrounding luminance value. The final determination is performed in the subsequent binarization process (3070). An example of a calculation to increase the luminance value can increase a constant multiple of the reliability to the luminance value.

  If the condition of step 11050 is not satisfied, the luminance value is set as it is at 11010. This is the same treatment as the character pixel because there are many character pixels adjacent to the pixel.

After the pixel has undergone any one of steps 11010, 11030, and 11060, it is determined in step 11070 whether all the pixels in the region have been processed. If not all are processed, the next pixel is selected at step 11080 and the processing returns to step 11010. If all are processed, the grayscale image generation process based on the determination result and the reliability is terminated.
The processing of FIGS. 9 to 11 will be supplemented by using the example of FIG. First, the pixel of a character candidate will be described. When there are many character pixels around 6090 and 6100, the character is binarized to black as shown in FIG. 6110 is white in FIG. 6C because there is no character pixel around it. Although 6070 has character pixels around it, the luminance is high, so in FIG. 6C, it is white. Next, the removal candidate pixels will be described. Reference numeral 6080 denotes a ruled line pixel. This is white in FIG. 6C because the luminance or hue is different from the character color. 6030 is white in FIG. 6C because there is no character pixel around it. On the other hand, although 6120 and 6130 are removal candidates, there are many character candidates around them, and therefore, black is displayed in FIG. 6C. Thus, even when other colors such as false colors and ruled lines exist, only the same color as the character can be dropped out.

  Next, a second embodiment of an image processing method and an image processing apparatus to which the present invention is applied will be described.

FIG. 12 is another diagram showing a flow of the dropout processing of the present invention. The processes with the same numbers as in FIG. 3 are the same as those in FIG. In this example, image correction 1 (12000) and image correction 2 (12010) are added before and after the binarization processing (3070) in FIG. This process may be either one.
Image correction 1 (12000) is image correction using a grayscale image. An example of correction is tilt correction. The merit of performing the inclination correction here is that quantization errors are less likely to occur after the inclination correction in the grayscale image as compared to the binarized image. Specifically, when an image including an oblique line or curve is corrected, a jagged line is hardly generated on the line. As an example of the inclination correction method, a bilinear method or a bicubic method can be used. In order to correct the inclination, it is necessary to detect the inclination. This may be executed within the image correction 1 (12000) or may be obtained by other processing. Another example of processing performed in image correction 1 is noise removal. As an example of noise removal, smoothing processing is performed from the density of adjacent pixels.
Image correction 2 (12010) is image correction using a binary image. One example of correction is tilt correction. The advantage of performing the inclination correction here is that the processing time is shorter than that of the grayscale image. Note that for inclination correction, the inclination detection may be executed within the image correction 2 (12010) or may be obtained by other processing. Since the inclination correction can be executed by either image correction 1 (12000) or image correction 2 (12010), it is usually sufficient to perform either one. Another example of processing performed in image correction 1 is noise removal. An example of noise removal is removal of isolated points.

  Next, an image processing method and an image processing apparatus to which the present invention is applied will be described.

  FIG. 13 is a processing flow for reading characters on a form using the dropout processing of the present invention. In the image input (13000), the target color image is input, and in the reading area selection (13010), an area including the character to be read is detected. As a method for detecting a region to be read, there are a method for detecting a frame on a form, a method for obtaining from a predetermined coordinate, and the like. Next, dropout processing (13020) is performed on the area to be read. For this processing, the technique of FIG. 3 or FIG. 12 of the present invention can be used. Next, a character area is detected from the dropped-out image (13030). As an example of character area detection, a technique such as line extraction can be used. Next, after character recognition (13040), the recognition result is output (13050).

  Next, a fourth embodiment of an image processing method and an image processing apparatus to which the present invention is applied will be described.

  FIG. 14 is a process flow for recognizing the date of receipt using the dropout process of the present invention. The numbers in FIG. 13 are the same as those in FIG. The target color image is input in the image input (13000), and the region of the receipt is detected in the receipt detection (14000). As an example of receipt area detection, there is Patent Document 3. Next, dropout processing (14010) is performed on the region of receipt. For this processing, the technique of FIG. 3 or FIG. 12 of the present invention can be used. In this process, inclination correction is also performed. Next, a date area is detected from the image of the receipt that has been dropped out (14020). As an example of the date area detection, a technique such as selecting a column of black pixel blocks existing near the center in the vertical direction can be used. Next, the date area can be cut out from the dropout image for date recognition. In date recognition (14030), characters are recognized using OCR. Furthermore, various date notation formats are stored as knowledge in advance and collated with the character recognition result to correct the character recognition result and output a consistent recognition result as a date. If the character recognition result is consistent with the date, the result is output (13050) and the process ends. If there is a contradiction, the date image is rotated 180 degrees (14050) and re-recognized.

1 is a configuration diagram of an image processing apparatus according to an embodiment of the present invention. The flowchart which shows the outline of the dropout process of embodiment of this invention. The figure which shows an example of the dropout process of embodiment of this invention. The figure which shows an example of the color clustering process of embodiment of this invention. The figure which shows HSV color space. The figure which shows an example of the pixel classification | category and binarization of embodiment of this invention. The flowchart which shows the outline of the pixel classification | category process of embodiment of this invention. The flowchart which shows the outline of the brightness | luminance correction process of embodiment of this invention. The flowchart which shows the outline of the reliability setting process of a character candidate in the brightness correction process of embodiment of this invention. The flowchart which shows the outline of the reliability setting process of a removal candidate in the brightness | luminance correction process of embodiment of this invention. The flowchart which shows the outline of the grayscale image production | generation process based on the determination result and reliability in the brightness correction process of embodiment of this invention. The figure which shows another example of the dropout process of embodiment of this invention. The process flow of the form reading using the dropout process of this invention. The processing flow of the stamp date reading using the dropout processing of the present invention.

Claims (2)

An image processing apparatus that drops out some pixels in an input image,
By color clustering the pixels in the image,
Select a background color and a text color from colors other than the background color,
And further comprises means for classifying the pixels in the processing region for dropout,
Based on the classification result and the surrounding information of each pixel, the luminance value of the pixel is corrected to generate a grayscale image,
Generate a binary image by binarizing the grayscale image,
The image processing apparatus classifies image pixels into characters, backgrounds, character candidates, and removal candidates,
If the color of the pixel is close to the background color or white or the brightness is above the standard, it will be judged as the background
If the color of the pixel is close to the character color, determine the pixel as a character,
For pixels that are excluded from the above judgment,
When the character color is close to black, if the saturation of the pixel is less than the first reference, the pixel is determined to be a character, and if the saturation is greater than or equal to the first reference, the pixel is determined as a character candidate. Judgment
When the color of the character has a color, if the difference between the hue of the pixel and the hue of the character is less than the second reference, the pixel is determined to be a character, and if the difference is greater than or equal to the second reference, it is determined as a removal candidate. An image processing apparatus. The image processing apparatus according to claim 1.
If there are character pixels or character candidate pixels around the candidate and removal candidate pixels, increase the reliability.
Set the brightness value of the input pixel to the character pixel,
Set the maximum brightness value for the background pixels,
For character candidates and removal candidate pixels, if the reliability is less than the third standard, the background is determined to be the maximum brightness, and the reliability is greater than the third standard and less than the fourth standard. If the reliability is higher than the fourth standard, set the luminance value of the input pixel.
Generate a grayscale image using the brightness value of the set pixel
An image processing apparatus.

Images