JP5517028B2 - Image processing device - Google Patents

Description

The present invention provides an image processing apparatus for separating and identifying handwritten areas such as handwritten characters and patterns from areas such as printed characters for digital image data obtained by reading a document in which handwritten characters and printed characters are mixed with a scanner or the like. about the location.

  In recent years, there has been a function for distributing digitized image data to a terminal device such as a personal computer (hereinafter referred to as a PC) via a network using a scanner function of a multifunction peripheral (hereinafter referred to as MFP (Multi Function Printer)). It attracts attention. The image distribution function separates images and characters and applies different compression processes to increase the overall compression efficiency, and OCR (Optical Character Reader) processing for document images read by a scanner. Various image processing technologies have been developed and commercialized, such as a search enabling technology that enables character images to be converted into text data and invisible text information added behind the images to be searched. Increases convenience.

  In such a document reading by a scanner, a mixture of handwritten characters and printed characters is often input as a document, and the technology is disclosed as an application such as annotating by handwritten parts and proofreading of handwritten documents. There is something that has been.

  In such an application, a technique for distinguishing and extracting handwritten characters and printed characters included in an image read by a scanner is important, and various methods are disclosed (for example, Patent Document 1 to Patent Document). 3).

  In the invention described in Patent Document 1, the magnitude of the density gradient is obtained from the edge portion of the character or figure included in the image, a histogram is created for the magnitude of the density gradient, and the handwritten image is estimated based on the created histogram. Whether or not the character is a handwritten character is determined by setting a range of density gradients that can be made.

  Further, in the invention described in Patent Document 2, the binarized data obtained by binarizing the input image is cut out in units of one character, and a plurality of different feature amounts for each character are calculated in units of characters. A separation factor between handwritten characters and type is calculated by aggregating feature values over the entire image, and handwritten characters and type are determined based on the calculated separation factor.

  In the invention described in Patent Document 3, it is determined whether or not the character is a handwritten character by extracting the thickness, linearity, and angle of the stroke.

  However, in the invention described in Patent Document 1, since processing is performed on a region surrounded by edges, it is difficult to separate a portion where type characters and handwriting overlap. For this reason, for example, when a strikethrough of a printed character as shown in FIG. 32 is added by handwriting, it is impossible to make a separate determination (first problem).

  In addition, since it is impossible to make an accurate determination or the accuracy is reduced in an image that has been edge-enhanced by reduction processing or spatial filter processing, a raw image output from a scanner and a high-resolution image are processed. Need to be targeted (second problem).

  To supplement the second problem, in the current image distribution by the MFP, the size is smaller than the scanner resolution due to the problem of the capacity of the image, and usually about 200 dpi to 300 dpi is used. Also, in order to improve the readability of character images, etc., spatial filter processing is usually performed, and images that have undergone reduction processing or edge enhancement processing are delivered instead of raw images of the scanner. become. In the invention described in Patent Document 1, since the density gradient of an edge of a character image or the like is used as a feature amount, it is difficult to perform highly accurate discrimination on an image after these processes are performed. Therefore, it is necessary to perform the determination based on an image close to the raw image of the scanner. This indicates, for example, that the image after distribution cannot be used for processing such as processing on a PC, and the application destination of the technique is reduced. Also, scanners often used in MFPs currently have a resolution of 600 dpi, and if the accuracy decreases due to reduction processing, it is necessary to perform processing on images with a large resolution, which increases memory and increases costs. Problems such as a reduction in processing speed will also occur.

  Further, the invention described in Patent Document 2 has the first problem because the feature amount is calculated in units of one character. In addition, handwritten characters that are freely written in the margins of a document image without a character frame or the like often collapse, and in many cases it is difficult to recognize them as characters.

  In the invention described in Patent Document 3, since identification is performed in units of characters, there is a problem similar to that of the invention described in Patent Document 2, and the run length in the stroke and the stroke outline are also present. Or, in the method of determining by taking statistics of the length of the skeletal line and the angle formed by the intersection of strokes, the characteristics differ depending on the character type, and for example, images with many curves such as hiragana and alphabet are difficult to discriminate in the same way as kanji. There is (third problem).

  Further, in the determination by the thickness of the stroke, the Gothic type is good, but in the Mincho type character, the thickness varies in the line and the stop part, and it is difficult to determine. Further, since it is difficult to identify a reduced image, the second problem described above exists.

The present invention has been made to solve such a problem, and an object of the present invention is to provide high accuracy and wide application range for digital image data obtained from a manuscript in which handwritten characters and printed characters are mixed. It is possible to distinguish between handwriting and type.
Image processing device,

The present invention is an image processing apparatus for recognizing and extracting a handwritten portion using image data obtained from a document including a handwritten portion and a printed portion as input image data, and extracts a thin line image from the input image data. Fine line image extracting means, stroke extracting means for decomposing the thin line extracted by the thin line image extracting means into character strokes, and first determining means for determining whether or not the handwritten portion is in accordance with the state of the pixel value inside the stroke The first determination means includes a histogram calculation means for calculating a histogram of pixel values from the pixels inside the stroke, and a second determination means for determining whether the handwriting portion is in accordance with the calculated histogram form. Determining means, and the second determining means includes detecting means for detecting the number of peaks in the histogram, and when the number of peaks is 2 or more, An image processing apparatus according to claim things determines only over click.

[Action]
According to the present invention, a fine line image is extracted from image data obtained from a document in which a handwritten part and a printed part are mixed, the extracted thin line is decomposed into character strokes, and the pixel values of the pixels inside the strokes are determined according to the state of the pixel values. To determine whether it is a handwritten part. In this determination, a histogram of pixel values is calculated from pixels within the stroke, the number of peaks in the calculated histogram is detected, and if the number of peaks is 2 or more, determination is made with only one peak.

  According to the present invention, it is possible to discriminate between handwriting and type with high accuracy and wide application range for digital image data obtained from a manuscript in which handwritten characters and type are mixed.

1 is an overall block diagram of an MFP which is an image processing apparatus according to a first embodiment of the present invention. It is a figure which shows the image delivery system using the image processing apparatus of the 1st Embodiment of this invention. It is a flowchart which shows the handwriting removal operation | movement in the image processing apparatus of the 1st Embodiment of this invention. It is a figure which shows the input image processed by the handwriting removal operation | movement shown in FIG. It is a figure which shows the handwritten part in the image shown in FIG. It is a figure which shows the pixel determined to be the handwriting and the pixel determined to be mixed in the part in which the type and the handwriting overlapped in the image shown in FIG. It is a figure which shows the output image from which the handwritten part shown in FIG. 5 was removed from the image shown in FIG. It is a flowchart which shows the detail of the handwritten area | region recognition process in FIG. It is a figure which shows the image data after the thin line detection in the flow shown in FIG. It is a histogram of the pixel value of a handwritten character stroke. It is a histogram of the pixel value of a type character stroke. It is a flowchart which shows the detail of the thinning process in FIG. It is a figure which shows the example which performed the binarization process with respect to the input image shown in FIG. It is a figure which shows the example which extracted the square as a thin line. It is a flowchart of operation | movement of a stroke extraction process. It is a figure which shows the example of a handwritten character image. It is a figure which shows the result of having binarized the handwritten character image shown in FIG. It is a figure which shows the result of having performed the thinning process to the handwritten character image shown in FIG. 16, and the result of having detected the end point and the intersection. It is the schematic diagram which decomposed | disassembled the handwritten character image shown in FIG. 16 into the stroke. It is a flowchart which shows the detail of the process which decomposes | disassembles the character in FIG. 15 in a stroke unit. It is a figure which shows the image after the stroke extraction of the 3rd stroke of the handwritten character image shown in FIG. It is a figure which shows the pattern table which determines the priority divided | segmented into 8 patterns by the direction from the pixel before the center as the now attention pixel. It is a figure which shows the example which determines a priority using the table of FIG. It is a figure which shows the example of the type character after a spatial filter process. It is a histogram at the time of processing the type character shown in FIG. 24 in the 1st Embodiment of this invention. It is a flowchart of the stroke division | segmentation image generation process in the 2nd Embodiment of this invention. It is a figure which shows the example of the histogram at the time of processing the type character shown in FIG. 24 by the 2nd Embodiment of this invention. It is a schematic diagram of the target stroke in the 2nd Embodiment of this invention. It is a flowchart of the histogram calculation process in the 3rd Embodiment of this invention. FIG. 26 is a diagram showing a histogram obtained by converting the type histogram shown in FIG. 25 into 16 gradations. It is a figure which shows the histogram which made 16 gradations for the handwritten stroke. It is a figure which shows the image which added the strikethrough of the printed character by handwriting.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
[First Embodiment]
<Overall block diagram of MFP>
FIG. 1 is an overall block diagram of an MFP that is an image processing apparatus according to a first embodiment of the present invention. The components in the figure will be described below.

  This image processing apparatus includes a scanner unit 101 that reads a document, a scanner image processing unit 102 that performs image processing such as known γ correction processing and spatial filter processing on image data input from the scanner unit 101, and a bitmap format. Encoder 103 that compresses and encodes the image data and outputs the code data, CPU (Central Processing Unit) 105 that controls the operation control command of the entire image processing apparatus (MFP), communication with external devices, and the flow of data on bus 111 A volatile memory 106 used as a work area for temporarily storing image data and compression-encoded code data, a hard disk (hereinafter referred to as HDD) 107 for storing and accumulating code data, etc. I / F (interface), an operation unit 108 for setting a start button, an operation mode of the MFP, and the like, and image data encoded for an external device Format conversion processing unit 109 that converts the format of image data when sending, external I / F 110 that transmits and receives image data including control commands and page description language format to an external device such as a PC via an external transfer path, connection A bus 111 capable of bi-directional data transfer between the configured components, a decoder 112 that decodes the code data and outputs image data, and inputs the image data to perform known color correction processing, γ conversion processing, pseudo level A printer image processing unit 113 that performs toning processing and the like and outputs it to the printer 114, and a printer unit 114 that outputs image data to a recording medium such as recording paper.

  The external I / F 110 is connected to an external transfer path 202 such as a LAN (Local Area Network), is connected to an external device such as a server device or a PC, or an external network such as the Internet, and the image read by the scanner unit 101 , A function of distributing (transferring) images stored in the HDD 107 is provided.

<Copying operation>
The basic document reading operation and copying operation in the MFP of this embodiment will be described below.

  When the user presses a start button (not shown) on the operation unit 108, the CPU 105 that has received a signal from the operation unit 108 via the bus 111 performs setting of necessary parameters prior to the copying operation, and the like. The entire MFP is controlled to perform the copying operation.

  First, the scanner unit 101 scans a document, performs photoelectric conversion by a CCD (Charge Coupled Device) (not shown), and performs color image data of red (hereinafter R), green (hereinafter G), and blue (B) as digital signals. Convert to and output. The image data output from the scanner unit 101 is subjected to known image processing by the scanner image processing unit 101. The purpose of this image processing is, for example, γ conversion or log conversion for converting a reflectance attribute signal read from a CCD into a density attribute, correction of MTF (Modulation Transfer Function) deterioration of the scanner optical system, and moire suppression. There are a spatial filter process for performing edge enhancement and smoothing processing, a background removal process for removing the document background, a color conversion process for converting the color space of the scanner into a standard color space such as sRGB.

  The image data output from the scanner image processing unit 102 is temporarily stored in the memory 106 via the bus 111. The image data stored in the memory 106 is input to the encoder 103 and compressed and encoded. Code data output from the encoder 103 to the bus 111 is temporarily stored in the memory 106, read from the memory 106, and stored in the HDD 107. The image data read from the memory 106 is also input to the decoder 112 in parallel with the storage in the HDD 107 and is subjected to a decoding process. The bus 105 is controlled by the CPU 105 as described above.

  Normally, data write access to the memory connected to the bus in this way is performed by each processing unit connected to the bus using a DMA (Direct Memory Access) method with a fixed amount of data in units of several tens to several kilobytes. In general, the bus connection section is composed of a buffer and a DMA controller. However, in this embodiment, the DMA controller is omitted for simplification of description. In addition, when reading data from the memory, an access to the input buffer memory is required because access is similarly performed by the DMA method.

  The reason for interposing writing to the memory 106 when the code data is stored in the HDD 107 is as follows. The HDD may change the reading / writing speed between the side closer to the center of the disk and the side farther from the disk, or may cause a reading / writing error, etc. The device is not suitable. Therefore, instead of directly writing the output data of the encoder 103 as a synchronization signal, the code data once stored in the memory 106 is asynchronously written so that it can be used stably. The same applies to reading. Note that the HDD operates in synchronism when viewed macroscopically in units of one page.

  Code data stored in the HDD 107 can be used for copying operations such as backup when a paper jam occurs during the copying operation, so-called electronic sort function that outputs multi-page documents in multiple copies and page order. In addition, it is used for image data distribution to an external device, reprinting for reprinting data stored in the past without a document, and the like.

  On the other hand, when the code data is input from the memory 106, the decoder 112 performs predetermined decoding and composition processing, and outputs the image data to the printer image processing unit 113. Similar to the scanner image processing unit 102, the image processing in the printer image processing unit 113 is not particularly limited in the present invention. For example, a color correction process for converting RGB signals into printer color material signals such as Cyan (hereinafter C), Magenta (hereinafter M), Yellow (hereinafter Y), and Black (hereinafter K) to perform color matching. In addition, black generation processing, γ correction processing for matching γ of image data with γ of the printer unit 114, pseudo gradation processing for performing conversion to halftone such as dither and error diffusion, and the like can be considered.

  The image data subjected to the image processing as described above is printed on a recording sheet (not shown) by the printer unit 114 and output, and the copying operation is completed. In the present embodiment, it is described that the encoder 103 performs compression coding of an image and the decoder 112 performs decoding. Normally, in the MFP, such an operation is performed to save the required memory amount and the HDD capacity, but this can be realized without performing compression encoding. The type of compression encoding is not limited.

  What has been described above is the copying operation, which is the most basic operation of the MFP. After the copying operation is completed, the image data in the middle of the copying operation is stored in the HDD 107 in a compressed and encoded state. As described above, by storing image data in the HDD 107, the image data can be taken into a PC or the like as electronic data, and can be reprinted when necessary. If it is only necessary to store the document as electronic data in the HDD 107, the processing after the decoder 112 is not necessary.

<Image distribution operation>
As described above, the MFP according to the present embodiment has a function of distributing an image to an external device via the external transfer path 202. FIG. 2 is a diagram showing an overall system configuration in the case of image distribution. As shown in the figure, this system includes an MFP 201, an external transfer path 202, a PC 203 as a client device, and a monitor 204 that displays an image distributed to the PC 203. Here, the MFP 201 has the configuration shown in FIG.

The image distribution operation will be described below.
When a delivery parameter setting necessary for the delivery operation is performed by the operation unit 108 by the user and a start button (not shown) is pressed, the CPU 105 that receives a signal from the operation unit 108 via the bus 111 prior to the delivery operation. Necessary operation parameters are set, and the entire MFP 201 is controlled to execute a predetermined distribution operation.

  The delivery parameter settings are, for example, the form of delivery image (color, gray, monochrome binary), compression level, compression type, output format, delivery function, and the like. As the distribution function, for example, an image having multiple layers for each image type to increase the compression rate, or adding a transparent text code to the image by OCR to make it searchable is generally used. Here, for example, it is assumed that the “erase handwritten image” function for erasing the handwritten image is selected from a document in which print data such as handwritten characters and printed characters are mixed (a plurality of functions are selected at the same time). Is also possible).

  Here, when the distribution target image is new scan data, the image data compressed and encoded by the encoder 103 is expanded on the memory 106 as in the copying operation. If the distribution target image is an image stored in the HDD 107, the target image is read from the HDD 107 and similarly developed on the memory 106.

  Next, conversion processing to an “erased handwritten image” image set by parameter setting by the user such as a distribution function is performed. The image conversion process is performed in software by the CPU 105 developing an image on the memory 106. Note that the image conversion process includes monochrome conversion and binary conversion of a color image, but a description thereof will be omitted. Detailed operation of the image conversion process will be described later.

  Here, the image data expanded on the memory 106 has been compression-encoded and needs to be decoded before the normal image conversion operation. Decoding may be performed via the decoder 112 or may be performed by the CPU 105 in parallel with the processing. In the following description, the compression encoding and decoding processes are omitted for the sake of simplicity.

  The image data that has been subjected to the image conversion is developed again on the memory 106, and converted into a desired format set by the user by the format conversion processing unit 109. The format here includes a compression type, a commonly used document format, and the like.

  The document image whose format has been converted by the format conversion processing unit 109 is temporarily stored in the memory 106, and then transferred from the external I / F 110 to the PC 203 via the external transfer path 110, and a display application or the like is used on the PC 203. Displayed on the monitor 204 and viewed.

<Image conversion processing operation (handwritten image removal operation)>
The image conversion processing operation in the present embodiment is a handwritten image removal operation. FIG. 3 shows a flowchart of the handwriting removal operation.

  S1: The input image (see FIG. 4) is determined on a pixel-by-pixel basis as either handwritten / printed / other (including printing other than characters and a background such as a white background), and the determination result is written to the memory 106. Details will be described later.

  S2 to S4: The determination result is identified on a pixel basis (S2), and when it is a handwritten and non-printing area (S2: YES → S3: NO), a handwritten removal image (pixel) is generated (S4). The handwritten removal image here is a white pixel as an example, but preferably, a non-thin line portion of surrounding pixels is selected (details will be described later in the description of the handwritten region recognition process). The generated handwritten removal image (pixel) is overwritten on the input image on the memory 106. Note that if it is a type region in the determination of S3 (S3: YES), it is determined that the pixel has a handwritten character on the print of the type, and the input image is output as it is. At this time, the handwritten pixel (FIG. 5A) and the mixed pixel (FIG. 5B) are accurately determined for the part where the type and the handwriting overlap in the input image shown in FIG. It is also possible to leave the overlapped part in both the type part and the handwritten part.

  S5: It is determined whether or not the process has been executed for all the pixels of the input image. If the process has not been completed (S5: NO), S2 to S4 are repeated.

  As described above, the output image (see FIG. 7) obtained by removing the handwritten portion shown in FIG. 6 from the image shown in FIG.

<Handwritten region recognition process (S1)>
Details of the handwritten region recognition processing in step S1 of FIG. 3 will be described with reference to the flowchart of FIG.

  S11: A thin line portion is extracted from the input image. Subsequent processes are targeted only for the thin line portion. After the processing is completed, the image data (binary image data) of the thin line identification flag is stored in the memory 106. Details of the thin line extraction processing will be described later. The image data after the fine line detection is as shown in FIG. (However, such an image is not actually generated.)

S12: The image data of the thin line portion extracted in S11 is decomposed into strokes, and each character stroke is extracted as a processing target. Details will be described later.
S13: A histogram is calculated for each stroke image extracted in S12.

  S14: Strokes are identified as handwritten and typed based on the histogram calculated in S13. In the present embodiment, the variance or standard deviation of the histogram is calculated, and those having a large variation (standard deviation or large variance) by threshold processing are determined as handwritten character strokes, and those having a small variation are determined as type character strokes. In accordance with the determination result, the pixel unit is converted into ternary data (handwritten characters / types / others) and stored in the memory 106 as a flag image.

  FIG. 10 is a histogram of pixel values of handwritten character strokes, and FIG. 11 is a histogram of printed character strokes. It can be seen that handwritten characters have more variation and greater variance.

<Thin line extraction process (S11)>
The operation of the thinning process in S11 will be described with reference to the flowchart of FIG.
S21: First, the entire input image is binarized. There are various methods of binarization, and the simplest and fastest processing is binarization with a fixed threshold for the entire image. However, there are many cases where a problem occurs with accuracy when extracting characters. .

  As a preferred example, an image signal is input in units of blocks of a predetermined rectangular area, and one threshold value is determined for each block in the threshold value determination process for the input image signal. Various methods can be considered as the threshold value determination processing method here, and are not particularly limited. A known method, for example, a method of taking a histogram as disclosed in Japanese Patent Application Laid-Open No. 2002-7763 and using statistical properties such as average and variance, and a simple method include luminance (Y ) The average value of the values may be used, or a method of determining by calculating a predetermined weight on the average value may be used.

  A binarization process is performed according to the threshold value determined in the threshold value determination process, and is stored in the memory 106 as binary image data. If the input is a color image, preprocessing such as conversion to a luminance signal by a known method is required. FIG. 13 shows an example in which binarization processing is performed on input image data.

S22: A black edge portion is detected by raster scanning the binary image.
S23: The white edge (the boundary between the black pixel and the white pixel) facing the horizontal direction and the vertical direction of the black pixel of the binary image is scanned.
S24: When a white pixel is detected in any direction, the line width is determined by the number of pixels.
S25: If the line width is equal to or smaller than a predetermined threshold, the line is extracted as a fine line, and the scanned portion is stored in the memory 106 as a thin line image (the non-thin line portion may be deleted from the input image).

  S26 to S27: It is determined whether the thin line portion and the non-thin line portion are mixed in the same black block of the binary image (S26). If mixed (S26: YES), All the thin line images are set as non-thin line images (S27).

  According to such a method, in the determination up to S25, a figure such as a triangle may be taken as a thin line halfway, but the erroneous determination part is erased in S26 to S27, and the entire triangle is determined as a non-thin line. Further, as such a noise removal technique, a simple method such as changing the determination as a non-thin line area when the area extracted from the thin line is a small area has some effects.

  FIG. 9 described above is an example of thin line extraction results up to S25. In this figure, a part of the computer image is extracted as a thin line, but the erroneous determination area can be erased by correcting the black pixel area described above.

  In the case of a quadrangle as shown in FIG. 14, white portions (four places) are extracted as thin line portions and black portions are extracted as non-thin lines. In the case of binarization, the entire quadrangular becomes a black lump, so that it is possible to make all non-thin line portions by the mixed determination of the thin line / non-thin line portions.

<Stroke extraction process (S12)>
Details of the stroke extraction process of S12 will be described with reference to the flowchart of FIG.
S31: The binarization process is performed in the same manner as S21. Note that the processing is performed here for convenience, but in the present embodiment, it is only necessary to read out the image binarized in S21 from the memory 106.

  S32: The thinned line image is thinned into a 1-dot image to extract the character core line. There are various methods for thinning, but the method described in Japanese Patent Application Laid-Open No. 2008-158847 is preferable. Executes the sequential algorithm of 4-neighbor distance conversion, stores the result in the memory 106 sequentially, and executes the thinning process sequentially using the pattern matching of 3 × 3 mask while changing the direction of raster scanning I will do it. Finally, a fine line (core line) image of 1-dot characters is extracted and stored in the memory 106.

  FIG. 17 shows the result of the binarization process performed on the handwritten character image shown in FIG. 16, and FIG. 18A shows the result of the thinning process.

  S33: Next, character strokes are detected (characters are divided into strokes) based on the image thinned into one dot. By this processing, the end points and the intersections are detected as shown in FIG. 18B, and the strokes are broken down from the end points (start points) or the intersections to the end points (end points) or the intersections. Details of this processing will be described later.

The extracted individual strokes may be stored in the memory 106 after labeling or the like as necessary.
FIG. 19A shown in FIG. 18B shows a schematic diagram of stroke decomposition.

  S34: Based on the stroke detected in the stroke detection process (S33), the binary image is used to identify black pixels around the core line as a stroke image area. What is necessary is just to comprise so that a predetermined | prescribed image area | region may be included around an intersection (For example, only the black pixel in the 5x5 rectangular area | region around an intersection is included). Then, an input image corresponding to each stroke image area is generated as a stroke image, and used as an input image for histogram calculation (S13).

<Character stroke detection processing (S33)>
Processing for detecting a character stroke (decomposing characters into stroke units) will be described in detail with reference to the flowchart of FIG.

  If this process is simply executed, a stroke different from the stroke order of characters may be obtained as shown in FIG. 19B. When such a thing occurs, there is a possibility that a portion where handwriting and type are mixed cannot be identified well. FIG. 21 shows the image after the stroke extraction of the third stroke, but it is necessary to distinguish between the case of starting from the end point and the case of starting from the intersection.

  S331: The end point (start point) is detected by raster scanning the 1-dot thin line image, for example, from the upper left to the lower right. If an unprocessed intersection is left in the previous processing from S331 to S337, the process starts from the detected intersection.

  S332 to S333: It is determined whether or not there is a next branch (intersection) (S332). If it is not an intersection (S332: NO), black pixels are scanned from surrounding pixels and advanced by one dot (S338). In this state, since it is not a branch point, one pixel is advanced.

  If there is a branch (S332: YES), it is recognized as an intersection and it is determined whether there are two or more unprocessed pixel directions at the intersection (S333). If the direction to be advanced in the previous scan is one direction, the process of S333 is executed.

  For example, considering the case of a branch of a three-way road, if one direction has been processed before, the possibility of proceeding is uniquely determined, so the processing from S335 onward is not necessary.

  S334: If there are two or more directions to proceed, the priority of the direction is calculated. The direction priority is calculated from the direction of black pixel scanning.

  FIG. 22 shows a pattern table in which the priority is divided into eight patterns A to H according to the direction from the previous pixel, with the center being the current pixel of interest (scan position), that is, the intersection. The lower the number, the higher the priority. In addition, the pixel of * represents the position where a black pixel cannot be taken (because the previous pixel becomes an intersection when a black pixel exists at the position of X).

  Referring to FIG. 23 as an example, when the pixel before the intersection is above the intersection, the priority is determined by referring to the pattern in FIG. 22C and the position of the next pixel from the intersection. . In the case of a three-way road, there is a possibility of advancing in two directions, so in the case of FIG. 23, there are directions of priority 4 and priority 5. Here, for the determination of the priority of the stroke currently being processed, the higher priority is selected, and the priority becomes 4.

  Next, the priority of strokes other than the stroke being processed is obtained. In the case of FIG. 23, a stroke extending in the horizontal direction when viewed from the intersection is assumed. In this case, processed pixels are not included. The stroke extending in the horizontal direction can be assumed to be a stroke extending from left to right and two strokes extending from right to left. Strokes extending from left to right have priority 1 using the pattern of FIG. 22A, and strokes extending from right to left also have priority 1 using the pattern of FIG. 22E. In this case, one with a high priority is selected and set as the priority, and the priority of the strokes other than the result processing is 1.

  If you want to simplify the processing of other strokes, set the priority of the stroke direction in advance, such as determining only the stroke from left to right without calculating the priority for each direction. Also good.

  S335 to S336: The priority of the stroke being processed is compared with the priority of other strokes (S335). In the example of FIG. 23, since the stroke being processed is lower than the priority of the other strokes (S335: NO), the stroke ends (ends) at this intersection.

  If the priority of the stroke being processed is high (S335: YES), a one-dot stroke is advanced in the direction of higher priority among the two directions of the intersection branch (S336). If the priority of the stroke being processed in FIG. 23 is higher than the other strokes, the robot moves forward in the right direction (priority 4).

  S337: It is determined whether or not the end point is the end point, and the processes of S332 to S336 are repeated until the end point is reached or the priority is lower than the other strokes at the intersection.

As described above, according to the first embodiment of the present invention, a thin line image is processed in an image processing apparatus that recognizes a handwritten portion for a document in which printed portions such as handwritten characters and printed characters are mixed. Because the handwritten character is determined according to the state of the pixel value inside the stroke, and it is divided into character strokes, regardless of the character type, such as a portion where type and handwriting overlap, characters with many curves, etc. It is possible to determine with high accuracy without targeting only the raw image of the scanner.
In addition, since the handwritten part and the type part are respectively determined and the handwritten type part is determined to be a mixed part, even when creating an image in which the handwritten image is erased, the typed image is erased unnaturally in the mixed part. Never done.
In addition, when extracting only a thin line image as a determination target of handwritten character strokes, if there is a mixture of thin lines and non-thin lines in the black block of the binarized image, the thin line determination part is changed to non-thin lines. It is possible to prevent a mixture of fine lines / non-thin lines in a continuous image, and the accuracy of handwriting determination is improved.

  In this embodiment, an example in which processing is performed by the CPU of the MFP has been described. For example, after an image read by the scanner of the MFP is distributed to an external device such as a server or PC, the processing by the CPU of the server or PC is performed. May be implemented. In that case, a single scanner device or the like may be used instead of the MFP.

[Second Embodiment]
FIG. 24 shows an example of type characters after the spatial filter processing. When the character width is slightly thick, there may be a difference in image density between the edge portion of the character image and the inside of the character.

  If such a character image is to be determined by the processing of the first embodiment, a stroke having a high density pixel and a low density pixel has a histogram as shown in FIG. The variance and standard deviation values tend to increase. This leads to a decrease in accuracy of determination of type characters. The present embodiment makes it possible to cope with such a case.

  The configuration of the image processing apparatus of the present embodiment is substantially the same as that of the first embodiment, and only the stroke divided image generation process (corresponding to S34 in FIG. 15) is different.

FIG. 26 is a flowchart of stroke divided image generation processing in the present embodiment.
S41: Search for black pixels around the core line (1-dot thin line) of the target stroke from the binarized image.

  S42: It is determined whether the pixel is a black edge or a pixel near the black edge. The pixel near the black edge may be determined by the distance from the black edge. Although there is a method that targets only black edge pixels, better accuracy can be obtained by excluding pixels of 1 to 2 dots from the black edge and the black edge. If the pixel is a black edge or a pixel near the black edge (S42: YES), the next step S43 is skipped so that the pixel is not included in the stroke divided image.

S43: If the pixel is not a black edge or a pixel near the black edge in the determination of S42 (S42: NO), the pixel is added to the stroke divided image.
S44: It is determined whether or not the process has been completed up to the end point, and the process ends when all the processes up to the pixels around the end point have been completed.

  As described above, according to the present embodiment, it is possible to calculate a histogram only from pixels within a character by excluding edges and edge neighboring pixels from the stroke divided image in the stroke divided image generation processing.

  An example of a type letter histogram in this case is shown in FIG. Compared to FIG. 25, it can be seen that a plurality of peaks in the histogram are eliminated and variation is reduced (dispersion and standard deviation are reduced).

Note that when no pixels can be acquired by the processing of S41 to S44, the target stroke is an ultra-thin line image, so all the pixels of the target stroke that have been excluded may be used as stroke-divided images, or the ultra-thin line image is printed. If it is clear in advance, there is a method of determining a type without calculating and determining a histogram (since a normal handwritten image is thicker than a type).
FIG. 28 shows a schematic diagram of the target stroke.

  As described above, according to the second embodiment of the present invention, even in an image in which an edge is emphasized by a spatial filter, the edge and pixels near the edge are excluded, a histogram is calculated, and a handwritten character stroke and a type character are calculated. Since the stroke is determined, the handwritten character stroke and the printed character stroke can be distinguished with high accuracy.

[Third Embodiment]
This embodiment solves the same problem as that of the second embodiment by another approach. Since the configuration of the image processing apparatus is substantially the same as that of the first embodiment, only the histogram calculation process (corresponding to S13 in FIG. 8) which is a different part will be described.

FIG. 29 is a flowchart of the histogram calculation process in the present embodiment.
S51: Similar to the first embodiment, the histogram 1 is calculated with the normal number of gradations.

  S52: The histogram 2 is calculated by reducing the number of gradations. For this, there is a method of calculating by reducing the number of gradations of the input image data, but preferably the frequencies of the histogram 1 may be combined. For example, if the histogram 2 is calculated with an image of 256 gradations having 16 gradations, the pixel values of 256 gradations take 0 to 255, and therefore the addition result of the frequencies of the pixel values 0 to 15 is the pixel value 0 of the histogram 2 What is necessary is just frequency. FIG. 30 shows a histogram 2 obtained by converting the type histogram of FIG. 25 into 16 gradations.

  S53: The number of peaks is detected from the histogram 2. It is only necessary to scan the histogram 2 from the pixel value 0 and detect how many portions have the peaks. It should be noted that accuracy is lowered when a slight increase / decrease is detected, so that it is more preferable to make the change effective only when the value changes by a predetermined threshold value or more.

  S54, S58: It is determined whether the number of peaks is 2 or more (S54). If there is only one peak (S54: NO), the result of the histogram 1 is output to the handwriting determination process S14 as it is in S58. .

  S55: If the number of peaks is 2 or more in the determination of S54 (S54: YES), the valley is detected by the histogram 2. The lowest pixel value and the highest pixel value are treated as valleys.

  S56: The frequency of the pixels included in the valley, mountain (peak), and valley is integrated for each peak, and the peak with the highest frequency is selected. Note that the frequency of a pixel corresponding to a valley is included in both adjacent peaks.

  S57: The data of the histogram 1 corresponding to the pixel values from the valley of the selected peak to the valley is output to the handwriting determination process S14. When the histogram 2 has 16 gradations and the histogram 1 has 256 gradations, the pixel value data corresponding to the valleys in the histogram 2 includes 16 pixel values in the histogram 1, so that, for example, half the Only pixels close to the peak value may be included, or pixel value data corresponding to valleys may not be included at all.

  Note that the output to the handwriting determination process S14 uses the histogram 1 for all the pixel values of the input image, but there are cases where there is no problem even if the histogram 2 is used to simplify the process.

  FIG. 31 is an example of the histogram 2 for handwritten strokes. In this case, although there is some increase or decrease, the peak is determined as one.

  As described above, according to the third embodiment of the present invention, even in an image whose edges are emphasized by a spatial filter, a single peak is detected when the number of peaks in the histogram is detected and the number of peaks is plural. Since the handwritten character stroke and the type stroke are determined only for the pixel values related to, the handwritten character stroke and the typed character stroke can be accurately identified.

  101 ... Scanner unit, 102 ... Scanner image processing unit, 105 ... CPU, 106 ... Memory, 201 ... MFP, S1 ... Handwriting region recognition processing, S4 ... Handwriting removed image Generation processing, S11: Fine line extraction processing, S12: Stroke extraction processing, S13: Histogram calculation processing, S14: Handwriting determination processing, S21: Binarization processing, S22: Edge detection Processing, S23... Edge search processing, S24... Line width detection processing, S25.

JP 2007-087196 A JP 2006-092345 A Japanese Patent Laid-Open No. 10-162102

Claims (3)

An image processing apparatus for recognizing and extracting a handwritten part as input image data using image data obtained from a document in which a handwritten part and a printed part are mixed,
A fine line image extracting means for extracting a fine line image from the input image data; a stroke extracting means for decomposing the thin line extracted by the fine line image extracting means into character strokes; and a handwriting part according to a state of a pixel value inside the stroke. First determining means for determining whether or not ,
The first determination means includes a histogram calculation means for calculating a histogram of pixel values from pixels within the stroke, and a second determination means for determining whether or not the handwritten portion is in accordance with the calculated histogram mode. Prepared,
The image processing apparatus according to claim 2, wherein the second determination unit includes a detection unit that detects the number of peaks in the histogram, and when the number of peaks is two or more, the determination is performed with only one peak . The image processing apparatus according to claim 1,
The image processing apparatus according to claim 1, wherein the detection means performs peak number detection on a histogram with a reduced number of gradations . In the image processing apparatus according to claim 1 or 2,
The thin line image extraction means includes a binarization processing means for binarizing the image, an edge detection means for detecting a line image edge from the binarized image, and an edge detected by the edge detection means. Search means for searching for a non-thin line image edge in at least two directions, a line width detection means for detecting a line width based on the search result, a thin line determination means for determining whether the line width is a thin line, or the 2 An image processing apparatus, comprising: a changing unit that changes a thin line area to a non-thin line area when a thin line area and a non-thin line area are mixed in a black block of an image that has undergone a value process.

Images