JP2002024824A - Image processing device, image processing method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate the optimum representative color of a character. SOLUTION: This device is characterized by having a histogram calculation means for calculating a histogram corresponding to an input image, a binary threshold calculation means for calculating a binary threshold which blur a prescribed region image of the image based on the calculated histogram, a binarization means for binarizing the input image by using the calculated binary threshold, and a calculation means for calculating the color of the prescribed region image of the input image based on the binarization result by the binarization means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image processing device, an image processing method, and a storage medium.

[0002]

2. Description of the Related Art In recent years, digitalization of documents has been advanced due to the spread of scanners. If you own a computerized document in full color, it will be about 24 Mbytes in the case of A4 size at 300 dpi, and it will take up too much memory to hold it, and it is not a size that can be sent to others by mail attachment or the like.

Therefore, JPEG is known as a method for compressing a full-color image. JPEG is very effective for compressing natural images such as photographs, and has good image quality. However, JPEG compression of high-frequency portions such as characters causes image degradation called mosquito noise and a poor compression ratio. Generally, an office document has many character portions. Therefore, a color office document can be simply represented by binarizing and compressing such as MMR and having a character area coordinate and a representative color of the character.
Further, for complex color documents such as magazines, JPEG compression of the background portion where the character area is removed by performing area division and MMR compression of a binary image in which the character area is binarized with an optimal threshold are generated. By attaching color information to MMR data,
A considerably complicated color document can be represented with a small data size.

Therefore, a technique for calculating a character color representative color in a character area is required. Conventionally, there has been the following method as an example for calculating a character color representative color.

First, a coarse three-dimensional histogram of multivalued image data of a black portion is obtained with reference to a binary image of a character area. Next, a fine histogram of pixels of the multi-valued image included in the highest value of the coarse three-dimensional histogram is obtained, and the highest value is set as a representative color.

[0006]

However, when a representative color of a character color is calculated by using the above method, if a large character of 12 points or more is read at 300 dpi or more, a reasonable color is calculated. I was able to do 10
The proportion of characters below the point corresponding to black in the binary image of the originally calculated representative color data was small, and a desired color could not be calculated.

Here, the case of calculating the representative color of a large character and the case of calculating the representative color of a small character will be described with reference to FIG.

FIG. 19 shows a sample in which green characters are written on a white background. Reference numeral 1901 denotes a binarization result of relatively thick characters. The multivalued image of the black portion of the binarization result 1901 has a level change such as 1902.
Looking at the change in the level 1902, since it is long at the portions 1904 and 1905 corresponding to the representative colors of the character portion, the distribution is made as shown in FIG. 20A on the color space RGB. Reference numeral 2002 denotes a cluster of green character colors in FIG. 19, that is, a representative color of a character portion. Since the mass of the character portion 2002 has a certain size, it is relatively easy to extract it.

On the other hand, in a thin character as shown by 1906 in FIG. 19, the level change in the multi-valued image has a shape like 1907 and corresponds to a portion 19 corresponding to the representative color of the character portion.
As soon as they reach 08 and 1909, they change to the background level again. The distribution in the RGB color space in this case is as shown in FIG.
In comparison with the data of FIG.
It is difficult to calculate the 5 points. The left side of the dotted line is black as a character by the binarization processing.
When the value is converted into a value and the representative color is calculated by the method described in the conventional method, 2006 is obtained as the value having the largest number. This results in a whitish green color in comparison with the desired character color, which is not preferable.

In order to prevent this phenomenon, it is conceivable that the binary image is thinned and a conventional representative color calculation process is performed using the thin line image. However, this method has the following drawbacks.

For the sake of simplicity, the description will be made using "." As an example.

Assume that a green "." Is written on a white background in FIG. It is conceivable that the level transition of “.” Changes like 2104. Ideally, the central depression should return to the white level, but the small point character "." When the binarization process is performed using the threshold value indicated by 2105, 21
A black circle whose center is not white as shown in 02 is a binarization result. On the other hand, when the thinning processing is performed, the result becomes 2103. The position of the multi-valued image indicated by the binary image is level 2
The point 104 indicates 2106 points, which is an undesirable level for the representative color.

Since such a "crushing phenomenon" occurs particularly in a character having a small number of points, it can be seen that the thinning processing is not effective.

On the other hand, a binary image as an output representing a character has been used for calculating a character representative color. However, it is preferable that the threshold for optimally representing a character is binarized so that the character is not blurred. Even in consideration of the subsequent OCR process, it is known that the OCR result is better when binarized so as to be crushed rather than squashed.

FIG. 22 shows a typical luminance histogram of a character area. Point 2201 is a desirable point as a binary image of a character. However, when binarization is performed at this point, the pixels that transition from the background to the character portion are also black.
Although it is preferable as a value and an output, it becomes noise when calculating the representative color of the character.

FIG. 22 shows this state. When binarization is performed at point 2201 in FIG. 22, binarization is performed at 2301 in FIG. 23, and the binarization result is 2302, 2303
The binary image contains many transitions from the background to the characters.

Accordingly, since the binary image as the output representing the character is used for calculating the character representative color as described above, the optimum representative color of the character portion cannot be calculated.

An object of the present invention is to provide an image processing apparatus, an image processing method, and a storage medium which can solve the above-mentioned problem and can calculate an optimum representative color of a character portion.

[0019]

To achieve the above object, the present invention provides a histogram calculating means for calculating a histogram corresponding to an input image, and a predetermined area image of the image is blurred based on the calculated histogram. Like 2
A binarization threshold calculating unit for calculating a binarization threshold; a binarization unit for binarizing the input image with the calculated binarization threshold; and a binarization result based on the binarization result of the binarization unit. Calculating means for calculating a color of a predetermined area image of the input image.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the drawings.

FIG. 1 shows an application configuration diagram of the present embodiment.

Reference numeral 102 denotes an input of the original image 101 and an original image 1
The binarization unit (a) generates a binary image (a) 103 by binarizing 01.

Reference numeral 104 denotes a binary image (a), which detects an area such as a character area and a photograph area from the binary image (a), and outputs coordinates and attributes thereof, for example, area information 105 such as characters and photographs. This is a region dividing unit to be generated.

An MMR compression unit 106 performs MMR compression on one part of a binary image corresponding to a region having a character attribute from the region information 105, and generates a compression code D107.

Reference numeral 108 denotes a character representative color calculation unit that calculates a representative color of a character included in a region corresponding to a region having a character attribute from the region information 105. Character representative color calculation unit 10
8, a binarizing unit (b) 1 for a character representative color calculating unit
081 to generate a binary image (b) 1082. The color information calculated here is newly written as the attribute of the area in the area information 105.

Reference numeral 109 denotes a JPEG compression unit for performing JPEG compression of a part of the original image included in the area corresponding to the area having the attribute of the natural image from the information of the area information 105 to generate a compression code C.

Further, the above configuration will be described in detail.

FIG. 3 shows an example of a processing flow in the binarizing section (a) 102 and the area dividing section 104.

Steps S301 to S303 show the flow of processing in the binarization unit (a) 102, and step S3
04 to S306 show the flow of processing in the area dividing unit 104.

In step S301, an original image 101, for example, an RGB color image is input, and luminance conversion is performed by the following equation to generate a luminance image J.

Y = 0.299R + 0.587G + 0.114B

In step S302, a histogram of the luminance data is obtained, and a threshold value T for performing binarization is calculated.

In step S303, the luminance image J is set to a threshold T.
To generate a binary image K.

In step S304, contour tracing of black pixels is performed, and all black areas are labeled.

In step S305, the character and the natural image are determined from the shape and position of the labeled black area.

In step S306, those which are combined from the shape and position of the character area are combined. Step S
The combining process of 306 may not be performed. However, in that case,
There is a drawback that the number of character areas for calculating the representative color increases and processing time is required. The advantage is that it is possible to respond to a change in character color finely.

Next, using the original image of FIG.
A case where the processing up to the combining processing of step 306 is performed will be described.

The original image shown in FIG. 2 is subjected to luminance conversion (S
301, S302), and calculating the histogram of the luminance results in FIG. From this histogram, a threshold T = 150 is calculated using data such as average and variance,
The binarized image is as shown in FIG. 5 (S303). This is the binary image 103 in FIG. FIG. 6 shows a state in which the resolution of the binary image shown in FIG. 5 has been reduced, contour line tracing processing has been performed, and all the images have been labeled (S304). From information such as the shape and position of this labeled black area, characters,
Then, the attribute of the natural image is determined (S305). However, such an image is not actually generated, but an image. Here, 601 is large and the inside is black, so that it is determined to be a natural image. Also, from 602 to 6
05 has a character inside and is determined to be a "frame" from the hollow shape. In the present embodiment, the frame information is not stored in the area information 105 and is ignored. However, the present invention is not limited to this, and it may be held as frame information or an application. Alternatively, an application that handles the background of the character area information is also conceivable. In that case, it is necessary to have a means for calculating the color of the background.

FIG. 7 is a diagram showing a black region extracted from the original image of FIG. 2 with the attribute of the character. Here, if necessary, a group of these black pixels is grouped based on the closeness of the position, the width, and the height, and as shown in FIG.
17 character areas 801 to 817 can be detected. In the present embodiment, grouping (S306) is performed, and these 17 pieces of coordinate data have an attribute of a character, and are stored in 109 of FIG. On the other hand, the coordinate data 601 in FIG. 6 has an attribute of a photograph and is stored in 109 in FIG.

Next, the flow of processing in the character representative color calculation unit 108 is shown in FIG. Since processing is performed on all coordinates stored in the area information 105, step S901 is performed.
Then, it is checked whether or not there are unprocessed character coordinates. If there is, the process proceeds to step S902, and if not, the process proceeds to end.

In step S902, it is checked whether or not the attribute of the coordinate is a character. If the attribute is a character, the process proceeds to step S903. If not, the process proceeds to step S90.
Return to 1.

In step S903, a luminance histogram of the original image corresponding to the area information is calculated. Since this luminance histogram is a part of a character area, it is highly likely that the luminance histogram has a simple shape as shown in FIG. 22 instead of a complicated shape as in the whole histogram as shown in FIG. .

Therefore, in step S904, a threshold value optimal for calculating the representative color, that is, a threshold value T2 at which characters are blurred is calculated. In FIG. 22, it is point 2202.

As an example of a method for calculating the threshold value T2, there is the following method. This will be described with reference to the flowchart of FIG.

In step S1001, 0 is substituted for a variable limit for counting the number of processes so as not to fall into an infinite loop.

In step S1002, an average value (average) of the histogram and its skew are obtained from the luminance histogram. This skew holds the value specially as skew_first. The calculation formulas are as follows.

[0047]

[Outside 1]

In step S1003, avera is added to HistUpper.
Substitute ge and 0 for HistLower. Step S10
Check if limit is 10 or more at 04,
If it is 10 or more, the process proceeds to step S1009.
(Here, it may be 5 instead of 10, and may be 20.) In step S1005, from HistUpper to His
of the histogram in tLower

[0049]

[Outside 2] Is calculated.

In step S1006, it is checked whether or not the condition of skew <my ^* 0.1 is satisfied. If the condition is satisfied, no further calculation is necessary and the process jumps to step S1010. If not, the process proceeds to step S1007.
In step S1007, skew <0.0 and skew_first <sk
Check whether the condition of ew ^* 0.1 is satisfied, and if satisfied, skip to 1010 assuming that no further calculation is necessary. If not, the process proceeds to step S1008. In step S1008, average is substituted for HistLower.
In step S1009, 1 is added to limit, and the process returns to step S1004.

By the repetition as described above, finally 1
At 010, average is substituted for the threshold T2, which is shown in FIG.
Is a threshold value for obtaining a faint binary image as shown by point 2202 in FIG.

Depending on the shape of the histogram, there is a possibility of calculating a threshold value with no black pixels at the time of binarization. Therefore, the number of pixels existing on the black side from the threshold value T2 is counted.
If the amount is extremely small, it is necessary to correct the color slightly more than white.
FIG. 17 shows the shape of the histogram that easily leads to such a result.
Shown in

Instead of performing the above complicated calculations,
As a method of calculating the threshold value T2, take a histogram,
There is a method of selecting a threshold value at which 5% of pixels are binarized to black with respect to the total number of pixels. FIG. 18 shows an example of such a threshold value calculation.

In step S905, the binarization unit (b) 1081 performs a binarization process on the partial area at the threshold value T2 to generate a binary image (b) as shown by 1082 in FIG. . As shown in FIG. 24, if binarization is performed at the threshold T2, that is, at the point 2202, binarization is performed at 2401 in FIG. 24, and binarization that does not include a transition portion 2402 and 2403 is performed. Becomes possible. If necessary, the binary image is further subjected to a thinning process. Since this binary image is a threshold value that is blurred, there is little possibility of failure such as thin line processing performed in the conventional representative color calculation processing described with reference to FIG. In step S906, the RGB values of the pixels of the original image corresponding to the black portions of the binary image (b)
Generate a histogram for each. The color space of the generated histogram is not limited to RGB, but Y if the original image is YUV
A UV histogram may be taken. In step S907, each peak of the RGB histogram is written as the character representative color as the attribute of the corresponding area in the area information 105.

Steps S906 and S907
The following methods are also conceivable. For example, instead of calculating a histogram for each of RGB, an RGB three-dimensional histogram is calculated. In that case, it is impossible to calculate the histogram in detail because of the performance of the computer.
In addition, it is preferable to calculate a rough histogram so as not to be confused by noise such as a color transition point. After obtaining the largest value in the rough histogram, there is also a method of calculating a fine histogram existing in the histogram again and obtaining the largest value.

Finally, according to the binary image area information, the area having the character attribute is subjected to MMR compression of the corresponding area of the binary image (a) 102 to generate a compression code D107, while the natural image JPEG compression of the corresponding area of the original image data is performed on the area having the attribute of. For the type of area such as a character or a natural image, and in the case of a character, three pieces of area information 105 holding a representative color, a compression code C111, and a compression code D112
Generate a combined format if necessary. This is the compressed data.

(Second Embodiment) FIG. 11 shows another application configuration example. In the present embodiment, instead of using a binary image based on a threshold as an image to be divided into regions, a differential filter is applied to calculate the edge amounts of all pixels with neighboring pixels, and binarize the edge amounts. Is divided using the binary image obtained by the above. As the region division method, contour tracing is performed as in the first embodiment.

In this case, what is different from the first embodiment is that, in the region extracted as a character, there is also a region that is inverted when ordinary binarization is performed.

FIG. 12 shows the difference between normal characters and inverted characters. Inverted characters are, for example, white characters written on a red base, and are not uncommon for color originals. In the first embodiment, such a character region has no attribute as a character, and is a region including a frame with an outer color and a region with an attribute as a natural image. In the present embodiment, by using the differential binary image for area division, an inverted character area as shown in FIG. 12 can also be area-divided as a character. Then, the brightness histogram shape of the normal character area becomes as shown in FIG.
It is shaped like A peak 1301 indicates a lump of the base, and a peak 1302 indicates a lump of characters. In the present embodiment, the binarized part for generating the partial binary image (b) (11082 in FIG. 11) for calculating the character representative color requires the inversion processing.

The determination as to whether or not to invert can be obtained by the following equation.

The following equation is shown as an example in the first application example.

[0062]

[Outside 3] When the skew_first is minus, it can be determined that the area is a normal character area as shown in FIG. 22, but when the skew_first is plus, it can be determined that it is an inverted character area as shown in FIG.

FIG. 16 is a flowchart of this process.
And a brief description. Right side of FIG. 16 (S1613-S1619)
Is exactly the same as FIG. This is the calculation formula when the left side (1605-1611) is a reverse character.

In step 1603, skew_first
Is greater than or equal to 0, a DoInvert flag is set to instruct the binarization unit whether to perform inversion processing.

If the DoInvert flag is set (if it is ON), the binarization unit (b) 11081 and the binarization unit (a) 1111 visible as an output also invert the binarization result. In the case of the offensive that also corresponds to the inverted character, the area dividing unit 1104 needs to detect an area having the attribute of a frame, and also needs to calculate an average color in the frame. This is because the background of the inverted character has a color other than white, but it needs to be expressed. These frame area average color calculation units are omitted in FIG.

On the other hand, in the configuration of FIG. 14 described later,
Since the entire surface of the JPEG is retained on the background, there is no need to have a region having the attribute of a frame even when corresponding to inverted characters.

(Third Embodiment) As another applied configuration example, a configuration as shown in FIG. 14 can be considered.

The configuration of FIG. 14 will be briefly described.

In this configuration, as the area division processing, a character area extraction processing 1402 for detecting the coordinates of only the character area
And stores the character area coordinates in 1403.

A binary image 1405 of this character area is generated by a binarizing unit 1404, and according to the binary image 1405,
An original 1413 in which the character portion of the original image is filled with the surrounding average color by the character portion painting portion 1408 is generated. The partial binary image is subjected to MMR compression to generate a compression code D.
The character-free image is JPEG-compressed to generate a compression code C.

At 1411, the character representative color calculation processing shown in FIG. 9 of the first embodiment is performed, and a representative color 1412 is generated.

FIG. 15 shows a configuration diagram for decompressing the compressed data having the configuration shown in FIG.

When decompressing the compressed data, a multi-valued image G is generated by performing JPEG decompression processing of the compression code C. On the other hand, the MMR expansion processing of the compression code D is performed to generate a binary image F of the partial area, and the representative value of the area is placed on the pixel where the binary image is black on the image G, and the binary image is The union processing of doing nothing is performed on the area of, and an image H is finally obtained.

Structural Example Compared with FIGS. 1 and 11, there is an advantage that the atmosphere of the original image data is not impaired because the entire JPEG image excluding the character area is retained.

(Other Embodiments) Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus (one device) For example, a copier,
Facsimile machine, etc.).

Another object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and a computer of the system or the apparatus. (Or CPU or MPU) by reading and executing the program code stored in the storage medium,
Needless to say, this is achieved. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.
In addition, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program code,
The operating system (OS) running on the computer performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

Further, after the program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, based on the instruction of the program code, It goes without saying that a CPU or the like provided in the function expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0078]

As described above, according to the present invention, a histogram corresponding to an input image is calculated, and a binarization threshold value for blurring a predetermined area image of the image is calculated based on the calculated histogram. By binarizing the input image with the calculated binarization threshold and calculating the color of the predetermined area image of the input image based on the binarization result, even if the character is a thin character, Since the data of the color of the transition part to the character can be deleted, it is possible to calculate the optimal representative color of the character.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an application configuration diagram according to a first embodiment.

FIG. 2 is a diagram illustrating an example of an original image.

FIG. 3 is a diagram illustrating an example of a processing flow in a binarizing unit and a region dividing unit.

FIG. 4 is a diagram showing a luminance histogram shape of an original image.

FIG. 5 is a diagram showing a binary image obtained by binarizing an original image.

FIG. 6 is a diagram illustrating a state where labeling is performed on a binary image.

FIG. 7 is a diagram illustrating a black region extracted from a source image with character attributes.

FIG. 8 is a diagram showing a character area of an original image.

FIG. 9 is a diagram showing a flow of processing in a character representative color calculation unit.

FIG. 10 is a diagram illustrating an example of a method for calculating a threshold T2.

FIG. 11 is a diagram showing an application configuration diagram in the second embodiment.

FIG. 12 is a diagram illustrating a difference between a normal character and a reverse character.

FIG. 13 is a diagram illustrating a luminance histogram shape of a reversed character portion.

FIG. 14 is a diagram showing an application configuration diagram in the third embodiment.

FIG. 15 is a diagram illustrating a configuration for decompressing compressed data according to the third embodiment.

FIG. 16 is a diagram showing a flow of processing for a reversed character.

FIG. 17 is a diagram showing an example of a histogram shape.

FIG. 18 is a diagram illustrating an example of a method for calculating a threshold value T2.

FIG. 19 is a diagram illustrating a case where a representative color of a large character is calculated and a case where a representative color of a small character is calculated.

FIG. 20 is a diagram showing a distribution in an RGB color space.

FIG. 21 is a diagram illustrating thinning of a binary image.

FIG. 22 illustrates a typical luminance histogram of a character area.

FIG. 23 is a diagram showing a result of binarizing an image.

FIG. 24 is a diagram showing a result of binarizing an image.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 1/46 Z F Term (Reference) 5B057 BA30 CA01 CA08 CA12 CA16 CE12 CE16 CG02 CG04 DA08 DA12 DB02 DB06 DB09 DC14 DC17 DC19 DC23 DC36 5C077 LL08 MP05 MP06 MP08 PP27 PP31 PP32 PP46 PP47 PQ19 RR02 RR15 RR21 5C079 LA02 LA06 LA34 LB12 MA11 NA13 5L096 AA02 AA06 BA17 DA02 EA43 FA06 FA16 FA32 FA33 FA35 FA37 FA44 FA72 GA40 GA38 GA38

Claims (11)

[Claims] 1. A histogram calculation means for calculating a histogram according to an input image, and a binarization threshold value calculation means for calculating a binarization threshold value such that a predetermined area image of the image is blurred based on the calculated histogram. A binarizing unit that binarizes the input image with the calculated binarization threshold; and calculates a color of a predetermined area image of the input image based on a binarization result of the binarizing unit. An image processing apparatus comprising: a calculating unit. 2. The calculating means calculates an average value of an image in the input image corresponding to a black portion of the image with reference to the image binarized by the binarizing means. The image processing apparatus according to claim 1, wherein a color of a predetermined area of the input image is calculated based on the average value obtained. 3. The calculating means calculates a histogram of an image of the input image corresponding to a black portion of the image with reference to the image binarized by the binarizing means, and calculates the calculated histogram. The image processing apparatus according to claim 1, wherein a color of a predetermined area of the input image is calculated based on a histogram. 4. The image processing apparatus according to claim 1, wherein said binarizing means further comprises an inverting means for inverting the binarized result. 5. The image processing apparatus according to claim 1, wherein the predetermined area image is a character image. 6. A histogram calculation step of calculating a histogram according to an input image, and a binarization threshold calculation step of calculating a binarization threshold such that a predetermined area image of the image is blurred based on the calculated histogram. A binarization step of binarizing the input image with the calculated binarization threshold; and calculating a color of a predetermined area image of the input image based on a binarization result of the binarization step. An image processing method comprising: a calculating step. 7. The calculating step calculates an average value of an image in the input image corresponding to a black portion of the image with reference to the image binarized in the binarizing step, 7. The image processing method according to claim 6, wherein a color of a predetermined area of the input image is calculated based on the average value obtained. 8. The calculating step calculates a histogram of an image of the input image corresponding to a black portion of the image with reference to the image binarized in the binarizing step, and calculates the calculated histogram. 7. The image processing method according to claim 6, wherein a color of a predetermined area of the input image is calculated based on the histogram. 9. The image processing method according to claim 6, wherein said binarizing step further includes an inverting step of inverting a binarized result. 10. The image processing apparatus according to claim 6, wherein the predetermined area image is a character image. 11. A code for calculating a histogram according to an input image, a code for calculating a binarization threshold value such that a predetermined area image of the image is blurred based on the calculated histogram, A computer readable computer comprising: a binarization code for binarizing the input image with a binarization threshold; and a calculation code for calculating a color of a predetermined area image of the input image based on the binarization result. Storage media.