JP2001143076A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To exactly distinguish areas of characters and photographs from an inputted image a short time. SOLUTION: This image processor is provided with a binarization part 102 to divide the inputted image into foreground and background, an integrated threshold setting part 103 to create histogram of a background run by every line (or column) from the results of the binarization part 102 and to set a first integrated threshold for each line (or column), an integration processing part 104 to perform integration processing to the foreground by each line (or column), based on the first integrated threshold, a labeling part 105 to attach the same label to continuous pixels to the foreground integrated by the integration processing part 104 and to simultaneously constitute rectangles and an attribute detecting part 106 to distinguish the characters (strings), graphics or the photographs, tables, etc., by using at least one featured value of the rectangles labeled by the labeling part 105.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic device such as an OCR (optical character recognition) device, a copying machine, a facsimile, etc.
The present invention relates to an image processing apparatus that divides an image into regions such as a table.

[0002]

2. Description of the Related Art In a conventional image processing apparatus, a run-length analysis method is used as a region dividing means in which a run-length distribution is examined over the entire image region and a character region, a graphic region, and the like are divided by the length of a white run and a black run. And Japanese Patent Application Laid-Open No. Sho 64-15, a method using a spectrum analysis method of analyzing a Fourier spectrum of an input image and dividing it into various regions.
As described in Japanese Patent Application Laid-Open No. 889, there is a projection analysis method in which projections (histograms) in the vertical and horizontal directions are alternately and repeatedly taken to divide an area from information on a peripheral portion.

[0003]

However, the prior art as described above has a problem that the division accuracy for an image having a complicated area configuration is low. Further, in the spectrum analysis method, a large amount of time is spent for arithmetic processing, and processing is performed on pixels of an image, so that there is a problem that a storage area is enlarged.

[0004]

SUMMARY OF THE INVENTION The present invention is an image processing apparatus designed to solve such a problem. That is, the present invention provides a method for separating an input image into a foreground and a background.
A binarization unit, an integration threshold setting unit that creates a histogram of a background run for each row (or column) from the result of the binarization unit, and sets a first integration threshold for each row (or column); An integrated processing unit that integrates the foreground for each row (or column) based on the threshold value, and the same label is assigned to pixels that are continuous with the foreground integrated by the integrated processing unit, and a rectangle is formed at the same time. A labeling unit; and an attribute detection unit that distinguishes a character (string), a graphic, a photograph, a table, or the like using at least one feature amount of a rectangle labeled by the labeling unit, and a detection result of the attribute detection unit. Is performed to divide the area of the input image.

In the present invention, an input image is separated into a foreground and a background by binarization by binarization means, and the foreground is integrated using the first integration threshold set by the integration threshold setting means based on the result. To process. Further, the same label is assigned to pixels that are continuous with the integrated foreground by the labeling means to form a rectangle, and the attribute is detected by the attribute detecting means based on the feature amount of the labeled rectangle.
Based on this attribute, the input image can be divided into regions such as characters, figures, and photographs.

[0006]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of an image processing apparatus according to the present invention. FIG. 1 is a schematic configuration diagram of the image processing apparatus according to the present embodiment. That is, the image processing apparatus according to the present embodiment mainly includes an image input unit 101, a binarization unit 102, an integrated threshold setting unit 103,
An integration processing unit 104, a labeling unit 105, and an attribute detection unit 106 are provided.

[0007] The image input unit 101 is composed of, for example, a scanner and is a unit for inputting image data. Binarization unit 10
2 performs a process of binarizing the image data input by the image input unit 101 to separate the foreground and the background. The integration threshold setting unit 103 creates a background run histogram for each row (or column) for the output of the binarization unit 102, and sets an integration threshold for each row (or column).

The integration processing unit 104 includes an integration threshold setting unit 10
A process for integrating the foreground for each row (or column) based on the output of No. 3 is performed. The labeling unit 105 labels the output of the integration processing unit 104 with continuous black pixels and simultaneously performs a process of forming a rectangle. Attribute detection unit 106
Is based on at least one feature amount (eg, width, height, aspect ratio, image density) for each labeled circumscribed rectangle of the foreground and the corresponding input image information (signal level distribution, etc.). ), Attributes to be divided into regions such as figures, photographs, and tables.

Next, the processing procedure in the image processing apparatus of the present embodiment will be described with reference to the flowchart of FIG.
In the following description, reference numerals not shown in FIG. 2 refer to FIG. 1 unless otherwise specified.

First, an original image is input from the image input unit 101 (step S201). Next, the binarizing unit 102 creates a density histogram of the image, sets a threshold value based on the histogram by using a discriminant analysis method or the like, and binarizes the image (step S202). ON in this binary image
The (black) pixel is set to the foreground, and the OFF (white) pixel is set to the background.

Next, in the integrated threshold setting unit 103,
A white run histogram is created for each row (or column) of the binary image, and an integration threshold is set from the distribution (step S203).

The setting of the integration threshold will be described with reference to FIG. Among the images shown in FIG. 3A, five typical situations are indicated by dotted lines. The dotted line has no foreground, has one white run in the width in the main scanning direction, and has a histogram distribution as shown in FIG. At this time, the integration threshold is set to infinity (indicating that no integration is to be performed, but the integration processing is not actually performed if the integration threshold is set to be larger than the main scanning width), and the foreground in this line is not integrated. This determination can be made based on whether or not the total number of frequencies in the histogram is one.

A dotted line is a portion where graphics / photos and characters are mixed. The histogram distribution in this line is
The result is as shown in FIG. The histogram has a peak at a run length that is a white run between characters, and a white run between paragraphs (between objects) is distributed at a longer run length. In order to quantify this information,
The threshold th1 for peak detection and the threshold th2 for distribution range detection are used. In this case, the integration threshold is set to the threshold th for peak detection.
In the run length (1) expected to be a white run between the characters detected in 1 and the run length detected by the distribution range detection threshold th2, the distribution that does not include the run length (1) It is longer than the run length (1) and is an intermediate value of the shortest run length (2).

The dotted line is a portion where the foreground is only a figure / photograph. The histogram distribution in this line is shown in FIG.
As shown in FIG. The histogram has frequency 1 only at the run length corresponding to the white run that continues from the end point in the main scanning direction to the figure / photograph portion. In such a case, the integration threshold is set to infinity similarly to the dotted line, and the foreground is not integrated in this line.

This determination is made based on whether or not the total number of frequencies in the histogram is two. In this example,
Although there is one photo, the integration process is not performed in the case of a plurality of photos according to the following steps. That is,
When there are a plurality of photographs, the total number of frequencies in the histogram is not 2, but even when threshold processing is performed with the threshold value for peak detection th1, there is nothing to be detected, and it can be determined that there is no character. Based on the result, it is determined that the integration process is not performed.

The dotted line indicates the case where the foreground is only characters.
The histogram distribution at this time is as shown in FIGS.
It becomes as shown in. In this case, the distribution shape is the same as that in FIG. 3C, and the flow of setting the integrated threshold value can be performed by performing the same flow.

Here, in the case of a document image as shown in FIG. 3A, the same label is attached to a continuous black pixel or a conventional projection analysis method, and a feature quantity is obtained for each labeled circumscribed rectangle. , A method of separating characters and figures / pictures (photographs) based on their features, and examining the distribution of run lengths over the entire image area,
The same result as in the above embodiment can be obtained by the run length analysis method in which a character area, a graphic area, and the like are divided according to the length of the white run and the black run.

However, in the projection analysis method, in the case of a document image as shown in FIG. 4A, the histogram in the main scanning direction (the number of black pixels) and the histogram in the sub-scanning direction of the area shown by the dotted line in FIG. In (number of black pixels), since there is no boundary between the character and the figure / pattern (photograph) part (the areas overlap), the character area and the figure / pattern (photograph) area as shown in FIG. And cannot be separated.

If they are forcibly separated, the figure / pattern portions will be different as shown in FIG. 5 (c). FIG. 4 (a) shows a method of separating by a feature amount for each labeled circumscribed rectangle.
The same result as in the above embodiment (see FIG. 5D) can be obtained for such a document image, but the labeling processing is performed for each character (in the case of kanji, etc., one by one, one by one). Therefore, the processing time becomes enormous (in FIG. 4, the number of characters is reduced due to a notation problem).

In the case where a document image includes a character having a large font size and a figure / picture having the same size, it is difficult to separate the character by this method. The run-length analysis method for examining the run-length distribution over the entire image area and dividing a character area, a graphic area, and the like according to the length of white runs and black runs provides the same results as in the above-described embodiment in both FIGS. 3 and 4. When characters of a plurality of font sizes are mixed in the same document image, a run-length distribution is taken over the entire image area.
There is a possibility that the image area may be divided into two or two originally different graphic areas may be integrated.

In the present embodiment, such a situation is dealt with by locally examining the histogram distribution. The histogram indicated by the dotted line in FIG.
(B) to (f)) correspond to dotted lines in FIG.
3 (see FIGS. 3 (b) to 3 (f)), and it can be seen that similar results can be obtained by setting the threshold in the same manner as in the image of FIG.

Next, when the run length of the white pixel is shorter than the threshold set by the integration threshold setting unit, the integration processing unit 104 integrates the black pixels sandwiching the white run as the same object (step S204). By this processing, the characters are integrated as one object as a character string.

Next, in the labeling unit 105, 8
If there is a black pixel in the vicinity, it is determined that the pixels are continuous, and the same label is assigned to the continuous black pixels (step S205).
This labeling will be described with reference to FIG. First,
Check for unlabeled black pixels from left to right in the main scan direction. Go to the right end in the main scanning direction,
If not, the same processing is performed from the left end of the next line.

A label 1 is added to a black pixel a detected first in FIG. Next, starting from the black pixel a and checking unlabeled black pixels in the vicinity of the pixel 8, the black pixel b corresponds. Therefore, this is determined as a continuous black pixel, and the same label 1 as the pixel a is assigned. wear.

Next, it is checked whether or not there is a black pixel which is not labeled with the eight neighboring pixels of the pixels a and b. This operation is repeated until there is no unlabeled black pixel in the vicinity of 8 of all the pixels to which the label 1 is added.

Next, the process of checking whether there is any unlabeled black pixel from the right pixel of the black pixel a is restarted. In FIG. 6A, a black pixel g is detected, and a label 2 is added to the black pixel g. Thereafter, the process of adding the label 2 is repeated until there is no unlabeled black pixel near 8 of all the pixels to which the label 2 is added, as in the case of the black pixel a. In the case of FIG. 6A, FIG.
The label 1 is added to the black pixels a to f, and the label 2 is added to the black pixels g to l.

Finally, the attribute determining unit 106 determines the attribute by obtaining the characteristic amount for each circumscribed rectangle labeled by the labeling unit 105 (step S206). As the feature quantity, information on the width, height, aspect ratio, pixel density, and input image corresponding to the rectangular area of the rectangle are used.

As shown in FIG. 7, a character string (heading by the circumscribed rectangle width Wmin and the text can be roughly distinguished) and a figure / photo / table can be distinguished by the circumscribed rectangle aspect ratio of the character string (vertical writing, horizontal writing). But the aspect ratio WHR <1 / N, N <WH
In the case of R, it can be generally determined to be a character string, N is a real number satisfying N> 1). However, since it cannot be completely distinguished by this alone, the area S of the circumscribed rectangle and the image density ID as shown in FIG.
The accuracy is increased by using such methods.

The area using the area S of the circumscribed rectangle and the image density ID can be distinguished by a character string (text) if the area is smaller than th2, and a table or area if the area is larger than th3 but smaller than th1. If the image density is larger than th3 and the image density is larger than th1, it is determined that the image is a graphic / photograph.

As described above, a character string, a table, and a figure / photo can be distinguished using the width, height, aspect ratio, and pixel density of a rectangle, but a figure and a photograph cannot be distinguished. Therefore, in the present embodiment, a figure is distinguished from a photograph by using information of an input image corresponding to a rectangular area (the number of colors and variability in a three-dimensional space).

This takes advantage of the fact that photographs generally have more colors than graphics, and that the variability (degree of change) is greater for photographs. In this embodiment, the vari
The ratio used for the circumscribed rectangular area of the pixel whose maximum difference between the target pixel and its eight neighboring pixels exceeds a certain value is used as the availability.

In the case of a graphic, the edge portion exceeds the above-mentioned fixed value, but hardly exceeds the other uniform regions. As a result, the variability decreases. This is shown in FIG.
And FIG. In FIG. 9, a table having an image density lower than th1 is a table, an image density is higher than th1, and the number of colors is th.
Those with less than 2 are figures, and those with an image density larger than th1 and more colors than th3 are photographs.

In FIG. 10, the one having the number of colors smaller than th2 is a figure, the one having the number of colors larger than th3 is a photograph, and the one having the number of colors larger than th2 is smaller than Vari.
The figure whose availability is smaller than th1 is a figure, th1
The larger ones are photographs. These steps S206
11 are collectively shown in FIG.

The rectangular data (coordinate data, attribute code, etc.) of the various regions obtained as described above are output to the outside together with the image data. This allows you to spend less time
The character area and the figure / photograph area of the input image can be accurately divided.

Next, another example of the image processing apparatus of this embodiment will be described. That is, in the integration processing of step S204 shown in FIG.
It is assumed that the width between characters is smaller than the width between objects (character line spacing, space between characters and figures / photographs, etc.). However, if this does not hold, character strings and figures / photographs will be integrated. There is fear.

In order to cope with this, in the integration processing of step 204 shown in FIG. 2, when there is a white run shorter than the threshold set by the integration threshold setting unit, each of the black pixels sandwiching the white run is determined. By determining the black pixel run length and determining, based on the length and the ratio, whether the black pixels sandwiching the white run should be integrated as the same object, it is possible to avoid integrating the character string and the graphic / photograph. Will be possible.

This will be described with reference to the schematic diagram of FIG.
When both L2 and L4 in FIG. 13 are shorter than the threshold set by the integrated threshold setting unit, in the above example, the character “A” and the graphic /
A photograph is integrated, and as a result of performing subsequent labeling processing and attribute determination processing, an incorrect result is obtained.
In this example, when the black pixel run lengths of the black pixels sandwiching the white run shorter than the threshold set by the integrated threshold setting unit are M and N, M <th3, N <th3, and min (N,
When M) / max (N, M)> th4, the black pixels sandwiching the white run are integrated as the same object.

In FIG. 12, Ll <th3, L3 <t
h3, L5> th3, min (L1, L3) / max
(L1, L3)> th4, min (L3, L5) / ma
x (L3, L5) <th4, and L1, L2, and L3 are integrated, but no integration processing is performed on L3, L4, and L5.

Here, th3 is a third integration threshold value, and th4 is a fourth integration threshold value. There is a method which is determined in advance, but a method of setting the target white run (for example, th3
= Target white run length x 0.9).

In step S203, a histogram is created for each row (or column) in the previous example and an integration threshold is set. However, the histogram is also created in a direction orthogonal to the direction in which the histogram is created. make,
An integration threshold in that direction is set (the histogram changes if the direction is different, so the threshold here is the second integration threshold). If integration processing is performed in both directions in step S204, mixed vertical writing and horizontal writing will be performed. Also in the document image, the area dividing performance is improved.

In step S206, the number of colors in the three-dimensional space is used as the feature amount. A histogram distribution of brightness (luminance) and a histogram of saturation (color difference) are separately obtained, and the respective results are used. For example, a figure and a photograph can be subdivided into gray and color areas.

In step S206, threshold processing is performed on each feature amount, and the region is divided based on the result. Character strings, tables, and figures / photos each have distinct features, so it is possible to set an objective threshold value for the feature values, but the distinction between figures and photographs is not so clear. (There are many objects that differ between figures and photographs depending on subjectivity), and the objectivity of threshold setting is slightly lower than the threshold for distinguishing from other objects (character strings / tables).

Therefore, instead of directly performing threshold processing on the number of colors and variability, which are feature amounts used for distinguishing a figure from a photograph, a multi-level score is output for each value as shown in FIG. Is calculated, and a threshold value process is performed on a multi-level result (likeness of a figure, likeness of a photograph) obtained, thereby compensating for the objectivity of the threshold setting.

For example, according to the way of taking the curve in FIG. 13, it is possible to make FIG. 10 as a result shown in FIG. Also, in the present embodiment, a multi-level result is finally subjected to threshold processing to determine a figure or a photograph. However, depending on processing using the result of the region division processing, the multi-level result may be determined. May be used as a region signal.

Here, a multi-level scor is used for the feature amount used to distinguish a figure from a photograph.
e has been described above, but by performing the same processing on the feature values used to distinguish between character strings and tables, FIG. 7, FIG. 8 and FIG. It is possible to deform as follows.

In step S206, the ratio of the maximum value of the difference between the target pixel and its eight neighboring pixels to the circumscribed rectangular area of the pixel having the maximum difference between the target pixel and its eight neighboring pixels is used for the characteristic amount variability. There is also a method using the number. In this case, the histogram of a photograph often has a gently spreading distribution, and in the case of a figure, the same region division as in the previous example is performed by setting a threshold using the fact that the histogram has many poles. Can be.

[0047]

As described above, according to the present invention,
Since the integration process of black pixels is performed using a local histogram, and then the regions such as character strings, tables, figures, and photographs are divided using labeling, complicated processing is performed without increasing the processing time. It is possible to improve the division accuracy of the region configuration.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of an image processing apparatus according to an embodiment.

FIG. 2 is a flowchart illustrating a processing procedure according to the embodiment.

FIG. 3 is a diagram illustrating an integrated threshold.

FIG. 4 is a diagram illustrating an example of a document image.

FIG. 5 is a diagram illustrating an example of area division.

FIG. 6 is a diagram illustrating labeling.

FIG. 7 is a diagram (part 1) for explaining a distinction between a character string and a graphic;

FIG. 8 is a diagram (part 2) for explaining the distinction between a character string and a graphic;

FIG. 9 is a diagram (part 3) for explaining the distinction between a character string and a graphic;

FIG. 10 is a diagram (part 4) for explaining the distinction between a character string and a graphic;

FIG. 11 is a flowchart illustrating a flow of attribute detection.

FIG. 12 is a diagram illustrating another integration process.

FIG. 13 is a diagram showing a relationship between a multi-level and a score.

FIG. 14 is a diagram showing a distinction between a photograph and a figure.

[Explanation of symbols]

101: Image input unit, 102: Binarization unit, 103: Integration threshold setting unit, 104: Integration processing unit, 105: Labeling unit, 106: Attribute detection unit

Claims (6)

[Claims] 1. A binarizing unit for separating an input image into a foreground and a background, and a histogram of a background run is created for each row (or column) from a result of the binarizing unit, and at least a row (or a column) is generated. An integration threshold value setting unit that sets a first integration threshold value for each, an integration processing unit that integrates a foreground for each row (or column) based on the first integration threshold value, and a foreground integrated by the integration processing unit. Labeling means for assigning the same label to consecutive pixels and simultaneously forming a rectangle; attribute detecting means for distinguishing attributes using at least one feature amount of the rectangle labeled by the labeling means An image processing apparatus comprising: dividing an input image into regions based on attributes distinguished by the attribute detecting unit. 2. The integration threshold setting means creates a histogram also in a direction orthogonal to the direction in which the histogram was created, and sets a second integration threshold different from the integration threshold for each column (or row). The image processing apparatus according to claim 1, wherein: 3. The image processing apparatus according to claim 1, wherein one of the rectangular feature quantities is information on an input image corresponding to the rectangular area, and the information is image signal level distribution information. . 4. The apparatus according to claim 1, wherein the integration processing unit integrates the two adjacent foregrounds when a background run between two adjacent foregrounds is smaller than the first integration threshold. Image processing device. 5. The integration processing means, wherein a background run between two adjacent foregrounds is smaller than the first integration threshold, and
When the run length of the two adjacent foregrounds is equal to or less than a predetermined third integration threshold and the ratio of the run lengths of the two adjacent foregrounds is equal to or greater than a predetermined fourth integration threshold, the adjacent two foregrounds are determined. The image processing apparatus according to claim 1, wherein two foregrounds are integrated. 6. The input image is a color image signal,
The signal binarized by the binarizing means is a luminance signal in the color image signal, and the information of the input image referred to by the attribute detecting means is information of the color image signal including the luminance signal. The image processing apparatus according to claim 1, wherein: