JP3353968B2 - Image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention divides an input image (hereinafter referred to as "mixed image") in which various types of images such as printed characters, handwritten characters, photographs, and pictures are mixed into regions for each type of image. The present invention relates to an image processing device that performs image processing such as compression.

[0002]

2. Description of the Related Art Conventionally, when filing an image (hereinafter referred to as a continuous tone image) having a continuously changing tone, such as a mixed image (document image) in which characters, pictures, photographs, and the like are mixed, as digital data. Regardless of whether such data is stored or communicated, image data is subjected to data compression in order to transmit data efficiently.

In general, when a mixed image is captured by an imaging device and subjected to image processing such as data compression, the mixed image is divided into regions having relatively different properties, such as characters, pictures, and photographs, and processing corresponding to each image region is performed. It is desirable to do.

As a method of dividing each region such as a character string, a picture, and a photograph from the mixed image, the whole image is decomposed into connected components, and a certain type of integration is performed to form a region as a set of connected components. The method of setting is generally performed.

[0005] As a conventional technique, for example, a document “Image area separation method of mixed image of characters / pictures (dots, photos)” [Transactions of the Institute of Electronics, Information and Communication Engineers D-II Vol. J75-D-
IINo. 1 pp39-47 Jan. 1992], in order to determine a character image, an edge enhancement process is first performed, and the image is ternarized by an appropriate threshold value determination.
There is a method in which an edge region is detected by pattern matching at a portion where black and white pixels continue, and a character image region is determined.

[0006] Further, as disclosed in Japanese Patent Application Laid-Open No. 61-296481, there is a method in which an input binary document image is reduced and an area is detected by integrating adjacent black pixels.

As shown in FIG. 49, the image information storage section stores a binary image (document image) obtained by optically scanning a document and performing photoelectric conversion, as shown in FIG. A reduced image storage unit that divides the document image stored in the small area into small areas, and reduces and stores each small area to one pixel;
The reduced image is scanned in the character string direction, and a black pixel integration processing unit that integrates adjacent black pixels, and in the image obtained by the black pixel integration processing unit, a connected region of black pixels is detected as an image region. And an area detecting unit.

The binary document image stored in the image information storage section is divided into small areas each having a predetermined number of pixels. If the number of black pixels in each of the small areas is equal to or more than a predetermined threshold, the black If the pixel is smaller than the threshold value, a white pixel is assigned and stored in the reduced image storage unit 2.

The black pixel integration processing section scans the reduced image in the character string direction and integrates adjacent black pixels by converting a white run shorter than a predetermined threshold into a black run. The region detection unit detects a region where black pixels are connected in the image obtained by the black pixel integration processing unit as an image region. This makes it possible to detect an image area included in the document.

Further, as a processing function of the conventional image information processing system, a process of classifying input mixed images by image type and converting them into digital data is performed.

In this processing, when a mixed image is stored in a storage medium as digital data, a compression method capable of compressing data with maximum efficiency for each image type is applied to reduce the total data amount and the desired information quality. One of the objectives is to reduce the maximum while maintaining the above. In addition, a binary tone image such as a character or a line drawing and a continuous tone image such as a photograph are classified, and a binarization processing method is adaptively selected so that an image represented by each of the binary tone images is better. Some are intended to do so.

Various processing methods have been devised for dividing and classifying the description area for each image type from the mixed image. In many of these cases, the feature amount presented by the type of image is extracted, and the feature amount is determined by a predetermined evaluation function or determination function to determine the image type. Many of the conventional examples of an image have feature amounts such as the frequency of occurrence of black pixels and the frequency of occurrence of edges in a predetermined block area, a histogram of luminance levels, a spatial frequency distribution, a direction distribution of line segments, and the like.

In the present invention, Japanese Patent Publication No. 4-18350 discloses an example in which the frequency distribution of the density gradient of an input image used as a feature is used as the feature. In this classification processing method, the horizontal direction of a digital image which is an input image,
A density gradient is obtained for each pixel unit in the vertical direction, and a direction calculated from the obtained values of the horizontal and vertical density gradients is counted in the divided small region to obtain a frequency distribution.
The variance of the frequencies is calculated from the frequency distribution, and whether or not the area is a character area is determined by the threshold determination between the variance and a predetermined threshold.

[0014]

In order for the image data to be satisfactorily classified according to the above-mentioned conventional method, the ideal image data in a relatively good state where the input image data does not contain noise or the like is required. It is targeted. However, in practice, there are many cases in which unevenness of illumination, dirt on a file document, etc., enter the input image data, such that a decrease in contrast or noise occurs partially in such an image, and black pixel detection and edge extraction are unstable. become. Therefore, in the conventional method using unstable black pixel detection and edge extraction as parameters,
It is very difficult to determine an accurate image type.

Further, in the above-described area division of the mixed image, in order to perform the area division without losing the end of the image area, it is necessary to faithfully reflect an edge portion such as a character end point when the image is reduced. Is required. However, in the conventional technique, the captured binary mixed image is divided into small areas, and the black pixels are allocated when the number of black pixels in each small area is equal to or larger than a predetermined threshold. If it is larger, if the small area is at the edge of the image area, the area will not be detected and part will be lost, and if the threshold is set to '0', much noise will be picked up, and the correct division will not be performed. It had the drawback that it could not be done.

Further, the mixed image in which black and white are inverted has a drawback that the entire document is extracted as a large area.

The distribution in the direction calculated from the above-mentioned density threshold can well reflect the directional distribution of the edge portion of the image. Therefore, the difference from other images becomes remarkable only in the image of the typeface character including many edge components in the vertical and horizontal directions, and is effective in determining whether the image to be determined is the typeface character. It is a feature quantity. Furthermore, using the variance of the distribution as an evaluation criterion for making judgments from this feature value means observing a bias in the directionality of the edges, and the calculation of the variance itself has a relatively light computational load, and is therefore practical. It is.

However, even if only the variance with respect to the directional distribution of the density gradient is determined by the threshold value, for example, an image having a low brightness (density) level, that is, a so-called low-contrast image, or a small area to be determined may be used as a character. Is small or the line thickness of the character itself is small, even if it is a typeface character image, the variance becomes small and it is difficult to make a clear judgment. This is because the direction distribution of the density gradient presented by the background increases in frequency, and no difference is recognized relative to that exhibited by the character portion. Since the direction distribution of the density gradient indicated by the background usually has no direction dependency, the direction dependency of the direction distribution of the edge of the character portion is buried in the background distribution.

Furthermore, when trying to select and classify various types of images (handwritten characters, photographs, pictures, and backgrounds) as well as typeface character images, it is impossible to distinguish them simply by observing the variance of the directional distribution of the density gradient. There was a problem.

Accordingly, the present invention provides an image processing apparatus which is capable of dividing an input image in which various images are mixed into regions for each type of image, and which is suitable for performing image data compression or the like suitable for various image types. With the goal.

[0021]

[0022]

The present invention achieves the above object.
In order to achieve this, in an image processing apparatus that divides each region of a character string, a pattern, a photograph, and the like from a mixed image in which characters, patterns, photos, and the like are mixed, an image memory that stores the mixed image, and an image memory that stores the mixed image. Edge extraction / binarization processing means for extracting an edge from the mixed image thus obtained and converting the binary image into a binary image; and dividing the binary image into small regions each having a predetermined number of pixels, and converting each small region into one pixel. A reduced image processing means for reducing, a block processing means for integrating and blackening a black pixel area in the reduced image, and a pair of an image area circumscribed rectangle from the existing range of the black pixels connected by blocking of the block processing means. Provided is an image processing apparatus including a rectangular area extracting means for obtaining angular coordinates.

Also, an image frame memory for digitizing and storing a mixed image containing characters, pictures, photographs, and the like, and image data output from the image frame memory are divided into image areas of the same type to form the same type of image. A similar image area dividing means for outputting the position and size of the area, and selectively extracting the inside of the same image area into a small area of a predetermined size,
Search block area selection means for outputting the position and size of the small area, and image data corresponding to the small area selected by the search block area selection means are read from the image frame memory as determination target areas and temporarily stored. a determination target block image memory, the horizontal direction of the difference value data and the differencing detection means that detect a difference value data in the vertical direction from the image data of the determination target region, modifying for the luminance corrected by pre Symbol differencing data calculating a parameter based on the correction parameters, the luminance level correction processing means for calculating a luminance level that is corrected from the image data of the determination target region, a difference integral value data from the previous SL differencing detecting means, and said luminance Vector quantization means for vector-quantizing the corrected luminance level data from the level correction processing means, and the vector quantization means A quantization vector counting means for counting each component of the quantization vector, a neural network (neural network) which receives the counted quantization vector component as an input and outputs a predetermined image type, and outputs Provided is an image processing apparatus comprising: an image type determination unit that evaluates and determines an image type determination result, and outputs an image type determined in accordance with position and size data of the same type image region.

Further, image input means for converting an input image signal into digital image data, a determination target area selection section for selecting and extracting a small area of a predetermined size from the input image signal, a local feature pattern detection means for detecting a local feature pattern comprising pixels of a predetermined number of pixels adjacent the image data (N), vector quantization of the vector quantizing the local feature Pas <br/> turn on the N-dimensional space A histogram generating means for counting the frequency of occurrence of the representative vector quantized by the vector quantization means to calculate a histogram, and a quantized vector histogram obtained by the histogram generating means as an input. Image type identification means for identifying the distribution shape and outputting a desired image type; Provides the configured image processing apparatus and the image type determination means for determining a.

[0025]

[0026]

The image processing apparatus configured as described above extracts edges from input images (document images) in which different types of images, for example, typeface characters, handwritten characters, photographs, picture images, etc. are mixed. Binary image processing is performed. Part 2
The value image is efficiently divided into a small area composed of a predetermined number of pixels without losing the edges of the image area, and even for a mixed image in which black and white are inverted, each small area is reduced to one pixel. Then, the black pixel areas in the reduced image are integrated and blocked. The diagonal coordinates of the circumscribed rectangle of the image area are obtained from the existing range of the black pixels connected by the blocking, and the image area is divided into the same kind of image area.

Further, the present image processing apparatus is capable of accurately dividing the input mixed image for each same-type image region, defining a determination target region within the same-type image region, and for example, determining a horizontal difference value, a vertical difference value, and a corrected luminance level. A shape distributed in a three-dimensional orthogonal space formed by the three parameters is modeled by patterning by vector quantization, and the modeled distribution pattern is recognized by a neural network to determine an image type. Is done.

Further, the image processing apparatus detects a characteristic pattern for each divided region of the mixed image in predetermined small block units, and identifies and determines the occurrence probability distribution of the characteristic pattern by a neural network. The feature amount of this feature pattern is, for example, a plurality of (N) adjacent pixels of the image. For example, the brightness pattern is detected without omission in a predetermined small block, and the detected brightness pattern is regarded as an N-dimensional vector. In the space, vector quantization of a vector (luminance pattern) is performed. For each of the predetermined number of quantized representative vectors, the occurrence frequency distribution is counted, and for the purpose of identifying the shape of the occurrence frequency distribution, the occurrence frequencies of the respective quantization vectors are arranged in parallel. , And an image type identification result corresponding to the distribution shape is output.

[0029]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows the structure of an image data compression apparatus according to a first embodiment of the present invention.

In this image data compression apparatus, an image input unit 1 comprises an optical system 1a and a solid-state image pickup device 1b such as a CCD, for example, and converts a color image composed of a binary image or a continuous tone image to be inputted into a digital image Convert to data and import.

The image data output from the image input unit 1 is input to a color / monochrome conversion processing unit 2 which converts the captured color image into data of only a luminance (brightness) component, converts the data into monochrome image data, and outputs it. I do. This is to reduce the amount of data handled in the initial stage and simplify the subsequent processing when performing image area division for extracting an area by specifying an image type.

The monochrome image data output from the color / monochrome conversion processing unit 2 is input to a binarization processing unit 3 and subjected to a binarization process. In the binarization processing section 3, black pixel extraction processing or edge extraction processing is performed.

Here, the above-described processing will be specifically described with reference to FIGS.

FIG. 2 shows a configuration for performing black pixel extraction processing.

First, difference processing is performed on the monochrome image data by the difference calculation processing units 11 and 12 in the horizontal direction and the vertical direction of the image for each predetermined pixel unit. An appropriate threshold value is set for the data obtained by the difference processing, and is set to “0” when the absolute value of the difference data is equal to or less than the threshold value, and to “1” otherwise.
This is performed by the value processing units 13 and 14.

The combining unit 15 calculates the logical sum (exclusive logical sum) of the horizontal and vertical binary data, separates the data into horizontal and vertical data again, and reconstructs the black pixel reproduction units 16 and 1.
In step 7, all the parts between '1' and '1' sandwiching '0' are made '1'.

The resultant binarized data in the horizontal and vertical directions is further ORed by the synthesizing unit 18 to obtain a black pixel bit map.

Next, another example of the binarization processing performed by the binarization processing unit 3 and an example of the edge extraction processing will be described with reference to FIG. Similarly to the case where the black pixel bitmap is generated, the difference calculation processing units 21 and 22 perform difference processing on the monochrome image data in the horizontal direction and the vertical direction of the image for each predetermined pixel unit. .

An appropriate positive or negative threshold value is set for the data obtained by these difference processing, and when the difference data is equal to or less than the positive threshold value or equal to or greater than the negative threshold value, it is set to “0”.
When the value is equal to or more than the positive threshold value or equal to or less than the negative threshold value, the binarization processing units 23 and 24 perform a binarization process of setting it to “1”. The reason for this is to extract only one edge of a binary segment consisting of two pairs of difference values. This is to reduce the dependence of the characteristic parameter on the change in the thickness of the line segment in a character image or the like. The logical sum (exclusive logical sum) of the binary data in the horizontal and vertical directions is synthesized by the synthesizing unit 2.
5 to obtain binary image data by edge extraction.

Next, returning to FIG. 1, the overall processing flow will be described again.

The binary image data output from the binarization processing unit 3 is input to a reduction processing unit 4 and performs a reduction process. Reduction processing is realized by simple thinning of pixels at appropriate intervals. By this reduction processing, in the binary image data, the separation between the region having no elements constituting the image such as the line spacing and the region having the components becomes clearer.

Then, the reduction of the data itself is performed, and the processing load is reduced. The output from the reduction processing unit 4 is input to the boundary extraction unit 5 for the continuous component area. This boundary extraction is realized, for example, by the configuration shown in FIG.

First, the 2 output from the thinning processing section 26
The quantified data passes through an LPF (low-pass filter) 27 to perform an edge smoothing process, and the data obtained by the smoothing process is subjected to an edge extraction by an edge extraction unit 28 based on threshold determination.

Based on the extracted edge data, the boundary calculation unit 29 can calculate a boundary of a continuous component area by performing a linear approximation with a horizontal line and a vertical line. For this linear approximation, for example, approximation by the least squares method can be used. As a result, a rectangular area for each of a plurality of components is obtained.

Next, based on the area data output from the boundary extraction section 5 of the continuous component area, the image type determination section 6 determines the image type of each area as needed.

The determination of the image type will be specifically described with reference to FIG. FIG. 5A shows a process of extracting features due to differences in image types from statistical data in advance and determining an evaluation criterion at the time of determination. FIG. 5B shows a process for determining the actual image type using the obtained evaluation criterion. The edge extraction unit 31 in FIG. 5A corresponds to the binarization processing unit 3 described in FIG.

The extracted edge data has the effect of reducing illumination unevenness or the like in the original input image if it is present. In addition, if the edge extraction method described with reference to FIG. 3 is used, it is possible to use a feature parameter that has a small dependence on the thickness of a line segment forming a character image or the like. Edge extraction unit 31
The binary image data output from is statistically sampled in units of a predetermined block, and subjected to KL conversion by the KL conversion unit 32. The KL conversion unit 32 converts the binary image data in block units represented by the orthonormal vector of the number of pixels constituting the block into uncorrelated data, and calculates the base vector by the base vector extraction unit 33. I do.

As the binarized image data for which the basis vector is calculated, an image sample in which image types are mixed is used. Therefore, arbitrary block data of image data in which various images are mixed is used. The calculated base vector is the inner product operation unit 3
4 and the binarized image data output from the edge extraction unit 31 and an inner product operation is performed in block units. The data on which the inner product operation has been performed indicates component values for each calculated base vector, and feature amounts corresponding to various images can be obtained. The obtained feature amount as the inner product result for each base vector is input to the neural network unit 35.

The number of inputs to the neural network unit 35 may be the number of base vectors, but only the effective vectors selected preferentially from the vectors that remarkably represent the feature quantity may be input to the neural network. . The neural network uses a hierarchical network. The output of the neural network is input as the number of image types to be determined from the teacher data input unit 36 for each block data where artificial data is artificially sampled, and learning is performed. For learning, a general back propagation method may be used.

The network constructed by the pre-learning for each block area is a circuit for determining a binarized pattern vector depending on the image type of the binarized image data.
The image type can be determined by inputting the binarized pattern data to the determination circuit. The reason why the neural network is used to determine the feature amount of each vector obtained by the KL transformation is to determine a subtle difference in the feature amount with higher accuracy.

Then, in the actual judgment shown in FIG.
The output of the edge extraction unit 31 is input to the inner product calculation unit 34,
An inner product operation is performed between the binarized image data output from the edge extraction unit 31 and the base vector previously obtained in block units. The output is input to a neural network 35 constructed by pre-learning, the type of image is determined, and the determined type of image is output.

In the configuration diagram of FIG.
Is output to the data compression unit 7, the data compression method is selected, and the output image from the image input unit 1 is subjected to data compression by the corresponding data compression method. As a matter of course, data compression is selectively applied only to the data of the portion where the image data exists together with the position and size data of the area output from the boundary extraction unit 5. Therefore, the output from the data compression unit is data of the positions and sizes of a plurality of partial areas where image data exists, image type data of each area, and compressed data of each area.

As described above in detail, in the image compression apparatus according to the present embodiment, in a mixed image input in the presence of a disturbance such as illumination unevenness, a binary pattern which is hardly affected by the disturbance is subjected to the optimal coordinate axis conversion and the neuro determination. Is provided, the total number of data in the mixed image file is reduced, and the image area division for each image type can be performed with high accuracy.

According to the first embodiment, in the initial stage, binarization and reduction processing are performed to divide the image area, and the number of image data to be handled and the number of image type determination points can be reduced. This is an image data compression apparatus that can reduce the load, facilitates hardware implementation, and is highly feasible.

Next, FIG. 6 shows and explains the concept of area division of an image processing apparatus as a second embodiment according to the present invention.

In this image processing apparatus, an image memory 45 for storing a document image captured by an imaging device or the like.
An edge extraction / binarization processing unit 46 for extracting edges from the document image stored by the image memory 45 and binarizing the image; and a document image binarized by the edge extraction / binarization processing unit 46 A reduced image processing unit 47, a block processing unit 48 that connects adjacent black pixels in the reduced image described above to form a block, and a circumscribed range from the existing range of the connected black pixels in the block processed image described above. And a rectangular area extraction unit 49 for obtaining the diagonal coordinates of the rectangle.

Next, FIG. 7 shows a more specific configuration of the image processing apparatus according to the second embodiment and will be described.

The edge extraction / binarization processing section 46 converts a 24-bit full-color document image captured by an image pickup device (not shown) and stored in the image memory 45 into 8
A luminance conversion circuit 50 for converting the image into a black and white gradation image, a gradation image memory 51 for storing the black and white gradation image, and a horizontal scanning of the black and white gradation image stored in the gradation image memory 51. , A binarization processing circuit 53 for binarizing the difference value, a horizontal binary image memory 54 for storing the horizontal difference binary image, and a similar operation in the vertical direction. Vertical difference processing circuit 55 for performing the above, a vertical binary image memory 56, and the binary image memories 54 and 56
A logical OR operation circuit 57 that obtains a logical OR of the binary images stored in the OR and obtains an edge binary image;
A binary image memory 58 for storing the value image.

In the reduced image processing section 47, a reduced image processing circuit 59 for reducing the binary image stored in the binary image memory 58, a reduced image memory 60 for storing the reduced image, A filter processing circuit 61 for removing arc standing points and the like in the reduced image stored in the image memory 60.

Further, the block processing section 48 scans the filtered image from the filter processing circuit 61 vertically and horizontally, and executes a white run (long white run) having a length equal to or longer than a predetermined value.
, A block image memory 63 for storing the detected long white runs, and a label processing circuit 64 for labeling the block images stored in the block image memory 63.

Then, in the rectangular area extracting section 49,
An address extraction circuit 65 for extracting the circumscribed rectangle diagonal coordinates (address) of each block from the existence range of black pixels to which the same label has been given by the label processing circuit 64; an address memory 66 for storing the extracted addresses; An address conversion circuit 67 converts an address stored in the address memory 66 into an address of an image before reduction.

In the image processing apparatus configured as described above, the image data is captured by the image capturing apparatus and stored in the image storage memory 45.
Is converted into an 8-bit black-and-white gradation image by the luminance conversion circuit 50,
It is stored in the gradation image memory 51. Various types of luminance conversion circuits 50 can be considered. For example, the configuration is such that each of RGB is multiplied by a predetermined value and the sum is obtained. In the horizontal difference processing circuit 52, the gradation image memory 51
Is scanned in the horizontal direction to obtain a difference, and the binarization processing circuit 53 binarizes the difference value to obtain a binary image.
The value is sent to the value image memory 54.

Such an operation will be described with reference to FIG.

FIG. 8 shows a partial image 68 in which a part of a document image is enlarged, gradation image data 69 corresponding to the partial image 68, and a horizontal binary image memory 54. A binary image 70 corresponding to the partial image 68 is shown.

In the partial image 68, one cell represents one pixel, and the gradation image data 69 is a gradation image corresponding to the partial image 68 stored in the gradation image memory 51. The data is scanned in the horizontal direction to calculate the difference between adjacent pixels. For example, when the difference value exceeds 50, that part is regarded as an edge, and 1 (black) of the pixel with the smaller luminance is assigned to the binary. And store it in the horizontal binary image memory 54.

Similarly, in the vertical direction difference processing circuit 55, the difference is calculated in the vertical direction and binarized to obtain a binary image.
Send to 6. In the OR operation circuit 57, the horizontal direction 2
The logical sum of the difference binary images stored in the value image memory 54 and the vertical binary image memory 56 is calculated, and the obtained edge binary image is stored in the binary image memory 58.

The reduced image processing circuit 59 divides, for example, a binary image composed of 512 × 512 pixels stored in the binary image memory 58 into 64 × 64 small areas composed of 8 pixels vertically and horizontally. Then, a black pixel (a pixel having a value of 1) is detected in each small area, and if it is detected, it is set to “1”;
It is stored in the reduced image memory 60.

Next, the filter processing circuit 61 filters the reduced image stored in the reduced image memory 60 to remove noise in the reduced image. Various filters are conceivable, for example, a 3 × 3 filter as shown in FIG.
Is used. This is to make the value of the target pixel the same as the value of the neighboring pixels when the values of the eight neighboring pixels of the target pixel are the same. Can be removed. The long white run detection circuit 62 scans the reduced image after the filtering process vertically and horizontally to detect a white run (long white run) having a length of, for example, 16 pixels or more, and detects "0" in the detected portion. Black pixels are integrated by assigning '1' (black pixels) to the other parts (white pixels), and a block image is generated and stored in the block image memory 63.

The label processing circuit 64 scans the block image stored in the block image memory 63 to check the connection state of each pixel. For example, by assigning the same label to four connected pixels, labeling is performed. I do.

The labeling operation will be described with reference to FIG.

In FIG. 10, the target pixel x in the document image
And its vicinity are enlarged. The image is binarized by the processing so far, and white pixels are given by 0 and black pixels by 1. Now, let the pixel of interest be x (image [i]
[J]) and the pixel above it is denoted by a (image [i] [j−
1]), the pixel on the left is b (image [i−1]
[J]), and labeling is performed according to the following conditions. (1) If x = 0, x is given a label 0. (2) If a = b and a = 0, give x a new label. (3) If a = b and a ≠ 0, give x the same label as a. (4) If a> b and b = 0, give x the same label as a. (5) If a> b and b ≠ 0, x is given the same label as b, and labels of all pixels having the same label as a are set to b.
Make the same as the label. (6) If a <b and a = 0, give x the same label as b. (7) If a <b and a ≠ 0, x is given the same label as a, and labels of all pixels having the same label as b are set to a
Make the same as the label. However, when the target pixel is at the upper end of the image, a = 0, and when the target pixel is at the left end of the image, b = 0.

The labeling is completed by sequentially performing this processing from the upper left of the image.

Next, the address extraction circuit 65
Labeled blocks stored in the block image memory 63.
Scan the image, and use the same label as shown in FIG.
The horizontal minimum / maximum value (x_sn,
x_en) And the minimum and maximum values in the vertical direction (y_sn, Y_en)
Detects the diagonal coordinates of the circumscribed rectangle of the image area (upper left coordinates
(X _sn, Y_sn), Lower right coordinates (x_en, Y_en)) As an ad
In the memory 66. In the address conversion circuit 67
Reduces the address stored in the address memory 66
To perform enlargement conversion according to the reduction ratio in the image processing circuit 59
Therefore, it is converted into the address of the image before the reduction.

For example, when the reduction ratio in the reduced image processing circuit 59 is 1/8, the address conversion is performed by the following equation, and the contents of the address memory 66 are rewritten.

[0076] _{x s '= x s × 8} y s' = y s × 8 x e '= (x e +1) × 8-1 y e' = (y e +
1) × 8-1 In this way, the circumscribed rectangle diagonal coordinates stored in the address memory 66 and the original image stored in the image memory 45 are output as a result of the area division processing.

Next, FIGS. 12 to 14 show output images in each processing step in this embodiment.

In FIGS. 12 to 14, the image memory 4
5, a region-separated document image 73 composed of characters and photographs, an edge extraction / binarization processed image 74 stored in the binary image memory 58, and a reduced processed image stored in the reduced image memory 60 75, the block processed image 76 stored in the block image memory 63, and the address memory 6
6, the image 77 indicating the image area division result of the document image stored in the image memory 45 by the address stored in the image memory 45.
And

As a result, it is possible to divide the image area without losing the edge of the image area. Further, since the image is cut out in a rectangular area, the area separation of the document image can be efficiently performed.

The image processing apparatus of the second embodiment performs edge extraction and binarization using the difference in luminance between adjacent pixels before performing image reduction processing. Can be faithfully reflected on the reduced image, and the image area can be efficiently divided without missing the edge of the image area and even with respect to the inverted black and white document image.

Further, since the image processing apparatus removes unnecessary black pixels which cause erroneous division by applying an arc elimination filter to the reduced image, the image area is efficiently divided. It becomes possible. Furthermore, since the black pixel area is blocked by detecting long white runs in the block processing unit, even if the character string direction is not known in advance, the image is scanned to detect short white runs, and the black run is detected. It is not necessary to perform a process of taking a logical product by replacing with a run, and the image area can be efficiently divided. Further, in this embodiment, since the black pixels that have been blocked are labeled, the connected state of the black pixels is simply represented, and the image area can be efficiently divided.

Next, an image processing apparatus according to a third embodiment of the present invention will be described with reference to FIG. This image processing apparatus is the same as that of the second embodiment described above except that only the edge extraction / binarization processing section 46 in FIG. 7 is modified, and the other components are the same. Only the explanation will be given. The same components as those shown in FIG. 7 in the present embodiment are denoted by the same reference numerals, and description thereof will be omitted.

In FIG. 15, a black-and-white gradation image stored in the gradation image memory 51 is scanned in the horizontal direction, and a horizontal direction removal processing circuit 78 for taking a ratio is provided. A circuit 79 and a vertical division processing circuit 80 for performing the same operation as the horizontal removal processing circuit 78 in the vertical direction are provided.

In the image processing apparatus constructed as described above, the horizontal division processing circuit 78 includes the gradation image memory 51.
Is scanned in the horizontal direction to obtain a ratio, and the binarization processing circuit 79 binarizes the value of the ratio to obtain a binary image in the horizontal direction.
The value is sent to the value image memory 54.

The binarization operation will be described with reference to FIG.

FIG. 16 shows a partial image 68 obtained by enlarging a part of a document image, gradation image data 69 corresponding to the partial image 68, and a binary image 8 corresponding to the partial image 68.
1 is shown.

In FIG. 16, a partial image 68 is an enlarged part of a document image, and one square represents one pixel. The gradation image data 69 is image data of a portion corresponding to the partial image 68 stored in the gradation image memory 51, and is a binary image of a portion corresponding to the partial image 68.
The binary image is scanned in the horizontal direction as shown in the figure to determine the magnitude of the luminance of the adjacent pixels, and the ratio is determined. Then, for example, when the value of the ratio is less than 0.9, “1” (black) is assigned to the pixel with the smaller luminance to be binarized and stored in the horizontal binary image memory 54. And the binary image is
The portion corresponding to the partial image 68 stored in the horizontal binary image memory 54. Similarly, the vertical division processing circuit 80 binarizes the image by taking the ratio in the vertical direction, Send to

Next, a description will be given of the area division performed by the same processing as in the above-described second embodiment. 17 to 1
9 shows an output image of each processing step in this embodiment.

17 to 19, the document image 82 to be divided into regions is composed of characters and pictures stored in the image memory 45. The binarized image 83 is a binarized image stored in the binary image memory 58, and the reduced image 84 is a reduced image stored in the reduced image memory 60. The block processing image 85 is a block processing image stored in the block image memory 63, and the area division image 86 is an image area division of the document image stored in the image memory 45 according to the address stored in the address memory 66. It is an image showing a result.

By such image processing, it is possible to divide an image area without losing the edge of the image area. Also,
Since the image is cut out in a rectangular area, the area separation of the document image can be performed efficiently.

Therefore, in the image apparatus of the third embodiment,
Before performing image reduction processing, edge extraction and binarization are performed using the luminance ratio of adjacent pixels, so that edge portions such as character end points can be faithfully reflected in the reduced image. And the image area can be efficiently divided. Furthermore, in the present embodiment, since binarization is performed based on the magnitude of the luminance ratio between adjacent pixels, it is possible to divide an area of a shaded document image.

For example, if the original image pattern is G, the illuminance pattern is F, and the observed image pattern is Y, Y is F
And G (Y = F × G or log Y = log F + log G). Therefore, if the change in the illuminance pattern F is sufficiently gentle compared to the change in the observed image pattern G,
Even in an image subjected to shading, it can be considered that the luminance ratio of adjacent pixels is preserved.

On the other hand, the luminance difference between adjacent pixels is not preserved, and is small in a dark part and large in a bright part, so that an edge cannot be correctly detected in a dark part. That is, the edge extraction is performed by taking the luminance ratio instead of the luminance difference between the adjacent pixels, so that it is possible to divide the area of the document image that is strongly shaded. Further, in the present method, a slight change in luminance is detected in a low luminance portion, so that not only an edge but also a low luminance portion having a luminance change can be simultaneously extracted.

As a modification for obtaining the same effect, the edge extraction / binarization processing section 46 in FIG.
This is made possible by the configuration shown in FIG. Instead of taking the luminance ratio, the logarithmic processing circuit 87 finds logarithmic luminance, multiplies it by a predetermined value, stores it in the gradation image memory 51, and scans it vertically and horizontally to obtain the difference, This is binarized.

Such a binarization operation will be described with reference to FIG.

In FIG. 21, logarithmic luminance data 88 is
The logarithmic luminance data of the portion corresponding to 68 stored in the gradation image memory 51 is scanned in the horizontal direction as shown in the figure to obtain the difference between adjacent pixels.
Is exceeded, the pixel with smaller logarithmic luminance is set to '1'
(Black) and binarize it, and store the binary image memory 1 in the horizontal direction.
4 is stored. The binary image 41 is a binary image corresponding to the partial image 68 stored in the horizontal binary image memory 14. Hereinafter, the same processing as in the first embodiment is performed.
The same effect as in FIG. 15 can be obtained.

FIG. 22 shows the structure of an image processing apparatus according to a fourth embodiment of the present invention. This fourth
In this embodiment, the edge extraction / binarization processing unit 46 shown in FIG. 7 of the second embodiment is modified, and the other components have the same configuration. In the components of the fourth embodiment, the same components as those shown in FIG. 7 are denoted by the same reference numerals, and description thereof will be omitted.

In this image processing apparatus, a differential filter processing circuit 89 is a circuit for extracting an edge from a black-and-white gradation image stored in the gradation image memory 51. The binarization processing circuit 91 is a processing circuit for binarizing the edge image stored in the edge image memory 90.

In the image processing apparatus configured as described above, the differential filter processing circuit 89 includes the gradation image memory 5
An edge is extracted by applying a differential filter to the black-and-white gradation image stored in No. 1 and stored in the edge image memory 90.

Various types of differential filters can be considered. For example, a Laplacian filter is used. In the binarization processing circuit 91, the edge image stored in the edge image memory 90 is binarized and stored in the binary image memory 58. Various binarization processes are conceivable. For example, a maximum value is obtained for a total of 9 pixels, that is, a target pixel and eight neighboring pixels, and the target pixel is binarized using 1/2 as a threshold value. In addition, it is of course possible to perform binarization by combining the average, the variance, the difference between the maximum and minimum values, and the like. When binarization is performed using a fixed threshold, the differential filter processing and the binarization processing can be performed simultaneously with another configuration example as shown in FIG.

Referring to FIG. 24, the operation of the differential filter processing and the binarization processing will be described.

In FIG. 24, a partial image 92 is an enlarged view of a part of a document image, and one square represents one pixel. The gradation image data 93 is a part of the image data corresponding to the partial image 92 stored in the gradation image memory 11, and is stored in the Laplacian filter 9.
4 to obtain edge image data 95. For example,
The maximum value is obtained for a total of 9 pixels, that is, the target pixel and eight neighboring pixels, and the target pixel is binarized using 1/2 as a threshold value.
The edge image 96 thus obtained is stored in the binary image memory 5.
8 and stored. In the following image processing, area division is performed by the same processing as in the second embodiment.

As a result, it is possible to divide the image area without losing the edge of the image area. Further, since the image is cut out in a rectangular area, the area separation of the document image can be efficiently performed.

Therefore, the image processing apparatus according to the fourth embodiment performs edge extraction and binarization using a differential filter before performing image reduction processing. And the image area can be efficiently divided without losing the edge of the image area. Further, in the present embodiment, since a two-dimensional differential filter is used as the edge extracting means, it is not necessary to scan the gradation image vertically and horizontally, and the configuration can be simplified.

Note that, by recognizing the directionality of the picture or the like of the target image in advance, and by appropriately setting the coefficients of the differential filter, it is possible to divide only the image region connected in a specific direction. Selective and adaptive area division according to the characteristics of the target image can be performed.

Next, FIG. 25 shows the structure of an image processing apparatus according to a fifth embodiment of the present invention, and will be described. This fifth
In this embodiment, the edge extraction / binarization processing unit 46 shown in FIG. 7 of the second embodiment is modified, and the other components have the same configuration. In the components of the fifth embodiment, the same components as those shown in FIG. 7 are denoted by the same reference numerals, and description thereof will be omitted.

In this image processing apparatus, a short white run detecting circuit 97 that scans the reduced image stored in the reduced image memory 60 vertically and horizontally and detects a white run (short white run) having a length equal to or less than a predetermined value. A white → black replacement circuit 98 for replacing the short white run detected by the short white run detection circuit 97 with a black run,
A horizontal block image memory 99 for storing a block image obtained by replacing short white runs with black runs by scanning in the horizontal direction.
And a vertical block image memory 1 for storing a block image obtained by replacing short white runs with black runs by vertical scanning.
00 and an AND operation circuit 101 that obtains a logical product of the block images stored in the block image memories 99 and 100 to obtain a block image.

In the image processing apparatus configured as described above, the short white run detection circuit 97 scans the reduced image stored in the reduced image memory 60 and outputs a white run (short white run) having a length equal to or less than a predetermined value. ) Is detected, and the pixel value in the detected white run is inverted from “0” (white pixel) to 1 (black pixel) by the white → black replacement circuit 98. This scanning is performed vertically and horizontally, and is stored in the horizontal block image memory 99 and the vertical block image memory 100. The AND operation circuit 101 calculates the AND of the block images stored in the image memories 99 and 100, generates a block image, and stores the block image in the block image memory 63.

This operation will be described with reference to FIG.

In FIG. 26, a partial image 102 is an enlarged view of a part of the reduced image stored in the reduced image memory 60. One cell represents one pixel. This is scanned in the horizontal direction, and for example, a white run having a length of less than 16 pixels is detected and replaced with a black run. The horizontal block image thus obtained is stored in the horizontal block image memory 99.

The horizontal block image 103 is an image of a horizontal block of a portion corresponding to the partial image 102, and a hatched portion indicates a replaced black run. Similarly, a vertical block image obtained by scanning the reduced image in the vertical direction and replacing short white runs with black runs is stored in the vertical block image memory 100.

The vertical block image 104 is an image of a vertical block of a portion corresponding to the partial image 102, and a hatched portion indicates a replaced black run. The logical product of the block images stored in the horizontal and vertical block image memories 99 and 100 is calculated by a logical product calculation circuit 101.
A block image is generated. The block image 105 is
This is a block image of a portion corresponding to the partial image 102 generated from the logical product of the horizontal and vertical block images 103 and 104, and the hatched portion represents the replaced black pixel. Less than,
In this embodiment, image processing similar to that of the second embodiment is performed to perform region division.

As a result, it is possible to divide the image area without losing the edge of the image area. Further, since the image is cut out in a rectangular area, the area division of the document image can be efficiently performed.

Therefore, the image processing apparatus of the fifth embodiment performs the short white run / black run conversion in the block processing for integrating the black pixels in the reduced image, so that the black pixels in the reduced image are almost completely converted. The image area can be integrated into a rectangular block, and the image area can be efficiently divided.

In the image processing apparatus described above, the edge extraction / binarization processing section 46, reduced image processing section 47, block processing section 48, and rectangular area extraction section 49 are described in the second to fifth embodiments. However, the present invention is not limited to this, and may be configured using other known techniques. Also, these combinations are not limited to those described in the embodiments, and it is a matter of course that each processing unit can be replaced with another.

If the direction of the character string is known in advance, the pixels need only be integrated in that direction, so that it can be realized with a simpler configuration.

Further, it goes without saying that the image processing in the apparatus shown in these embodiments may be performed on software.

FIG. 27 shows the structure of an image processing apparatus according to a sixth embodiment of the present invention.

This image processing apparatus is roughly classified into a similar image region extracting unit that divides an input digital image (mixed image) into rectangles for each similar image region while keeping the image type unknown, and a divided similar image region. An image type determination unit that determines and determines the image type of the partial area.

In this image processing apparatus, a document file in which different types of images, for example, typeface characters, handwritten characters, photographs, picture images, and the like are mixedly described,
A full-color image is picked up by a video camera or an image scanner and input to the digital image converter 111 as an analog image input signal.

The digital image converter 111 converts an analog image input signal and outputs a digital image signal. The output data is stored in the image frame memory 112 once.
And is temporarily stored. The stored image data and the transmission image data already converted to the digital image signal are directly input to the image frame memory 112 and similarly temporarily stored.

The digital image data output from the image frame memory 112 is
3 is input to the same-type image region dividing unit 114 that forms the third image.
The same-type image region dividing unit 114 performs a process of extracting a boundary of an area where an image of the same type is present in a state where the image type is unknown and the background and dividing the image.

A plurality of similar image areas divided from one mixed image are each calculated as a rectangular area, and divided area address data indicating the horizontal and vertical positions of one determined edge angle, and the size of the rectangular area ( Width / height) is output. As a specific division processing method, various methods conventionally devised may be used, or the embodiment shown in FIG. 6 may be used. For example, in order to reduce the processing load by reducing the data to be handled, a full-color image output from the image frame memory 112 is first converted to a monochrome image, and further binarized.

Then, the binarized binary image is reduced, filtered with a low-pass filter to extract edges, and the extracted edge image is binarized. The edge portion of the binarized edge image is approximated by a line segment in each of the horizontal and vertical directions to extract a rectangular boundary. In this manner, the same type image area can be obtained.

The divided area address data and the size output from the same type image area dividing section 114 are input to the search block area selecting section 115. The search block area selection unit 115 selects which part of the obtained same-type image area is actually used as the target area for the image type determination. The selected region of interest is a rectangular small block region of a predetermined size having a size that is set in advance or adaptively changes according to the size of the same type of image region.

FIG. 28 is a simplified illustration of one mixed image thus divided. In the case of this figure, nine divided similar image areas A to H exist. For example, the determination target block is placed in the same type image region H, moves in this region, and the subsequent determination process is performed, and the image type of the same type image region H is specified. This rectangular small block area is a determination target block that is actually a target of image type determination in processing described later, and its size is at least smaller than the size of the same type image area.

The position of the determination target block selected by the search block region selection section 115 is selected a plurality of times in the same type image region so as not to overlap the same position. The data of the position and size of the selected determination target block is input to the image frame memory 112, and the image data of the corresponding image memory portion is output from the image frame memory 112.

The full-color image of the block to be determined output from the image frame memory 112 is stored in the block memory
7 and temporarily stored. The full-color image data output from the determination target block image memory 117 is input to the monochrome image conversion unit 118 and converted into monochrome image data including luminance level data.

In this embodiment, it is assumed that the image type is determined based on only the luminance information of the image and that the color information is not used. This is, in part, to reduce the data to be handled in the preceding stage as much as possible in the entire processing process, to reduce the load of the calculation processing, and to make the processing more practical. One reason is that even if the entire input image or a part of the input image is a monochrome image, it does not affect the determination of the image type at all, and realizes highly versatile processing.

The monochrome image data of the judgment target image output from the monochrome image conversion unit 118 is first input simultaneously to the horizontal difference detector 119 and the vertical difference detector 120, and the output data is stored in the buffer memory 121 and the buffer memory 121, respectively. 122 and temporarily stored. In the horizontal difference detector 119, in the image data composed of the luminance components of the respective pixels in the determination target area output from the monochrome image converter 118, the difference value between the luminance level of an arbitrary pixel and an adjacent pixel is determined in the horizontal direction. Is calculated.

Specifically, if the monochrome image of the image to be determined is a rectangular image of M × N pixels, and the horizontal and vertical pixel arrays are described by i and j, respectively, as shown in FIG. The luminance level of the arbitrary pixel is y _ij, where i, j: 0, 1, 2,..., M−1 (1), and the horizontal difference value Δh _{ij to be obtained} is Δh _ij = y _{(i + 1 ) j−} y _ij where i, j: 0, 1, 2,..., M−2 (2) Similarly, the vertical difference value Δv _ij becomes Δv _ij = y _{i (J + 1)} −y _ij where i, j: 0, 1, 2,..., N−2 (3).

The calculation results of the horizontal difference map and the vertical difference map formed by these horizontal / vertical difference values are temporarily stored in the buffer memory and input to the brightness correction parameter calculation unit 123 to correct the brightness. Are calculated.

In the brightness correction parameter calculating section 123, the process proceeds according to the block diagram shown in FIG. The average of the difference data of the portion from which the steep edge portion was removed from the horizontal difference map was averaged the horizontal difference (avrΔh), and the difference data of the portion from which the steep edge portion was removed from the vertical difference map was averaged. This is called an average vertical difference (avrΔv) and calculated.

As shown in FIG. 29C, the horizontal direction (H), the vertical direction (V) and the difference value (D) in the image to be determined are determined.
In a three-dimensional orthogonal space (H, V, D)
0, avrΔh) and a vector v (0, 1, avrΔv) make a normal vector n to a plane vector p (h, v, d), and a plane vector p = α × vector h + β × vector v (4) Vector n = (avrΔh, avrΔv, -1) (5)

A bit map in which a steep edge portion on the horizontal difference map is “0” and the others are “1”, and a steep edge portion on the vertical difference map is “0” and others are “1”. A bitmap "" is created at the time of the above processing, and a bitmap obtained by calculating a logical sum of these two bitmaps is used as a mask pattern. However, the steep edge is determined by whether or not the absolute value of the difference value is larger than a predetermined threshold. In this way, the normal vector n and the mask pattern are output from the luminance correction parameter calculation unit 123 as the luminance correction parameters.

The normal vector n and the mask pattern data output from the brightness correction parameter calculation unit 123 are input to the brightness level correction processing unit 124, and the brightness of the determination target block is input from the monochrome conversion unit 118, which is input simultaneously. Modify the data. Brightness level correction processing unit 124
In the brightness level correction process performed in step (1), the process shown in FIG. 31 is performed as follows.

First, as shown in FIG. 31A, the brightness level image data from the monochrome conversion unit 118 is multiplied by the previously obtained mask pattern, and the brightness data at the steep edge portion is removed. Next, FIG.
In the horizontal / vertical / luminance three-dimensional orthogonal space (H, V, Y) of the determination target block as shown in (b), the coordinate axis having the direction of the obtained normal vector n is changed to the modified coordinate axis m.
Y is set, and for all the pixels in the block, the corrected brightness level my _ij is calculated by projecting the brightness level y _ij on the corrected coordinate axis mY, and the frequency is obtained for each corrected brightness level to calculate the corrected brightness histogram. . The corrected luminance histogram is normalized with respect to the corrected luminance.

The frequency distribution in the hatched portion in FIG. 31B shows the frequency distribution of the luminance level before correction.

There are various normalization methods. For example, the corrected luminance histogram is f (my) where my = 0 to 255 (6), and the integration frequency at an arbitrary corrected luminance my _{ij obtained} by integrating the corrected luminance from 0 to the maximum is g (my). g (m
y) is

[0140]

(Equation 1) And it expressed, to the sum frequency of the histogram and g _t. Normalization to modify luminance, and g (my) / g _t = minimum Fixed my satisfying 5% luminance level min _my, g (m
y) / g _t = 95% is assumed to be the maximum corrected luminance level max_my, and the normalized corrected luminance level can be obtained from an arbitrary corrected luminance level my _{ij according} to the following equation.

[0141]

(Equation 2) This is a normalization process in which the brightness level range of the histogram of the corrected brightness level is obtained and linearly converted to a fixed predetermined range (normalization in the direction of the my axis in FIG. 31B). When the minimum corrected luminance level and the maximum corrected luminance level are determined, the ratio of the integrated frequency of the histogram to the total frequency is not limited to the above examples of 5% and 95%, but may be set to appropriate values.

[0142] Although a next normalization of the frequency which is intended to normalize along the peak of power and have you in FIG. 31 (b)
Is the normalization in the frequency axis direction.

As described above, the brightness level correction processing section 124
Then, any pixel (i, j) of the determination target block from the monochrome conversion unit 118 is corrected to a luminance level my _ij , which is normalized in the x-axis direction, and finally normalized normalized luminance level nmy _ij Is required.

A series of brightness correction processes performed by the brightness correction parameter unit 123 and the brightness level correction processing unit 124 are performed when low-frequency illumination unevenness or the like is acting on the image of the block to be determined. Performing the process of reducing,
This is performed for the purpose of relieving that the difference in the dynamic range of the image of the block depends on the distribution shape of the histogram. Also, compared to using a normal high-pass filter for the purpose of removing low-frequency components, there is no convolution operation and the processing is simple. After all, the brightness level correction processing unit 124
From the calculated corrected normalized brightness level nmy _ij
Is output.

In FIG. 27, the three characteristic parameters of the horizontal difference value Δh _ij output from the buffer memory 121, the vertical difference value ΔV _ij output from the buffer memory 122, and the normalized corrected luminance level nmy _ij are represented by vector A new three-dimensional orthogonal space (Δ is input to the quantizer 125 and defined using these feature parameters as components.
H, ΔV, nmY), the feature vector f (Δh, Δ
v, nmy) is vector-quantized.

FIGS. 30A to 30D are diagrams showing the distribution of occurrence of feature vectors for each image type.
(A) is a typeface character, (b) is a handwritten character, (c) is a photograph image, (d) is a figure showing an example of a feature vector in a picture image. In each figure, the vertical axis represents the corrected brightness nm
Y (modify Y), and the vertical axis represents the vertical difference value ΔV (d
delta V) and the horizontal difference value ΔH (delta H)
The distribution of each feature vector is plotted on a three-dimensional orthogonal space in which is taken. From these figures, it can be seen that it is possible to determine the image type according to the difference between the feature vectors.

The vector quantizer 125 receives and processes the feature vectors f _ij (Δh _ij , Δv _ij , nmy _ij ) of all the arbitrary pixels (i, j) in the block to be determined. A quantized vector, which is an output of the vector quantizer, is calculated in advance from a plurality of determination target block images having a plurality of image types, and the three-dimensional intersecting space (Δh, Δv, nmy) is calculated. From the occurrence distribution plotted in That is, the generation distribution is optimally divided into a predetermined number of local spaces so that the probability of occurrence of the feature vector in each local space is equal,
The average vector in each local area is set as a representative vector of the local area. These representative vectors are the quantization vectors.

There are various methods for calculating a specific representative vector. One example is the LBG algorithm [“An Algorithm for Vector Quantizer Desirable”.
gn "YOSEPH LINDE etc. IEEE TRANSACTION ON COMUNICAT
IONS, VOL.COM-28, N0.1 JANUARY 1980｝], the Euclidean distance between all the feature vectors existing in the local area and the representative vector of the local area is the representative of other local areas. The representative vector is determined so as to be the shortest with respect to the Euclidean distance from the vector.

Therefore, if vector quantization is executed in a straightforward manner, when an arbitrary feature vector is input to the vector quantizer 125, as described above, Euclidean processing is performed on all of the predetermined number of representative vectors obtained in advance. The distance is measured and the one with the shortest distance is selected. The selected representative vector is output as a quantized vector of the input feature vector.

When the vector quantization is performed in a straightforward manner as described above, the load of the calculation processing generally increases.

In this embodiment, a vector quantizer based on a hierarchical feedforward neural network is used.

FIG. 32 shows the configuration of the neural network. The feature vector f _ij (Δh _ij ,
Δv _ij , nmy _ij ) is obtained by inputting a three-dimensional input vector from the input unit 131 to the neural network 132, and during learning, the Euclidean distance calculating unit 133.
Is also input at the same time. Neural network 132
Is a feedforward hierarchical network.
An input input layer 134, an intermediate layer 135 having a predetermined number of layers and neuro elements, and an output layer 13 having K outputs
6.

The output from the output layer 136 is output to the output unit 1
37. Here, K is the number of vectors set in advance through trial and error, and matches the number of representative vectors described above. Learning of the neural network 132 constituting the vector quantizer is performed using the attached teacher data generation unit 138. Teacher data generator 138
Then, the data of the representative vector f _k output from the representative vector data memory 139 in which the data of the representative vector f _k [k = 0, 1,... A feature vector output from the input unit 131 is input to the Euclidean distance calculation unit 133, and an arbitrary feature vector, all representative vectors, and f
_{The k-} distance is calculated using the straightforward method above.

Next, the shortest distance selecting unit 140 selects the representative vector f _k having the shortest distance, inputs the selected representative vector f _k to the output unit 137, and associates the representative vector f _k with the K representative vectors f _k set in the output layer. Instructs any one of the output branches to fire. For example, the output to be fired is "1", and the others are "0". This represents the feature vector input to the neural network 132 and the quantization vector to be associated with the feature vector.

Such a series of learning processes is performed by a known back propagation method. A plurality of feature vectors are selected without specifying an image type from mixed images, and an output error is set to a set value. Repeated until: On the other hand, when learning of the neural network 132 is completed and all weights are determined, the teacher data generation unit 138 is cut off. That is, the neural network 132 receives the feature vector as an input,
This means that a function as a vector quantizer for outputting a corresponding quantization vector is provided.

With the vector quantizer constructed in this way, the amount of calculation can be reduced and the speed can be increased. However, when a neural network is used, a state in which the output is not necessarily limited to one for the input feature vector may occur, but in this case, even if there are a plurality of quantized vectors as the output, Regardless, the candidate quantization vector is handled.

A single quantized vector or a plurality of candidate quantized vectors output from the vector quantizer 125 are input to a quantized vector counting unit 126. In the quantization vector counting unit 126, the quantization vector f
_For each _k , the occurrence frequency is counted, and the quantization vector f
Generate a histogram of _k . Furthermore, the frequency of the histogram of the quantization vector is normalized such that the maximum frequency of the quantization vector _fk becomes a constant value in order to maintain the universality with respect to the change in the number of pixels of the block to be determined.

The histogram of the quantized vector f _k is input to the feature pattern determination unit 127, which checks the shape of the histogram and outputs the determined image type. The feature pattern determination unit 127 is configured by a feedforward hierarchical neural network. The input layer has K inputs of the number of quantization vectors. The intermediate layer has the number of layers and neuron elements appropriately determined by trial and error, and the output layer has outputs of the number of image types to be determined. For example, typescript,
If you want to determine the five image types of handwritten characters, photos, design images, and background images from the mixed images and separate them,
It will have 5 outputs.

The learning of the neural network by the feature pattern determination section 127 is performed by using the type of the block image to be determined selected by the search block selection section 115 having a correspondence relationship with the pattern shape of the histogram of the input quantization vector as the training data. To the output layer. When learning,
The image type of the determination target block as teacher data is determined by artificial recognition. The learning process is repeatedly performed by the back propagation method while changing the type of the determination target image in various ways until the output error becomes equal to or less than the set error. The feature pattern determination unit 127 utilizes the fact that there is a correlation between the shape (feature pattern) of the histogram of the input quantization vector and the type of image,
Recognition processing is performed by a neural network.
The reason for using a neural network is to avoid a huge amount of calculations that make comparisons with a large number of reference templates, as in general pattern matching, and even if the correlation between feature patterns and image types is slightly degraded. This is because it has an advantage that the influence on the determination probability is small. The feature pattern determination 127 for which the learning has been completed is an image type determination unit that determines and outputs an image type with respect to the histogram shape of the already input quantization vector.

The output of the characteristic pattern determination section 127 is input to the image type determination section 128, and the image type final determination of the determination target block is performed. Vector quantizer 12
Similarly to the case of 5, the neural network output of the feature pattern determination unit 127 may not output only one image type. Therefore, an evaluation criterion for the determination result is required, and a process of determining the evaluation criterion is required.

In the present embodiment, taking the configuration shown in FIG. 33 as an example, the input data corresponds to I ₁ , I _1
2, ~, when inputted as I _5, by determining the image type judgment unit 141, determination of whether to determine whether to hold the image type determination is made. If, I _1> th ₁ where, i = 1, 2, ... 5 and, (the maximum value of I ₁₎ over (second largest value of I ₁₎
> If th _2, the process moves to decision image type output unit 142, and image type max (I _i) image type determining the indicating the maximum value among the I _i. Otherwise, the decision suspension unit 143
And the image type is not determined. In addition,
th ₁ and th ₂ indicate predetermined threshold values, and are statistically preset so that the error probability of the image type finally determined in the present embodiment with respect to the artificially desired image type is minimized. The determined determination result of the arbitrary block image type to be determined is input to the determination result storage unit 144 and is temporarily stored. Next, returning to the search block area selection unit 115 again, a different determination target block is selected in the same type of image area, and the process up to the determination result accumulation unit 144 is repeated a predetermined number of times. Therefore, the determination result accumulation unit 144 includes
The determination results of different determination target blocks in the same type image area are accumulated. After a predetermined number of determination results of the determination result storage unit 144 are input to the majority decision unit 145 and a majority decision is made, the final determination of the corresponding image type of the same type of image region is output from the image type determination unit 128.

The finally determined image type output from the image type determination unit 128, the address data and the size of the same type image area are output from the mixed image image area separation apparatus of this embodiment.

FIG. 34 shows the structure of an image processing apparatus according to a seventh embodiment of the present invention. This seventh
The same reference numerals as in the sixth embodiment shown in FIG. 27 denote the same components in the embodiment, and a description thereof will be omitted. That is, the seventh embodiment has the same configuration up to the quantization vector counting unit 126 shown in FIG. 27, and differs from the configuration up to the final output thereafter.

As described in the sixth embodiment, the quantization vector counting unit 126 vector-quantizes a feature vector for an arbitrary determination target block in an arbitrary homogeneous image region, and the histogram frequency is normalized. Is output. Now, when the determination target blocks in an arbitrary homogeneous image area selected by the search block area selection unit 115 are numbered and identified according to the selection order, the normalized quantization vector becomes the quantization vector f _k [L] k = 0,1, ..., K
L = 0, 1,..., L. Here, it is assumed that the number of vectors to be quantized is K and the total number of selected determination target blocks is L.
In other words, L is the number of times of searching in an arbitrary same-type image area for each determination target block. The quantized vector f _k [L] output from the quantized vector counting unit 126 is input to each of the K counting and accumulating units 1 to K connected thereto, and is accumulated and added L times. and outputs a vector F _k. That is,

[0165]

(Equation 3) Is output. Incidentally, when it is desired to variously changed optionally is number of searches in order to avoid vector F _k is dependent L, and by dividing the vector F _k by L, and the frequency of the output histogram may be an average.

As described above, in order to obtain a histogram of a quantization vector from the accumulated frequencies of a plurality of determination target blocks in an arbitrary same-type image area, a series of processing steps from the search block area selection unit 115 to the count accumulation unit 151 Must be repeated a predetermined number of times (L times).

Then, the value vector F _k for which the number of searches is output
Is input to the feature pattern determination unit 127 configured in the same manner as in the sixth embodiment, and signals corresponding to the type of image to be identified, for example, I _{1 to} I ₅ are output.

Next, the output image type signals I _{1 to} I _1
5 outputs the image type that is input to the maximum likelihood determination unit 152 and indicates the maximum value as the maximum likelihood determination image type. In this embodiment, no suspension is performed when determining the image type.

As the final output from the mixed image image area separation apparatus of the present embodiment, the image type output from the maximum likelihood determining section 152, and the address data and the size of the same type image area as in the sixth embodiment are output. .

If the configuration of the seventh embodiment and the image type determination procedure are realized, the pattern of the histogram of the quantization vector associated with the image type will be the feature extraction that produces the effect of expanding the search area within the same type image area. A large population to carry out. In addition, it is excellent that the determination processing for each block unit is not required, even though the search area is expanded.

FIG. 35 shows the structure of an image processing apparatus according to an eighth embodiment of the present invention. here,
The same reference numerals as in FIG. 27 denote the same parts in the eighth embodiment, and a description thereof will be omitted.

Then, the normalized corrected luminance level nmy _ij is
The corrected luminance level is input to the corrected luminance histogram calculation unit 155, and the frequency is calculated for the normalized corrected luminance level nmy _ij to generate a histogram. The generated histogram is generated by linearly dividing the difference between the maximum value and the minimum value of the normalized corrected luminance level by a predetermined number S, and generating the minimum unit of division as the step width of frequency counting.

Therefore, the corrected luminance histogram calculator 15
The number of outputs from 5 is given as the number of steps S,
The number of steps S is a minimum number required and is determined in advance by trial and error so that the correlation between the image type and the histogram shape is appropriately maintained. Here, the output from the corrected luminance histogram calculation unit 155 is defined as y _s s = 0, 1,..., S.

On the other hand, the horizontal difference value h _ij and the vertical difference value v _ij temporarily stored in the buffer memories 121 and 122 are simultaneously read out and input to the gradient vector direction detection unit 156. Gradient vector direction detector 1
56 is the following equation (1) when h _ij > 0 θ = tan ⁻¹ (Δv _ij
/ Δh _ij ) (2) When h _ij <0 θ = tan ⁻¹ (Δv _ij)
/ Δh _ij ) + π. Here, θ is a gradient vector direction corresponding to an arbitrary pixel (i, j) of the determination target block.

The gradient vector azimuth θ is input to the gradient vector azimuth histogram calculation unit 157, and counts the frequency with respect to the gradient vector azimuth θ to generate a histogram. This generated histogram is linearly divided by a predetermined number R uniformly at a predetermined step angle, and the azimuth belonging to the minimum unit of division is used as a unit of frequency counting, as in the case of obtaining the corrected luminance histogram.

Therefore, the number of outputs from the gradient vector azimuth histogram calculation unit 157 is given by R, and the number of steps R is the minimum necessary number and the correlation number between the image type and the histogram shape is kept moderate. Is determined beforehand by trial and error. Here, the output from the gradient vector direction histogram 157 is defined as θ _r r = 0, 1,..., R.

[0177] Note that as generalization properties are not lost in the characteristic pattern of the histogram shape of the gradient vector orientation theta be different total counting time number, is that you normalized by the maximum frequency of theta _r as required desirable.

As described above, a total of [R + S] outputs of _ys and θ _r from the corrected luminance histogram calculation unit 155 and the gradient vector direction histogram calculation unit 157 are input to the feature pattern determination unit 127. Feature pattern determination unit 1
Reference numeral 27 denotes a hierarchical neural network.
The input layer has [R + S] inputs, the intermediate layer has an appropriately determined number of layers and neuron elements by trial and error, and the output layer has an output of an image type to be determined. For example, if it is desired to determine five image types, that is, a print character, a handwritten character, a photograph, a picture image, and a background image, from the mixed image and to separate them, there are five outputs.

The neural network learning of the feature pattern determination unit 127 is performed by the search block selection unit 115 selected by the search block selection unit 115 having a correspondence relationship with the pattern shape of the input corrected luminance histogram y _s and gradient vector azimuth histobram θ _r. Are given to the input layer as teacher data.

At the time of learning, the image type of the block to be determined as teacher data is determined by artificial recognition.
This learning process is based on the back propagation method.
Until the output error becomes equal to or less than the set error, the determination is repeatedly performed while variously changing the type of the image to be determined. The feature pattern determination unit 127 performs recognition processing using a neural network by utilizing the correlation between the shapes (feature patterns) of the input corrected luminance histogram y _s and the gradient vector direction histogram θ _r and the type of image. Is going. As in the sixth embodiment, the purpose of using the neural network is to avoid a huge amount of calculation for comparing and determining with a large number of reference templates as in general pattern matching, and to correlate the feature pattern with the image type. This is because the advantage is that even if the performance is slightly deteriorated, the influence on the determination probability is small. Feature pattern determination unit 127 learning is completed already to the histogram shape of the modified luminance histogram y _s and slope wavevector histogram theta _r inputted, it becomes an image type determination unit for outputting determined image type.

In the present embodiment, the following features are captured and used as indicating the correlation between the histogram shape of the corrected luminance histogram y _s and the gradient vector direction histogram θ _r and the image type.

FIGS. 36 and 37 show a comparison between a character pattern of a histogram shape and a print character, a photograph image, and a picture image as examples of the image type. FIG.
6 (a), (b) and (c) show corrected luminance histograms in which the horizontal axis represents the corrected luminance and the vertical axis represents the occurrence frequency (occurrence probability), and FIG. 37 (a), (b) and (c). The parentheses indicate the azimuth angle at every 15 degrees and the frequency counted at the step angle with respect to the azimuth direction. In the typeface character, the distribution of the luminance level has two biases, ie, high and low, indicating bimodality, and the direction of the gradient vector direction is remarkably directional at every 90 degrees. In a photographic image, the luminance level distribution is unimodal, and the gradient vector orientation does not show any direction-dependent performance. Further, in the picture image, the frequency distribution of the luminance level has multi-modality, and the gradient vector direction has a remarkable frequency every 45 degrees in addition to the frequency every 90 degrees seen in the previous typeface character. Shows direction dependence. As described above, the corrected luminance histogram and the gradient vector direction indicate a combination of characteristic patterns depending on the image type.

FIG. 38 summarizes the characteristics for the character type indicated by the two parameters. The handwritten character has a feature that the luminance histogram shows bimodality and the gradient vector has no direction dependency. In the background, the distribution of the luminance level shows a sharp unimodal property, but the distribution frequency should be set to zero if the occupancy level is below a certain threshold to avoid recognition errors with the monomodal property shown by the typeface characters. In this case, in this case, the frequency of the corrected luminance level is all zero, and there is a feature that the direction of the gradient vector does not depend on the direction.

Also, the luminance level distribution of the background has a smaller level range than that of the printable characters, so that the difference in the luminance level distribution can be clarified by the threshold judgment.

Next, the output of the feature pattern determination section 127 is input to the image type determination section 128, and the image type final determination of the determination target block is performed. Since it is conceivable that the neural network output of the feature pattern determination unit 127 does not output only one image type, the image type is determined using the processing shown in FIG.

The finally determined image type output from the image type determination unit 128, the address data and the size of the same type image area, are output from the mixed image area separation apparatus of this embodiment.

FIG. 39 shows a ninth embodiment as shown in FIG.
Modified luminance histogram calculator 15 in the configuration shown in FIG.
5 shows a configuration example in which the gradient vector direction histogram calculation unit 157 and subsequent units perform different processing. The processing content is the same as in the eighth embodiment. Here, the components shown in FIG.
The same reference numerals are given to members equivalent to 5 and their description is omitted.

As described in the present embodiment, the modified luminance histogram calculation unit 155 and the gradient vector direction histogram calculation unit 157 extract a feature vector for an arbitrary determination target block in an arbitrary homogeneous image region, and The frequency is normalized and output. The number of blocks to be determined in an arbitrary homogeneous image region selected by the search block region selection unit 115 is determined in accordance with the selection order, and the normalized feature vector is identified by the vector θ _L = [θ ₁ , θ ₂ , Θ ₃ ,..., Θ _R ] vector y _L = [y ₁ , y ₂ , y ₃ ,..., Y _S ] where L = 0, 1,. Here, L is the total number of selected blocks to be determined. L is the number of searches when searching in an arbitrary homogeneous image area in units of determination target blocks. Outputs from the corrected luminance histogram calculation unit 155 and the gradient vector direction histogram calculation unit 157 (vector θ
_L and the vector y _L ) are input to the R + S count frequency updating units R + S connected to the respective components, and are updated so as to become the maximum values of the components in the process of repeating the process L times.

That is, the components θ of those feature vectors
_R, y _S is a vector _{θ'= [max (θ 1)} , max (θ 2), max (θ 3), ...,
max (θ _R )] _L vector y ′ = [max (y ₁ ), max (y ₂ ), max (y ₃ ),.
max (y _S )] outputs _L. As described above, in order to obtain a histogram of an updated feature vector including a component of a feature vector indicating the maximum value of a plurality of determination target blocks in an arbitrary homogeneous image region, the search block region selection unit 115 to the count frequency update unit 159 Must be repeated a predetermined number of times (L times).

The feature vector θ ′ and the feature vector y ′, for which the number of searches has been output, are input to a feature pattern determination unit 127 formed of a hierarchical neural network similar to the feature pattern determination unit 127 of the seventh embodiment. hand,
A signal corresponding to an image type to be identified, for example, I _{1 to} I ₅
Is output. Output image type signals I _{1 to} I
₅ outputs the image type that is input to the maximum likelihood determining unit 160 and indicates the maximum value as the maximum likelihood determination image. In this embodiment,
When determining the image type, no suspension is performed.

As the final output from the mixed image image area separating apparatus of this embodiment, the image type output from the maximum likelihood determining section 160, and the address data and the size of the same kind of image area as in the eighth embodiment are output. . By realizing the configuration of the present embodiment and the image type determination procedure, the pattern of the corrected luminance histogram and the gradient vector azimuth histogram associated with the image type makes the feature more remarkable in a plurality of search regions within the same type image region. The vector components shown can be selectively extracted, and the difference in the feature pattern for each image type is clarified.

As described above, the image processing apparatuses of the second to ninth embodiments perform edge extraction /
By performing the binarization process, the edge of the image area is not chipped, and even for a document image in which black and white are inverted,
The image area can be efficiently divided.

Further, the image processing apparatus accurately divides a mixed image in which different image types are mixed for each image region of the same type, determines a determination target region in the image region of the same type, and determines a horizontal difference value, a vertical difference value, and a correction value. The image area of a mixed image having a high accuracy rate with relatively light load processing is obtained by calculating the statistics of the distribution pattern indicated by the three parameters of the luminance level, modeling and recognizing the distribution pattern, and determining the image type. Separation can be realized.

The classification and recognition of the distribution pattern are as follows.
In order to make the difference remarkable based on the statistical data, the shape distributed in the three-dimensional orthogonal space formed by the three parameters is patterned and modeled by vector quantization, and the shape of the modeled pattern is recognized by a neural network. Accordingly, it is possible to realize appropriate clustering based on statistics, which is similar to the case of artificial determination that enables accurate image type determination.

Further, the image-separated image is subjected to data processing, which is effective when performed according to the type of the image.
For example, it is possible to automate data compression, adaptive binarization, halftone processing, and various natural image processing (such as filtering) for achieving an intentional image reproduction effect.

FIG. 40 is a block diagram showing the structure of an image processing apparatus according to a tenth embodiment of the present invention. In the components of the tenth embodiment, the same components as those shown in FIG. 27 are denoted by the same reference numerals, and description thereof will be omitted. The position of the determination target block selected by the search block region selection unit 115 is selected a plurality of times in the same type image region so as not to overlap the same position. The data of the position and size of the selected determination target block is input to the image frame memory 112, and the image data of the corresponding image memory portion is output from the image frame memory 112.

The full-color image of the determination target block output from the image frame memory 112 is output to the determination target block image memory 117 constituting the image type determination unit 170.
And is temporarily stored. The full-color image data output from the determination target block image memory 117 is input to the monochrome image conversion unit 118 and is converted into monochrome image data including luminance level data.

In this embodiment, it is assumed that the image type is determined based on only the luminance information of the image and that the color information is not used. This is partly because, in the overall processing, data handled in the preceding stage is reduced as much as possible to reduce the load of calculation processing and to perform processing with higher practicality. One reason is that even if the entire input image or a part of the input image is a monochrome image, it does not affect the determination of the image type at all, and realizes highly versatile processing.

Next, the monochrome image data (for example, FIG.
1) is first input to the local luminance pattern detector 171, and the luminance y _{0 of the} pixel of interest and a predetermined number (N−
1) Local luminance pattern Y (y ₀ , y ₁ , y ₂ , y ₃ ,..., Y _N ) composed of luminances y ₁ , y ₂ , y ₃ ,.
_N-1 ) is detected. For example, three luminance components (y ₀ , y
FIG. 42 shows an occurrence frequency distribution for each image type obtained by plotting a local luminance pattern vector composed of ( ₁ , y ₂ ) in a three-dimensional space. It can be seen that each image type has a characteristic of the distribution shape.

The local luminance pattern Y output from the local luminance pattern detector 171, that is, the luminance combination Y (y
₀ , y ₁ , y ₂ , y ₃ ,..., Y _N-1 ) are input to the vector quantizer 125 and feature representative vectors f ₀ , f ₁ in an N-dimensional orthogonal space defined by these luminance components. ,
The vector is quantized to f ₂ ,..., f _k .

In the vector quantizer 125, a local luminance pattern vector Y is input and processed with all the arbitrary pixels in the block to be determined as the luminance of the pixel of interest. The quantized vector output from the vector quantizer 125 is obtained from the occurrence distribution plotted on an N-dimensional orthogonal space by calculating the local luminance pattern vector from a plurality of determination target block images having a plurality of image types in advance. .

Although various vector quantization methods can be used at this time, basically, the occurrence distribution shape of the local luminance pattern vector in the N-dimensional space must be determined so as to be distinguished depending on the image type. Further, it is desirable that the characteristics of the image type can be expressed by the minimum necessary representative vector. In this embodiment, the method of determining the representative vector is as follows.
It is important as a point.

The vector quantization in the tenth embodiment is as follows.
First, an occurrence frequency distribution as shown in FIG.
That is, for each image type of the image to be identified, by selecting the determination target block obtains the occurrence frequency distribution to the local luminance pattern vector Y _i. This processing is performed on different types of images with substantially the same number of blocks. Therefore, frequency distributions for all the selected blocks can be obtained. However, for one selected block, ideally all vectors Y _i , that is, if quantization of each luminance component is Q steps, i = 0, 1, 2,..., Q ^N −1 Is obtained, the number of components N of the vector Y _i is obtained.
Becomes larger, the number of vectors becomes enormous. Therefore, the quantization width may be increased and the number of quantization of each luminance component may be reduced as necessary. Let L be the total number of vectors.

The frequency itself x _im of an arbitrary local luminance pattern vector Y _i for each block to be determined is counted for all blocks and divided by the total block number M. As a result, a vector Y in block units as shown in FIG.
_i degrees numerical frequency distribution P _i (x _im) is obtained. The variance sigma _i numerical degree of the vector Y _i calculated for all vectors Y _i. That is,

[0205]

(Equation 4) It is.

Next, the vectors Y _i are rearranged in descending order of the variance σ _i . Those whose internal variance σ _i is equal to or smaller than a predetermined value are excluded as representative vectors and are removed, and the remaining vectors Y _i ′ are set as representative vectors for vector quantization. The predetermined value of the variance is a value that can achieve a satisfactory identification rate by identifying an image type using the above-described method when a vector having a predetermined value or more is used as a representative vector.

In the vector quantizer 125 shown in FIG. 40, the local luminance pattern vector Y _i detected from the determination target block is converted into the selected representative vector Y _i ′.
. That is, as the local luminance pattern vector Y _i , the representative vector having the shortest distance in the N-dimensional orthogonal space is set as the quantization vector. The distance may be the Euclidean distance (the square root of the sum of squares of the vector component difference) or the city area distance (the sum of absolute values of the vector component difference).

The purpose of obtaining the representative vector in this manner is as follows.

[0209] The features of each image type in this embodiment, trying to identify the shape of the occurrence frequency distribution of the vector Y _i. For a vector whose partial shape is significantly different depending on the image value, the variance σ _i becomes large. On the contrary, when the partial shapes are almost similar, the variance of the vector Y _i forming the shape is “0”. Shows a value close to. It is effective to identify image values, that is, to distinguish occurrence distribution shapes, by preferentially identifying portions having different distribution shapes. Therefore, it is because help the distribution shape by the image species on effectively identifies that approximates the distribution shape vector Y _i indicating the relatively large variance.

The quantization vector output from the vector quantizer 125 shown in FIG.
26, the cumulative frequency is counted, and a histogram is calculated. The histogram of the quantized vector g _k is input to the feature pattern determination unit 127, which recognizes its shape and outputs the determined image type. The feature pattern determination unit 127 is configured by a feedforward hierarchical neural network. The input layer has K inputs of the number of quantization vectors. The intermediate layer has the number of layers and neuron elements appropriately determined by trial and error, and the output layer has outputs of the number of image types to be determined. For example, typescript,
If you want to determine the five image types of handwritten characters, photos, design images, and background images from the mixed images and separate them,
It will have 5 outputs.

The learning of the neural network by the feature pattern determination unit 127 is performed by instructing the type of the image of the determination target block selected by the search block selection unit 115 having a correspondence with the pattern shape of the histogram of the input quantization vector. The data is provided to the output layer. At the time of learning, the image type of the determination target block as teacher data is determined by artificial recognition. This learning process is repeatedly performed by the back propagation method while the type of the determination target image is variously changed until the output error becomes equal to or less than the set error.

The feature pattern judging section 127 performs recognition processing using a neural network, utilizing the fact that the shape (feature pattern) of the histogram of the input quantization vector and the type of image are correlated. The reason for using a neural network is
The advantage is that it avoids a huge amount of calculation to compare and determine with a large number of reference templates as in general pattern matching, and that even if the correlation between the feature pattern and the image type is slightly deteriorated, there is little effect on the judgment probability. This is because The feature pattern determination unit 127 that has finished learning is an image type determination unit that determines an image type based on the histogram shape of the quantization vector already input and outputs the image type.

The output of the characteristic pattern determination section 127 is
The input is input to the image type determination unit 128, and the image type final determination of the determination target block is performed. The neural network output of the feature pattern determination unit 127 may not output only one image type.

Therefore, an evaluation standard (rule) for the determination result is provided, and the final determination of the image type of the corresponding same-type image area is output from the image type determination unit 128.

The finally determined image type output from the image type determination section 128, the address data and the size of the same type image area are output from the mixed image image area separation apparatus of this embodiment.

Next, an eleventh embodiment according to the present invention will be described. The configuration of the image processing apparatus according to the eleventh and subsequent embodiments is the same as that of the tenth embodiment shown in FIG. 40 except that the vector quantized by the vector quantization performed inside the vector quantization unit 125 is performed. Is different, and the description of the configuration is omitted.

The representative vector of the vector quantizer used in the eleventh embodiment is obtained as follows.

First, for each image type, the luminance y _{0 of the} target pixel and a predetermined number (N
-1) luminance _{y 1, y 2, y 3} , ..., vector Y _i (y _0, y ₁ as a topical luminance pattern consisting of y _N-1,
y _2, y _3, ..., repeatedly detect y _N-1). The vector Y _i detected in one block is the luminance component y _j
(J = 0,1,2, ..., N -1) if the quantization Q step, i = 0,1,2, ..., are present ways Q ^N -1, the number components of the vector Y _i N Becomes larger, the number of vectors becomes enormous. Therefore, the quantization width may be increased and the number of quantization of each luminance component may be reduced as necessary.

The value of the block unit occurrence frequency for each image type of an arbitrary vector Y _i detected in one determination target block is represented by Freq (Y _ij ), and Fr in a plurality of blocks
The occurrence frequency distribution of eq (Y _ij ) is calculated. Here, j is given by a serial number by the number of image types to be identified. For example, typeface characters = '0', handwritten characters = '1', photos = '2',
Numbers are assigned such as picture image = “3”.

Next, as shown in FIG. 44, for any Y _ij , the average occurrence frequency Avr (Freq
(Y _ij )), and the absolute value of the mutual average frequency is calculated. For example, if there are four types of images, six types of values are obtained. The minimum value [d
_min (Y _i )], and this value is similarly calculated for all the vectors.

The vectors Y _i are rearranged in descending order of the minimum value [d _min (Y _i )] of each vector, and a vector Y _i having a value equal to or larger than a predetermined value is set as a representative vector in the vector quantization.

The representative vector calculated in this manner is used as a vector quantized by the vector quantizer 125 shown in FIG. In the quantization method, as in the tenth embodiment, the representative vector having the closest distance in the N-dimensional orthogonal space is used as the quantization vector. The distance may be the Euclidean distance (the square root of the sum of the squares of the vector component differences) or the city distance (the sum of the absolute values of the vector component differences).

The average occurrence frequency Avr (Y _ij ) for each image type with respect to an arbitrary vector Y _ij gives the average of the occurrence frequency of the vector in block units for each image value. Therefore, it reflects the probability of taking the vector Y _i that is the image type indicates the most likely representative value of it. For the probability of taking a vector Y _i in the image value differences is small, so that hardly identify images species by the vector Y _i. Conversely, if the minimum value of the difference in the probability of taking a certain vector Y _i depends on the image type, the vector Y _i
In some cases, the image type can be easily identified, and the vector that can be easily identified is observed with priority, and if it is identified, effective identification becomes possible.

Next, an image processing apparatus according to a twelfth embodiment of the present invention will be described.

A representative vector of the vector quantizer used in the twelfth embodiment is obtained as follows. First, for each image type, the luminance y _{0 of the} pixel of interest and a predetermined number (N−1) adjacent to the luminance y _{0 of the} target pixel with a sufficient number of determination target blocks
Pieces of luminance _{y 1, y 2, y 3} , ..., vector Y _i (y ₀ as a topical luminance pattern consisting of _{y N-1, y 1,} y 2,
y ₃ ,..., y _N−1 ) are repeatedly detected. The vector Y _i detected in one block is calculated for each luminance component y _j (j
= 0, 1, 2,..., N−1), if there are Q steps, there are i = 0, 1, 2,..., Q ^N −1, but the number of components N of the vector Y _i is large. Since the number of vectors becomes enormous, a large quantization width may be used to reduce the number of quantization of each luminance component as necessary.

The value of the block unit occurrence frequency for each image type of an arbitrary vector Y _i detected in one determination target block is represented as Freq (Y _ij ), and Freq (Y _ij ) The occurrence probability distribution P (Freq (Y
_ij )). Here, j is given by a serial number by the number of image types to be identified. For example, numbers are given such as typeface character = '0', handwritten character = '1', photograph = '2', and picture image = '3'. Freq (Y _ij ) is normalized by dividing the total number of sample blocks for each image type.

Next, as shown in FIG. 45, at any Y _i , all combinations of arbitrary two image types j and j ′ are extracted, and their occurrence probability distributions P (Freq (Y _ij )) and P ( Freq (Y _ij ')) The overlapping areas, in other words, the simultaneous occurrence probabilities are calculated for the number of combinations.

The maximum value of the simultaneous occurrence probability [S
_max (Y _i )] and obtain the minimum value [S _{max of} each vector.
(Y _i)] Sorts vector Y _i in the order of small, its value as a representative vector in the vector quantization described above the vector Y _i indicating a predetermined value or less.

The representative vector thus calculated is used as a vector to be quantized by the vector quantizer 125 shown in FIG. In the quantization method, the representative vector having the shortest distance in the N-dimensional orthogonal space is used as the quantization vector as in the tenth embodiment. The distance may be the Euclidean distance (the square root of the sum of squares of the vector component difference) or the city area distance (the sum of absolute values of the vector component difference).

[0230] or simultaneous occurrence probability among the image type of the probability distribution for any vector _{Y i P (Freq (Y ij} )) can be identified at what probability the vector Y _i there are two images species different Represents If the co-occurrence probability is high, the error probability increases. Therefore, a certain vector Y _i
If the degree of similarity between the two image types is small, the discrimination of the image type by the vector Y _i is effective. Therefore, if the vector effective for identification is observed with priority, efficient image type identification becomes possible.

Next, an image processing apparatus according to a thirteenth embodiment of the present invention will be described.

The representative vector of the vector quantizer used in the thirteenth embodiment is obtained as follows. First, for each image type, the luminance y _{0 of the} pixel of interest and a predetermined number (N−1) adjacent to the luminance y _{0 of the} target pixel with a sufficient number of determination target blocks
Pieces of luminance _{y 1, y 2, y 3} , ..., vector Y _i (y ₀ as a topical luminance pattern consisting of _{y N-1, y 1,} y 2,
y ₃ ,..., y _N−1 ) are repeatedly detected. The vector Y _i detected in one block is calculated for each luminance component y _j (j
= 0, 1, 2,..., N−1), if there are Q steps, there are i = 0, 1, 2,..., Q ^N −1, but the number of components N of the vector Y _i is large. Since the number of vectors becomes enormous, a large quantization width may be used to reduce the number of quantization of each luminance component as necessary.

First, in an N-dimensional space where the local luminance pattern can be expressed as a vector, the vector is plotted with a sufficient number of samples for each image type. As a result, an occurrence probability distribution of a vector for each image type is obtained. As shown in FIG. 46, these distributions are vector-quantized into a predetermined number of vectors using the LBG algorithm. The LBG algorithm is, for example, [“An Algorithm for Vector Qu
antizer Design "YOSEPH LINDE etc. IEEE TRANSACTION
ON COMUNICATIONS, VOL.COM-28, NO.1 JANUARY 1980
] Is described in detail. In short, the representative vector is determined so that the Euclidean distance between all the feature vectors existing in the local region and the representative vector of the local region is the shortest with respect to the Euclidean distance between the representative vectors of other local regions. Is done.

Then, the calculated quantized vectors of the respective image types are combined to form one set of quantized vectors, and a vector having a large distance between individual vectors is set as a representative vector priority candidate. A predetermined number is selected from the representative vectors having the highest priority, and is set as the final representative vector. In a vector existence space in which the distances between the representative vectors are small, that is, the vector existence space has a dense distribution, it can be determined that the distribution shapes of the image types are similar. Therefore, by preferentially selecting vectors in a non-dense space, the distribution of occurrence of these vectors significantly represents the difference between the image types, and realizes efficient image type identification.

Next, an image processing apparatus according to a fourteenth embodiment of the present invention will be described.

The representative vector of the vector quantizer used in the fourteenth embodiment is obtained as follows. First, for each image type, a predetermined number of adjacent thereto and the luminance y ₀ of the target pixel in sufficient and and the same number of determination target blocks (N-1) pieces of luminance _{y 1, y 2, y 3} , ..., y N vector Y _i (y ₀ as a topical luminance pattern consisting _{-1,
y 1, y 2, y 3} , ..., repeatedly detect y _N-1).
If the quantization of each luminance component y _j (j = 0, 1, 2,..., N−1) is Q steps, the vector Y _i detected in one block is i = 0, 1, 2,. , Q ^N −1, but if the number N of components of the vector Y _i increases, the number of vectors becomes enormous. Therefore, a large quantization width is used to reduce the number of quantization of each luminance component as necessary. No problem.

Next, the frequencies of all possible vectors from the detected vector Y _i are counted. A predetermined number is selected as needed from the vector having the largest count value, and is used as a representative vector at the time of vector quantization. If the occurrence probability is low for any of the image types, the frequency does not differ depending on the image type, so that there is no need to use it for identification. Therefore, by removing such vectors that do not contribute to identification,
The number of inputs to the characteristic pattern determination unit 127 in FIG. 40 can be reduced, and an efficient identification unit configuration can be realized.

Next, an image processing apparatus according to a fifteenth embodiment of the present invention will be described. The representative vector of the vector quantizer used in the fifteenth embodiment is obtained as follows. First, for each image type, the luminance y _{0 of the} pixel of interest and a predetermined number (N
-1) luminance _{y 1, y 2, y 3} , ..., vector Y _i (y _0, y ₁ as a topical luminance pattern consisting of y _N-1,
y _2, y _3, ..., repeatedly detect y _N-1). The vector Y _i detected in one block is the luminance component y _j
(J = 0,1,2, ..., N -1) if the quantization Q step, i = 0,1,2, ..., are present ways Q ^N -1, the number components of the vector Y _i N Becomes larger, the number of vectors becomes enormous. Therefore, the quantization width may be increased and the number of quantization of each luminance component may be reduced as necessary.

As shown in FIG. 47, the histogram of each image type of the detected vector Y _i is calculated, and is divided by the maximum value of the frequency to normalize. Each of the obtained histograms is totally differentiated in N dimensions, and an absolute value is obtained. The fully differentiated N-dimensional histogram is binarized with a predetermined threshold value.

The N of each binarized image type is
The dimensional binary pattern is ANDed for all image types. In the binary N-dimensional pattern distribution after the logical product, LB
A predetermined number of representative vectors are extracted by applying the G algorithm (described in the thirteenth embodiment). In identifying the shape of the histogram, it is possible to approximate the shape more effectively by approximating the portion where the shape change is large with many vectors and the portion where the shape change is small with a small number of vectors. The shape of the differential pattern reflects the degree of shape change. Therefore, if the above processing method is used, a change in the distribution shape can be easily extracted in common for each image type, and more effective shape approximation of the N-dimensional histogram according to the image type can be realized.

Next, FIG. 48 shows the structure of an image processing apparatus according to a sixteenth embodiment of the present invention.

Also in the sixteenth embodiment, up to the local luminance pattern detecting section 172 of the image type determining section 170,
The configuration is the same as that of the tenth embodiment shown in FIG. The configuration and operation from the local luminance pattern detection unit 172 to the image type determination output, which are features of the present embodiment, will be described.

The local luminance pattern vector Y _i output from the local luminance pattern detecting section 172 is provided in parallel with the vector quantizer VQ constituting the vector quantizing section 173.
_{1, VQ 2, ..., VQ} j, ..., are input to the VQ _J. Here, J indicates the number of types of images to be identified. Vector quantizer VQ _j quantizes using the representative vector corresponding to the image type.

A representative vector of the vector quantizer VQ _j is obtained as follows.

First, for each image type, the luminance y _{0 of the} pixel of interest and the predetermined number (N
-1) luminance _{y 1, y 2, y 3} , ..., vector Y _i (y _0, y ₁ as a topical luminance pattern consisting of y _N-1,
y ₂ , y ₃ ,..., y _N-1 ) are repeatedly detected for L blocks. The vector Y _i detected in one block is represented by each luminance component y _j (j = 0, 1, 2,..., N−
If the quantization of 1) is a Q step, i = 0,1,2,2
.., Q ^N −1, but the number N of components of the vector Y _i
Becomes large, the number of vectors becomes enormous. Therefore, a large quantization width may be used to reduce the number of quantization of each luminance component as necessary. The obtained vector Y _i is, for each image type, in the N-dimensional space L
A predetermined number K of representative vectors are obtained using the BG algorithm.

The detected vector output from the local luminance pattern detecting section 172 and input to the vector quantizer VQ _j 173 is calculated by calculating the Euclidean distance from the previously calculated representative vector, and A representative vector having a small distance is selected, and the vector quantization is completed.

This process is executed simultaneously by each vector quantizer VQ _j . The output from the vector quantizer VQ _j is input to a variance sum calculator 174 to generate a frequency distribution of quantized vectors detected from one determination target block. Is calculated. That is, any vector quantizer V detected from any determination target block 1
If the vector (representative vector) quantized in Q _j is Y _jkL (k = 0, 1,..., K; L = 1, 2,..., L) and its occurrence frequency Freq (Yjk), The total E _j is

[0248]

(Equation 5) Given by

The variance sum E _j for each image type simultaneously output from the variance total calculator 174 is input to the evaluation unit 175, and the image type indicating the minimum value is output. The output image type is the identification result of the determination target block.

In this embodiment, the representative vectors used for vector quantization are calculated in advance so as to have a uniform occurrence distribution for each image type. Therefore, when a corresponding image type is input at the time of identification, the frequency of occurrence of the quantization vector becomes uniform, and the variance increases in other cases, so that it is possible to distinguish which image type is distributed.

The present invention is not limited to the above-described embodiments, and it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

[0252]

As described in detail above, according to the present invention, an input image in which various images are mixed can be divided into regions for each type of image, and for example, the entire mixed image can be compressed by image data compression suitable for various image types. It is possible to provide an image processing device that can increase the data compression ratio and improve the image quality of an image.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an image processing apparatus as an embodiment according to the present invention.

FIG. 2 is a diagram illustrating a configuration for performing black pixel extraction processing;

FIG. 3 is a diagram illustrating a configuration example for performing binarization processing and edge extraction processing;

FIG. 4 is a diagram illustrating a configuration for extracting a boundary of a continuous component area;

FIG. 5A shows a configuration in which features based on differences in image types are extracted from statistical data in advance to determine an evaluation criterion at the time of determination, and FIG. 5B shows a configuration using the obtained evaluation criterion. FIG. 4 is a diagram illustrating a configuration for determining an actual image type.

FIG. 6 is a conceptual configuration diagram of the image processing apparatus of the present invention.

FIG. 7 is a diagram showing a configuration of an image processing apparatus as a second embodiment according to the present invention.

FIG. 8 is a diagram for explaining a difference / binarization process of the image processing apparatus shown in FIG. 7;

FIG. 9 is a diagram illustrating an arc standing point removal filter according to a second embodiment.

FIG. 10 is a diagram for explaining label processing in the second embodiment.

FIG. 11 is a diagram for explaining rectangular region extraction in the second embodiment.

FIG. 12 is a diagram showing a first example of an output image in each processing step in the second embodiment.

FIG. 13 is a diagram showing a second example of an output image in each processing step in the second embodiment.

FIG. 14 is a diagram showing a third example of an output image in each processing step in the second embodiment.

FIG. 15 is a diagram showing a configuration of an image processing apparatus as a third embodiment according to the present invention.

FIG. 16 is a diagram for explaining the ratio / binarization process shown in FIG. 15;

FIG. 17 is a diagram showing a first example of an output image in each processing step in the third embodiment.

FIG. 18 is a diagram showing a second example of an output image in each processing step in the third embodiment.

FIG. 19 is a diagram showing a third example of an output image in each processing step in the third embodiment.

FIG. 20 is a diagram for describing logarithmic difference / binarization processing in the third embodiment.

FIG. 21 is a diagram illustrating another configuration when performing a binarization operation having the same effect as the third embodiment.

FIG. 22 is a diagram illustrating a configuration of an image processing apparatus as a fourth embodiment according to the present invention.

FIG. 23 is a diagram illustrating another configuration when binarization is performed using a fixed threshold value in the fourth embodiment.

FIG. 24 is a diagram for explaining edge extraction processing by a differential filter in a fourth embodiment.

FIG. 25 is a diagram showing a configuration of an image processing apparatus as a fifth embodiment according to the present invention.

FIG. 26 is a diagram for describing block processing by short white run / black run conversion in the fifth embodiment.

FIG. 27 is a diagram illustrating a configuration of an image processing apparatus as a sixth embodiment according to the present invention.

FIG. 28 is a diagram for explaining a method of dividing a mixed image into the same type of image area in the sixth embodiment.

FIG. 29 is a diagram for explaining a method of correcting a luminance level in the sixth embodiment.

FIG. 30 is a diagram for explaining a method of calculating a corrected luminance histogram in the sixth embodiment.

FIG. 31 is a diagram showing a configuration of a vector quantizer using a neural network used in an embodiment of the present invention.

FIG. 32 is a diagram for explaining a method of evaluating a determined image type result in the sixth embodiment.

FIG. 33 is a diagram illustrating a configuration example for describing image type determination using a plurality of determination target areas.

FIG. 34 is a diagram illustrating a configuration of an image processing apparatus as a seventh embodiment according to the present invention.

FIG. 35 is a diagram illustrating a configuration of an image processing apparatus as an eighth embodiment according to the present invention.

FIG. 36 is a diagram showing a second typical example of a feature pattern for each image type represented by the gradient vector direction and the corrected luminance level in the eighth embodiment.

FIG. 37 is a diagram showing a typical feature pattern distinguished by a gradient vector direction and a corrected luminance level in the eighth embodiment.

FIG. 38 is a diagram illustrating characteristics of character types indicated by two parameters in the eighth embodiment.

FIG. 39 is a diagram showing a configuration of an image processing apparatus as a ninth embodiment according to the present invention.

FIG. 40 is a diagram showing a configuration of an image processing apparatus as a tenth embodiment according to the present invention.

FIG. 41 is a diagram for explaining a luminance level pattern in the tenth embodiment.

FIG. 42 is a diagram showing an example of a distribution shape for each image type of a local luminance pattern in the tenth embodiment.

FIG. 43 is a diagram for describing vector quantization in the tenth embodiment.

FIG. 44 is a diagram for describing vector quantization in an eleventh embodiment according to the present invention.

FIG. 45 is a diagram for describing vector quantization in a twelfth embodiment according to the present invention.

FIG. 46 is a diagram for describing vector quantization in a thirteenth embodiment according to the present invention.

FIG. 47 is a diagram for describing vector quantization in a fifteenth embodiment according to the present invention.

FIG. 48 is a diagram illustrating a configuration of an image processing apparatus as a sixteenth embodiment according to the present invention;

FIG. 49 is a schematic configuration diagram of a conventional image processing apparatus.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image input part, 1a ... Optical system, 1b ... Solid-state image sensor,
2 ... color / monochrome conversion processing unit, 3 ... binarization processing unit,
4 ... reduction processing unit, 5 ... continuous component area boundary extraction unit,
6 image type determination unit, 7 data compression unit, 11, 12 ...
Difference operation processing unit, 13, 14,... Binarization processing unit, 15, 2
5: synthesis unit, 16, 17: black pixel reproduction unit, 20: boundary line calculation unit, 21, 22: difference calculation processing unit, 23, 24 ... 2
Value processing unit, 26: thinning processing unit, 27: LPF (low-pass filter), 28: edge extraction unit, 31: edge extraction unit, 32: KL conversion unit, 33: base vector extraction unit,
34 ... inner product operation unit, 35 ... neural network unit,
36 ... Teacher data input unit.

Continuation of the front page (56) References JP-A-4-326667 (JP, A) JP-A-1-144778 (JP, A) JP-A-1-181280 (JP, A) JP-A-61-296481 (JP) , A) JP-A-62-82771 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 1/40-1/409

Claims (3)

(57) [Claims] 1. An image processing apparatus which divides each region of a character string, a pattern, a photograph, and the like from a mixed image in which characters, pictures, photographs, and the like are mixed, wherein: an image memory for storing the mixed image; Edge extraction and binarization processing means for extracting an edge from the stored mixed image and converting it into a binary image; and dividing the binary image into small areas each having a predetermined number of pixels.
Reduced image processing means for reducing each small area to one pixel; block processing means for integrating and blocking the black pixel areas in the reduced image; and a range of black pixels connected by blocking the block processing means And a rectangular area extracting means for obtaining diagonal coordinates of an image area circumscribed rectangle from the image processing apparatus. 2. An image frame memory for storing mixed images in which characters, pictures, photographs, and the like are mixed, and image data output from the image frame memory divided into image areas of the same type, A similar image region dividing unit that outputs a position and a size, a search block region selecting unit that selectively extracts a small region of a predetermined size from within the same image region, and outputs the position and the size of the small region, Image data corresponding to the small area selected by the search block area selecting means is read from the image frame memory as a determination target area and temporarily stored in a determination target block image memory; and difference value data from the image data of the determination target area And a correction parameter for correcting the brightness based on the difference value data, and the correction parameter Brightness level correction processing means for calculating a corrected brightness level from the image data of the determination target area; horizontal / vertical difference value data from the difference detection means; and a corrected brightness level from the brightness level correction processing means. Vector quantization means for vector-quantizing the data; quantization vector counting means for counting each component of the quantization vector from the vector quantization means; and A neural network (neural network) for outputting an image type, and evaluating and determining a determination result of the output image type, and outputting the determined image type in combination with the position and size data of the same type image region. An image processing apparatus comprising: an image type determining unit. 3. An image input means for inputting image data, a determination target area selection unit for selectively extracting a small area of a predetermined size from the input image data, and an adjacent pixel from the image data of the determination target area Local feature pattern detection means for detecting a local feature pattern composed of a predetermined number (N) of pixels, vector quantization means for vector-quantizing the local feature pattern in an N-dimensional space, and quantization by the vector quantization means. Histogram generating means for counting the frequency of occurrence of the converted representative vector to calculate a histogram, and inputting the quantized vector histogram obtained by the histogram generating means as input, identifying a distribution shape thereof, and obtaining a desired image. An image type identification unit that outputs a type, an image type determination unit that obtains the identification result, and determines an image type, The image processing apparatus characterized by comprising.