JPH08251404A - Method and device for discriminating attribute of image area - Google Patents

Abstract

PURPOSE: To more easily and precisely discriminates the attribute of an area based on received image data. CONSTITUTION: A 1st neural network 12 in which at least a neuron number of one intermediate layer is smaller then the neuron number of an input layer, the neuron number of the input layer and that of an output layer are equal to each other and the identify mapping has already been learned is made up of a characteristic extract network 12a consisting of the input layer and the intermediate layers and of one or plural networks and made up of an area discrimination network 12b linked with the characteristic extract network 12a to receive an output from the characteristic extract network 12a and providing the output of a discrimination result as to the area attribute of image data based on the image data received by the input layer of the characteristic extract network 12a.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image area attribute discriminating method and apparatus used for image processing in a copying machine or the like.

[0002]

2. Description of the Related Art Conventionally, in a copying machine, various image processing is performed on multi-valued image data obtained by reading an image of a document in order to improve image quality.
The image processing in that case is performed according to the type of image.
For example, edge enhancement processing and binarization processing are performed on a character image in order to clarify the character, gradation processing is performed on a photographic image, and halftone processing is performed on a halftone image. Is subjected to smoothing processing to prevent moire.

There are cases where character images, photographic images, halftone images and the like are mixed in a copy document. In that case, it is necessary to divide the document image into respective areas. In the area division, a block area, which is a small area, is extracted from the document image, and the attribute of the extracted block area is determined.

For example, in Japanese Unexamined Patent Publication No. 4-114560, image data corresponding to a block area of 64 × 64 pixels is extracted from a document image, and a histogram feature amount and a linear density feature amount are extracted based on the extracted image data. Then, it has been proposed to input this to a neural network to determine the attribute.

Inoue et al.'S report, "Image Area Separation Method Using Neural Networks" (13th Simulation Technology Conference, Japan Society for Simulation Technology, June 1994), shows a small area of 8 × 8 pixels. It has been proposed to extract the average luminance and the maximum density difference in 1) as feature quantities and input them to a neural network to determine the attributes.

[0006]

In the above-mentioned various conventional proposals, as a neural network,
A three-layer perceptron including an input layer, an intermediate layer, and an output layer, or a modified version thereof is used. Further, various feature quantities extracted from the image data of the block area are used as input signals given to the neural network. That is, conventionally, the attributes are discriminated by the neural network by combining various characteristic quantities having physical meanings.

However, when it is necessary to discriminate the area attribute in real time as in a copying machine, the circuit for extracting the characteristic amount becomes complicated and it is difficult to configure the circuit itself, and the processing speed is high. It was also disadvantageous in terms of flexibility and cost. Further, it is extremely difficult to determine the threshold value for each feature amount for various copy originals, a lot of experience and know-how are required, and much time and labor are required for experiments for determining the threshold value. there were.

The present invention has been made in view of the above problems, and an object thereof is to more accurately and easily determine the attribute of the area based on the input image data.

[0009]

According to a first aspect of the present invention, there is provided a method wherein the number of neurons in at least one intermediate layer is smaller than the number of neurons in the input layer and the number of neurons in the input layer and the number of neurons in the output layer are the same. One neural network is trained to identify the identity map, and the input layer to the intermediate layer of the learned first neural network are used as the feature extraction sensor, and the output from the feature extraction sensor is , Inputting to the input layer of the second neural network, inputting image data having a specific region attribute as sample data to the input layer of the feature extracting sensor, and connecting coefficient of the feature extracting sensor. After the second neural network is trained without changing, the image data is input to the input layer of the feature extraction sensor and To determine the region attributes.

According to a second aspect of the present invention, the number of neurons in at least one intermediate layer is smaller than the number of neurons in the input layer and the number of neurons in the input layer and the number of neurons in the output layer are the same, and the identity map is learned. One or more feature extraction sensors formed from the input layer to the intermediate layer of the completed first neural network, and coupled so that the output from the feature extraction sensor is input. ,
A second neural network that outputs a determination result regarding the area attribute of the image data based on the image data input to the input layer of the feature extraction sensor.

According to a third aspect of the present invention, there is provided at least two of the feature extracting sensors, each input layer of which is connected in parallel with each other, and the second neural network includes one of the feature extracting sensors. First, image data whose area attribute is a flat portion of a photographic image has been input as sample data and has already been learned, and another one of the above-mentioned feature extraction sensors is used.
The image data whose area attribute is the edge portion of the photographic image and which has been learned by inputting as sample data is used.

[0012]

The feature extracting sensor extracts the feature amount of the input image data. The feature amount in this case is not a feature amount having a physically clear meaning. For example, when image data such as a character image or a photographic image is input as an input, the image properties such as “character image-likeness” and “photo image-likeness” are displayed.

From the second neural network, an output corresponding to the feature quantity extracted by the feature extracting sensor is obtained by learning. The neural network determines the area attributes such as the character area, the photograph flat area, the photograph edge area, and the halftone area for the input image data.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing a structure of an attribute discrimination device 1 according to the present invention, FIG. 2 is a view for explaining a block area BA extracted from a document PP, and FIG. 3 is a view for showing a structure of a neural network 12. 4, FIG. 4 is a diagram showing a process of creating the feature extraction network 12a, FIG. 5 is a diagram showing an example of an hourglass-type neural network NNA, FIG. 6 is a diagram showing an example of the region discrimination network 12b, and FIG. 7 is a neural network. 12 is a flowchart showing a method of creating No. 12.

The attribute discriminating apparatus 1 is incorporated in, for example, a digital copying machine (not shown). The image reader unit of the copying machine reads the document PP set on the document table, whereby multivalued image data DM of the image (original image) PM of the document PP is obtained. The image reader unit includes a line sensor having a reading density of, for example, 400 dpi, and scans the document PP in the vertical direction (sub-scanning direction) to generate image data DM with 256 gradations, for example.
Get. The attribute discrimination device 1 discriminates the attribute AT for the block area BA, which is a small area included in the original image PM, based on the obtained image data DM.

As shown in FIG. 1, the attribute discriminating apparatus 1 includes a block area extracting section 11 and a neural network 1.
It consists of two. The block area extraction unit 11 extracts image data DMa corresponding to the block area BA from the input image data DM. Block area B
A is, for example, a square area of 8 × 8 pixels, and is assigned to the original image PM so that the block areas BA do not overlap each other. The image data DM of the original image PM is input to the block area extraction unit 11 line by line or line by line. Therefore, for example, the input image data DM should be stored in an appropriate memory according to the pixel address. Thus, the image data DMa corresponding to the block area BA can be extracted.

The neural network 12 is output from the block area extraction unit 11 for each block area BA 6
Based on the four image data DMa, each block area B
The determination result as to whether the attribute AT of A is a character area, a photograph flat area, a photograph edge area, or a halftone dot area is output. The neural network 12 includes a feature extracting network 12a and a region identifying network 12b as a second neural network.

Next, a method of creating the neural network 12 will be described with reference to FIG. First, a method of creating the feature extraction network 12a will be described. That is, as shown in FIG. 4, four neural networks NNA (N
NA1 to 4) are prepared (step # 1). As shown in FIG. 5, each neural network NNA has five hourglass-type layers, and the number of neurons from the first layer (input layer) to the fifth layer (output layer) is 64, 60, and 10. , 6
0 and 64. That is, the neural network NN
In A, the number of neurons in the third layer (intermediate layer) is smaller than the number of neurons in the input layer, the number of neurons in the input layer is the same as the number of neurons in the input layer, and is symmetrical with respect to the third layer.

In each neural network NNA, the response function of the input layer S, the intermediate layer (third layer) Ab, and the output layer R is a linear function, and the intermediate layer (second layer) A
The response function of a and the intermediate layer (fourth layer) Ac is a sigmoid function. The portions from the first layer to the third layer are networks A1 to 4 as the feature extraction sensor, and the portions of the fourth and fifth layers are the networks B1 to B4.

Each of these neural networks NNA
On the other hand, input signals each having a specific distribution characteristic are input to the input layer to learn the identity map (step # 2).

That is, in the first neural network NNA1, learning is performed so that a learning character image is input to the input layer S and the same image as the input character image (restored character image) is output from the output layer R. Let In that case,
The input value to each input layer and the output value from each output layer corresponding to each input layer become equal.

Second neural network NNA2
For input, a learning photo flat image is input to the input layer S, and learning is performed so that the same image as the input photo flat image (restored photo flat image) is output from the output layer R.

Third neural network NNA3
For input, a learning photo edge image is input to the input layer S, and learning is performed so that the same image as the input photo edge image (restored photo edge image) is output from the output layer R.

Fourth neural network NNA4
, A halftone image for learning is input to the input layer S, and the same image as the input photo edge image (restored halftone image) is output to the output layer R.
Make it learn to output from.

The learning of the neural network NNA is
According to the well-known technique of back propagation. In learning, many samples of a character image, a photo flat image, a photo edge image, and a halftone dot image are created, and 8 × 8 pixel image data obtained from each sample is used as sample data in the input layer. Enter in S. For each neural network NNA, learning is performed so that the mean square error for the entire sample becomes small.

Each neural network NNA learns the characteristics of each image such as “character image likeness”, “photo flat image likeness”, “photograph edge image likeness”, and “halftone dot image likeness” in each intermediate layer Ab. Such a feature amount is acquired. That is, the feature amount of each image is acquired by learning the neural network NNA. However, the feature amount in this case is not a feature amount having a physically clear meaning. Each intermediate layer Ab has a smaller number of neurons than each input layer S. Therefore, the intermediate layer Ab has
It can be considered that the characteristics of the information input to the input layer S appear compressed or aggregated. Regarding the acquisition of the feature quantity in the middle layer of the neural network, the report of Irie et al., "Acquisition of Internal Representation by Multilayer Perceptron" (IEICE Transactions Vol. J73-
D-II, No. 8, P1173-8, August 1990)
Can be referred to.

Learned neural network NNA1
4 to the third to third layers (network A)
The extracted features 1 to 4) are the feature extraction network 12a (step # 3). In the feature extraction network 12a, the input layers S of the networks A1 to A4 are connected in parallel, and 64 (= 8 × 8) pieces of image data are simultaneously input to the networks A1 to A4. 40 (= 10 × 4) from the output layer in the networks A1 to A4
Data is output.

As shown in FIG. 6, the region discriminating network 12b has three layers, and the numbers of neurons in the input layer S, the intermediate layer A, and the output layer R are 40, 40, and 4, respectively. The response function of the input layer S is a linear function, and the response functions of the intermediate layer A and the output layer R are sigmoid functions. The output layer of the feature extraction network 12a is connected to the input layer S of the area discrimination network 12b (step # 4).

To the neural network 12 configured as described above, an input signal having the same distributive feature as the one learned by the neural network NNA is input and the coupling coefficient of the feature extracting network 12a is changed. Instead, the area discrimination network 12b is learned (step # 5).

That is, first, a learning character image is input to the neural network 12, and learning is performed so that the output of the neuron r1 of the output layer R of the area discrimination network 12b becomes "1". Next, a learning photo flat image is input, and learning is performed so that the output of the neuron r2 of the area discrimination network 12b becomes "1". Further, the photograph edge image and the halftone dot image for learning are sequentially input, and the neuron r3 of the area discrimination network 12b is input.
4 is learned so that the output of 4 becomes "1". In this learning process, the coupling coefficient of the area discrimination network 12b changes. The neural network N
The learning of NA is based on the back propagation method which is a well-known technique. The sample used for learning may be the same as or different from the sample used for learning the neural network NNA. Character image,
Learning is performed so that the mean square error becomes small for each of the photograph flat image, the photograph edge image, and the halftone dot image.

In this way, the neural network 1
2 is created. Created neural network 1
When the image data DMa is input to 2, the four neurons r1 to r4 of the output layer R obtain outputs S1 to 4 corresponding to the character area, the photograph flat area, the photograph edge area, and the halftone dot area, respectively. To be That is, the neural network 12 determines the attribute AT of the block area BA based on the data input to the input layer S of the networks A1 to 4, and when it is the character area, the output S1 of the neuron r1 is "1". , The output S2 of the neuron r2 is close to “1” in the case of a photograph flat area, and the output S3 of the neuron r3 is close to “1” in the case of a photograph edge area. , The output S4 of the neuron r4 is close to “1”.

Output S from the neural network 12
Based on 1 to 4, the attribute AT of the block area BA is determined. For example, when one output is "1", it is determined to be the area corresponding to the output. Or
When there is one output that exceeds a certain threshold, it is determined to be a region corresponding to that output. Alternatively, the area corresponding to the largest output is determined as the area.

Further, based on the attribute AT of each block area BA determined in this way, smoothing by morphology or the like is performed, thereby correcting the determination result for each block area BA and enlarging each area. Reduce misjudgment. As a result, the original image PM is divided into four areas, that is, a character area, a photograph flat area, a photograph edge area, and a halftone dot area.

For example, edge enhancement processing and binarization processing are performed on the character area, and processing for obtaining natural gradation or specific gradation is performed on the photograph flat area and the photograph edge image. A process of emphasizing is performed, and a smoothing process is performed on the halftone dot area to prevent moire.

A photographic image is, like a silver salt photograph,
It refers to a grayscale image having a sufficiently high pixel density with respect to the reading density of the original image PM, a photographic flat image is a portion where the density change is small, and a photographic edge image is a portion where the density change is large. The photographic flat image is distinguished from the white background portion of the character image. For example, the whitening process is performed on the white background portion of the character image, while the process utilizing the gradation is performed on the photographic flat image. Further, the difference between the dot image and the photographic image decreases as the dot becomes finer. For example, when the read density of the original image PM is 400 dpi and the dot density is 200 lines / inch, the read image data DM is not different from that of a photographic image. Therefore, in that case, 2
A halftone dot image of 00 lines / inch or more may be included in the photographic image.

According to the above-described embodiment, 64 extracted from the image data DM to the neural network 12 is used.
By directly inputting each piece of raw image data DMa, the attribute AT of the block area BA can be determined. That is, it is not necessary to give a feature amount as an input signal to the neural network 12, and the image data to be discriminated can be input as it is. Therefore, the input to the neural network 12 is easy.

Therefore, it is not necessary to previously extract a characteristic amount having a physical meaning as in the conventional case, and a circuit or a program therefor is unnecessary, and the circuit configuration, processing speed, flexibility, cost, etc. Is advantageous in that.

According to the above-described embodiment, since the neural network 12 is used to discriminate the attribute AT, the learning effect of the neural network 12 facilitates the attribute A.
The determination of T is performed, and the reliable determination of the attribute AT is performed.

Incidentally, if it is determined whether or not it is a halftone dot area by using a threshold value according to the spatial frequency spectrum component which is a feature amount without using the neural network 12, a halftone dot image with coarse mesh is obtained. Since the low-frequency spectrum component increases and the halftone dot image with fine meshes has the high-frequency spectrum component, it is possible to simply determine whether or not the halftone dot image is according to the spatial frequency spectrum component. First, it requires a lot of experience and know-how to determine the threshold, and inevitably causes a lot of misjudgment.

After the neural network 12 is trained, a process for discrimination is performed by a product-sum operation of input data and a coupling coefficient obtained by learning, and a search of a table showing a response function. Therefore, the processing speed of the calculation can be improved. In particular, since the coupling coefficients of the networks A1 to A4 do not change, the feature extracting network 12a can be easily configured by diverting only the coupling coefficient obtained by learning of the neural network NNA, and the arithmetic processing can be performed. The calculation speed is high because it can be simplified.

Therefore, in the copying machine using the attribute discriminating apparatus 1, it is possible to accurately divide the area of the original PP.
The image data DM obtained from the document PP can be subjected to appropriate processing in real time according to the area, and a clear copy image can be output.

In the above embodiment, the square area of 8 × 8 pixels is set as the block area BA, but 4 × 4 pixels,
Areas of various sizes such as 3 × 3 pixels, 16 × 16 pixels, and 64 × 64 pixels may be used as the block area BA. It does not have to be a square. Further, although the block areas are allocated so as not to overlap with the original image PM, they may be sequentially shifted and allocated so as to overlap with each other.

That is, as shown in FIG. 8A, with respect to the original image PM, for one pixel PXa whose attribute is to be discriminated, a block area BAa for 8 × 8 pixels around the pixel PXa is provided.
Image data DMa corresponding to the pixel PXa
And the block area BAa can be sequentially shifted by one pixel.

Further, as shown in FIG. 8B, with respect to 4 (= 2 × 2) pixels PXb whose attributes are to be discriminated, image data of a block area BAb for 8 × 8 pixels around the pixel PXb Can be extracted as the image data DMa corresponding to the pixel PXb, and the block area BAb can be shifted by two pixels.

In the above-described embodiment, four types of attribute determinations, that is, a character region, a photo flat region, a photo edge region, and a halftone dot region are performed, but it is configured to perform three or less types or five or more types of attribute determinations. You may.

In the above embodiment, the block area extraction unit 11 is realized by software by a memory storing programs and data and a CPU executing the programs. The neural network 12 is realized by a computer simulator.
Therefore, as described above, the neural network 12 can be realized by a table showing the learned coupling coefficient and response function, and a program for calculating and searching them. Such an aspect is also included in the neural network of the present invention. Further, the neural network may be directly realized by hardware.

In the above embodiment, the neural network NNA having five layers is used, but it may have four layers or less or six layers or more. Four networks A1
Although 4 was used, 3 or less or 5 or more may be used. Although a three-layer network is used as the area discrimination network 12b,
It may have four, five, or more layers. Although one area discriminating network 12b is connected to all the networks A1 to A4, separate area discriminating networks may be connected to the networks A1 to A4. The number of neurons in each layer, the presence / absence of coupling coefficient, the type of response function, the learning method, and the like can be variously changed other than the above. In addition, the configuration, processing content, processing order, etc. of each part or the whole of the attribute determination device 1 can be appropriately changed in accordance with the gist of the present invention.

[0048]

According to the inventions of claims 1 to 3,
The attribute of the area can be more accurately and easily determined based on the input image data.

After learning the neural network, information processing for discrimination is performed by sum-of-products calculation of input data and the coupling coefficient obtained by learning, and retrieval of a table showing a response function. Since it can be performed, the calculation speed can be improved.

As the coupling coefficient of the feature extracting network, the coupling coefficient of the learned first neural network can be diverted as it is, so that the feature extracting network can be easily constructed, and Since the calculation process can be simplified, the calculation speed can be improved.

According to the third aspect of the present invention, it is possible to determine whether it is a flat portion of the photographic image or whether it is an edge portion of the photographic image.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an attribute discriminating apparatus according to the present invention.

FIG. 2 is a diagram illustrating a block area extracted from a document.

FIG. 3 is a diagram showing a configuration of a neural network.

FIG. 4 is a diagram showing a process of creating a feature extraction network.

FIG. 5 is a diagram showing an example of an hourglass-type neural network.

FIG. 6 is a diagram showing an example of an area discrimination network.

FIG. 7 is a flowchart showing a method for creating a neural network.

FIG. 8 is a diagram for explaining another example of a block area allocation method.

[Explanation of symbols]

1 Attribute Discrimination Device (Image Region Attribute Discrimination Device) 12 Neural Network 12a Feature Extraction Network 12b Region Discrimination Network (Second Neural Network) NNA Neural Network (First Neural Network) A1-4 Networks (Feature Extraction Sensor) )

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Shinji Okamoto 2-3-13 Azuchi-cho, Chuo-ku, Osaka-shi, Osaka, Osaka International Building Minolta Co., Ltd. (72) Inventor Tetsuya Ito Azuchi-cho, Chuo-ku, Osaka-shi, Osaka 2-chome 3-13 Osaka International Building Minolta Co., Ltd.

Claims (3)

[Claims] 1. A first neural network in which the number of neurons in at least one intermediate layer is smaller than the number of neurons in the input layer and the number of neurons in the input layer and the number of neurons in the output layer are the same, the identity map is learned. The input layer to the intermediate layer of the learned first neural network are used as the feature extraction sensor, and the output from the feature extraction sensor is input to the input layer of the second neural network. Image data having a specific region attribute is input as sample data to the input layer of the feature extraction sensor, and the second neural network is learned without changing the coupling coefficient of the feature extraction sensor. After that, the image data is input to the input layer of the feature extraction sensor to discriminate the region attribute thereof. Image area attribute determination process. 2. The number of neurons in the at least one intermediate layer is smaller than the number of neurons in the input layer, the number of neurons in the input layer is the same as the number of neurons in the output layer, and the first neural network for which the identity mapping has been learned. One or a plurality of feature extraction sensors configured from the input layer to the intermediate layer, and an output of the feature extraction sensor are coupled so as to be input, and an input of the feature extraction sensor An image area attribute discriminating apparatus comprising: a second neural network which outputs a discrimination result about the area attribute of the image data based on the image data input to the layer. 3. The input layer has at least two feature extracting sensors connected in parallel with each other, wherein the second neural network has one of the feature extracting sensors having a region attribute of a photographic image. Image data that is a flat part has been input as sample data and has been learned,
The image area attribute determination device according to claim 2, wherein the image area attribute determination apparatus has already learned by inputting image data whose area attribute is an edge portion of a photographic image as sample data to another one of the feature extraction sensors.