JPH08249300A - Neural network and its forming method - Google Patents

Abstract

PURPOSE: To provide a neural network capable of inputting an input signal to be a discrimination object as it is by eliminating the need of giving a feature quantity as the input signal. CONSTITUTION: The neural network consists of a network for feature extraction 12a consisting of an input layer to an intermediate layer among a first neural network which is already learned so that the number of neurons in at least one intermediate layer is smaller than that in the input layer, the number of neurons in the input layer and that in an output layer are mutually equal and that an input value to each input layer and an output value from each output layer corresponding to each input layer are equal, and a second neural network 12b combined so that an output from the network for feature extraction 12a are inputted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network and a method of creating the neural network.

[0002]

2. Description of the Related Art A neural network is capable of processing information by learning many practical examples, and is also capable of performing complicated information processing at high speed, so that image recognition, speech recognition, speech synthesis, Various uses have been tried in the fields of automatic translation and the like. As an example of its use, there is area attribute discrimination in an image.

That is, for example, in a copying machine, various image processing is performed on multi-valued image data obtained by reading an image of a document in order to improve image quality. The image processing in that case is performed according to the type of image. For example, edge enhancement processing and binarization processing are performed on a character image in order to clarify the character, gradation processing is performed on a photographic image, and halftone processing is performed on a halftone image. Is subjected to smoothing processing to prevent moire.

There are cases where a copy document contains a mixture of character images, photographic images, halftone images and the like. In that case, it is necessary to divide the document image into respective areas. In the area division, a block area, which is a small area, is extracted from the document image, and the attribute of the extracted block area is determined.

In Japanese Patent Laid-Open No. 4-114560, image data corresponding to a block area of 64 × 64 pixels is extracted from a document image, and a histogram feature amount and a linear density feature amount are extracted based on the extracted image data. It has been proposed to input this to a neural network to discriminate attributes.

Inoue et al.'S report, "Image Area Separation Method Using Neural Network" (13th Simulation Technology Conference, Japan Society of Simulation Technology, June 1994), describes a small area of 8 × 8 pixels. It has been proposed to extract the average luminance and the maximum density difference in 1) as feature quantities and input them to a neural network to determine the attributes.

[0007]

However, in the past, as a neural network, an input layer, an intermediate layer,
And a three-layer perceptron consisting of an output layer and its improved version. Further, various feature quantities extracted from the image data of the block area are used as input signals given to the neural network.

Therefore, in the prior art, when an input signal is applied to a neural network, it is necessary to extract a feature quantity having a physical meaning in advance so that discrimination by the neural network can be accurately performed. It is necessary to carefully decide and decide what kind of characteristic amount to use. Since such a circuit or program is complicated and requires a lot of time and labor to create it, there is a limit in terms of the overall circuit scale, processing speed, flexibility and cost.

Generally speaking, when a neural network is used in a new field, it is often unclear what should be input as a feature amount for discrimination by the neural network. In such a case, it is expected that a large amount of time will be spent to create a circuit or a program for extracting the characteristic amount, and an accurate discrimination result by the neural network cannot be obtained.

The present invention has been made in view of the above problems, and it is not necessary to give a feature amount as an input signal, and a method for creating a neural network and a neural network capable of directly inputting an input signal to be discriminated. The purpose is to provide.

[0011]

According to a first aspect of the present invention, in a neural network, the number of neurons in at least one intermediate layer is smaller than the number of neurons in the input layer, and the number of neurons in the input layer is the same as the number of neurons in the output layer. Yes, from the input layer to the intermediate layer of the first neural network that has been learned so that the input value to each input layer and the output value from each output layer corresponding to each input layer become equal. It comprises a feature extraction network and a second neural network coupled so that the output from the feature extraction network is input.

The method according to the second aspect of the present invention is directed to the first neural network in which the number of neurons in at least one intermediate layer is smaller than the number of neurons in the input layer and the number of neurons in the input layer and the number of neurons in the output layer are the same. Then, inputting an input signal having a certain distribution characteristic to the input layer, and learning so that the input value to each input layer and the output value from each output layer corresponding to each input layer become equal, The input layer to the intermediate layer of the learned first neural network are used as the feature extraction network, and the output from the feature extraction network is input to the input layer of the second neural network. The input signal having the same distributive feature as the input signal is input to the input layer of the feature extraction network, and the feature extraction network is connected. Train the second neural network without changing the coefficient.

[0013]

The feature amount of the input data is extracted by the feature extracting network. The feature amount in this case is not a feature amount having a physically clear meaning. For example, when image data such as a character image or a photographic image is input as an input signal having a distribution characteristic, the image characteristics such as “character image-likeness” and “photo image-likeness” are expressed.

From the second neural network, an output corresponding to the feature quantity extracted by the feature extraction network is obtained by learning. The neural network performs various information processing such as image recognition, voice recognition, voice synthesis, and automatic translation according to the content of learning.

[0015]

1 is a block diagram showing the configuration of an attribute discriminating apparatus 1 having a neural network 12 according to the present invention,
2 is a diagram for explaining a block area BA extracted from the document PP, FIG. 3 is a diagram showing a configuration of the neural network 12, FIG. 4 is a diagram showing a process of creating the feature extracting network 12a, and FIG. 5 is an hourglass type. Neural network N
FIG. 6 is a diagram showing an example of NA, and FIG.
FIG. 7 is a diagram showing an example of b, and FIG. 7 is a flowchart showing a method of creating the neural network 12.

The attribute discriminating apparatus 1 is incorporated in, for example, a digital copying machine (not shown). The image reader unit of the copying machine reads the document PP set on the document table, whereby multivalued image data DM of the image (original image) PM of the document PP is obtained. The image reader unit includes a line sensor having a reading density of, for example, 400 dpi, and scans the document PP in the vertical direction (sub-scanning direction) to generate image data DM with 256 gradations, for example.
Get. The attribute discrimination device 1 discriminates the attribute AT for the block area BA, which is a small area included in the original image PM, based on the obtained image data DM.

As shown in FIG. 1, the attribute discriminating apparatus 1 includes a block area extracting section 11 and a neural network 1.
It consists of two. The block area extraction unit 11 extracts image data DMa corresponding to the block area BA from the input image data DM. Block area B
A is, for example, a square area of 8 × 8 pixels, and is assigned to the original image PM so that the block areas BA do not overlap each other. The image data DM of the original image PM is input to the block area extraction unit 11 line by line or line by line. Therefore, for example, the input image data DM should be stored in an appropriate memory according to the pixel address. Thus, the image data DMa corresponding to the block area BA can be extracted.

The neural network 12 is output from the block area extraction unit 11 for each block area BA 6
Based on the four image data DMa, each block area B
The determination result as to whether the attribute AT of A is a character area, a photograph flat area, a photograph edge area, or a halftone dot area is output. The neural network 12 includes a feature extracting network 12a and a region identifying network 12b as a second neural network.

Next, a method of creating the neural network 12 will be described with reference to FIG. First, a method of creating the feature extraction network 12a will be described. That is, as shown in FIG. 4, four neural networks NNA (N
NA1 to 4) are prepared (step # 1). As shown in FIG. 5, each neural network NNA has five hourglass-type layers, and the number of neurons from the first layer (input layer) to the fifth layer (output layer) is 64, 60, and 10. , 6
0 and 64. That is, the neural network NN
In A, the number of neurons in the third layer (intermediate layer) is smaller than the number of neurons in the input layer, the number of neurons in the input layer is the same as the number of neurons in the input layer, and is symmetrical with respect to the third layer.

In each neural network NNA, the response functions of the input layer S, the intermediate layer (third layer) Ab, and the output layer R are linear functions, and the intermediate layer (second layer) A
The response function of a and the intermediate layer (fourth layer) Ac is a sigmoid function. The portions from the first layer to the third layer are networks A1 to 4, and the portions of the fourth and fifth layers are networks B1 to B4.

Each of these neural networks NNA
On the other hand, input signals each having a specific distribution characteristic are input to the input layer to learn the identity map (step # 2).

That is, in the first neural network NNA1, learning is performed so that a learning character image is input to the input layer S and the same image as the input character image (restored character image) is output from the output layer R. Let In that case,
The input value to each input layer and the output value from each output layer corresponding to each input layer become equal.

Second neural network NNA2
For input, a learning photo flat image is input to the input layer S, and learning is performed so that the same image as the input photo flat image (restored photo flat image) is output from the output layer R.

Third neural network NNA3
For input, a learning photo edge image is input to the input layer S, and learning is performed so that the same image as the input photo edge image (restored photo edge image) is output from the output layer R.

Fourth neural network NNA4
, The learning halftone image is input to the input layer S, and learning is performed so that the same image as the input halftone image (restored halftone image) is output from the output layer R.

The learning of the neural network NNA is
According to the well-known technique of back propagation. In learning, many samples of a character image, a photo flat image, a photo edge image, and a halftone dot image are created, and 8 × 8 pixel image data obtained from each sample is used as sample data in the input layer. Enter in S. For each neural network NNA, learning is performed so that the mean square error for the entire sample becomes small.

Each neural network NNA learns the characteristics of each image such as “character image likeness”, “photo flat image likeness”, “photograph edge image likeness”, and “halftone dot image likeness” by learning. Such a feature amount is acquired. That is, the feature amount of each image is acquired by learning the neural network NNA. However, the feature amount in this case is not a feature amount having a physically clear meaning. Each intermediate layer Ab has a smaller number of neurons than each input layer S. Therefore, the intermediate layer Ab has
It can be considered that the characteristics of the information input to the input layer S appear compressed or aggregated. Regarding the acquisition of the feature quantity in the middle layer of the neural network, the report of Irie et al., "Acquisition of Internal Representation by Multilayer Perceptron" (IEICE Transactions Vol. J73-
D-II, No. 8, P1173-8, August 1990)
Can be referred to.

Learned neural network NNA1
4 to the third to third layers (network A)
The extracted features 1 to 4) are the feature extraction network 12a (step # 3). In the feature extraction network 12a, the input layers S of the networks A1 to A4 are connected in parallel, and 64 (= 8 × 8) pieces of image data are simultaneously input to the networks A1 to A4. 40 (= 10 × 4) from the output layer in the networks A1 to A4
Data is output.

As shown in FIG. 6, the region discriminating network 12b has three layers, and the numbers of neurons in the input layer S, the intermediate layer A, and the output layer R are 40, 40, and 4, respectively. The response function of the input layer S is a linear function, and the response functions of the intermediate layer A and the output layer R are sigmoid functions. The output layer of the feature extraction network 12a is connected to the input layer S of the area discrimination network 12b (step # 4).

To the neural network 12 configured as described above, an input signal having the same distributive feature as the one learned by the neural network NNA is input to change the coupling coefficient of the feature extracting network 12a. Instead, the area discrimination network 12b is learned (step # 5).

That is, first, a learning character image is input to the neural network 12, and learning is performed so that the output of the neuron r1 of the output layer R of the area discrimination network 12b becomes "1". Next, a learning photo flat image is input, and learning is performed so that the output of the neuron r2 of the area discrimination network 12b becomes "1". Further, the photograph edge image and the halftone dot image for learning are sequentially input, and the neuron r3 of the area discrimination network 12b is input.
4 is learned so that the output of 4 becomes "1". In this learning process, the coupling coefficient of the area discrimination network 12b changes. The neural network N
The learning of NA is based on the back propagation method which is a well-known technique. The sample used for learning may be the same as or different from the sample used for learning the neural network NNA. Character image,
Learning is performed so that the mean square error becomes small for each of the photograph flat image, the photograph edge image, and the halftone dot image.

In this way, the neural network 1
2 is created. Created neural network 1
When the image data DMa is input to 2, the four neurons r1 to r4 of the output layer R obtain outputs S1 to 4 corresponding to the character area, the photograph flat area, the photograph edge area, and the halftone dot area, respectively. To be That is, the neural network 12 determines the attribute AT of the block area BA based on the data input to the input layer S of the networks A1 to 4, and when it is the character area, the output S1 of the neuron r1 is "1". , The output S2 of the neuron r2 is close to “1” in the case of a photograph flat area, and the output S3 of the neuron r3 is close to “1” in the case of a photograph edge area. , The output S4 of the neuron r4 is close to “1”.

Output S from the neural network 12
Based on 1 to 4, the attribute AT of the block area BA is determined. For example, when one output is "1", it is determined to be the area corresponding to the output. Or
When there is one output that exceeds a certain threshold, it is determined to be a region corresponding to that output. Alternatively, the area corresponding to the largest output is determined as the area.

Further, based on the attribute AT of each block area BA determined in this way, smoothing by morphology or the like is performed, thereby correcting the determination result for each block area BA and enlarging each area. Reduce misjudgment. As a result, the original image PM is divided into four areas, that is, a character area, a photograph flat area, a photograph edge area, and a halftone dot area.

For example, a high-pass filter process and a binarization process are performed on the character region, and a process for obtaining a natural gradation property or a specific gradation is performed on the photograph flat area and the photograph edge image. A process of emphasizing is performed, and a smoothing process is performed on the halftone dot area to prevent moire.

A photographic image is, like a silver salt photograph,
This is a grayscale image having a sufficiently large pixel density with respect to the reading density of the original image PM, and a photographic flat image is a portion where the density change is small, and a photographic edge image is a portion where the density change is large. The photographic flat image is distinguished from the white background portion of the character image. For example, the whitening process is performed on the white background portion of the character image, while the process utilizing the gradation is performed on the photographic flat image. Further, the difference between the dot image and the photographic image decreases as the dot becomes finer. For example, when the reading density of the original image PM is 400 dpi, the density of halftone dots is 200 lines /
In inches, the read image data DM is not different from that of a photographic image. Therefore, in that case, 20
A halftone image of 0 lines / inch or more may be included in the photographic image.

According to the above embodiment, 64 extracted from the image data DM to the neural network 12 is used.
By directly inputting each piece of raw image data DMa, the attribute AT of the block area BA can be determined. That is, it is not necessary to give a feature amount as an input signal to the neural network 12, and the input signal to be discriminated can be input as it is, so that the input to the neural network 12 is easy.

Therefore, it is not necessary to previously extract a characteristic amount having a physical meaning as in the prior art, and a circuit or program therefor is not required, and circuit configuration, processing speed, flexibility, cost, etc. Is advantageous in that.

According to the above-described embodiment, since the neural network 12 is used to discriminate the attribute AT, the learning effect of the neural network 12 allows the attribute A to be easily calculated.
The determination of T is performed, and the reliable determination of the attribute AT is performed.

After the neural network 12 is trained, a process for discrimination is performed by a product-sum operation of input data and a coupling coefficient obtained by learning, and a search of a table showing a response function. Therefore, the processing speed of the calculation can be improved. In particular, since the coupling coefficients of the networks A1 to A4 do not change, the feature extracting network 12a can be easily configured by diverting only the coupling coefficient obtained by learning of the neural network NNA, and the arithmetic processing can be performed. The calculation speed is high because it can be simplified.

Therefore, in the copying machine using the attribute discriminating apparatus 1, it is possible to accurately divide the area of the original PP.
The image data DM obtained from the document PP can be subjected to appropriate processing in real time according to the area, and a clear copy image can be output.

In the above embodiment, the square area of 8 × 8 pixels is set as the block area BA, but 4 × 4 pixels,
Areas of various sizes such as 3 × 3 pixels, 16 × 16 pixels, and 64 × 64 pixels may be used as the block area BA. It does not have to be a square. Further, although the block areas are allocated so as not to overlap with the original image PM, they may be sequentially shifted and allocated so as to overlap with each other.

That is, as shown in FIG. 8A, with respect to the original image PM, for one pixel PXa whose attribute is to be discriminated, a block area BAa for 8 × 8 pixels around the pixel PXa is provided.
Image data DMa corresponding to the pixel PXa
And the block area BAa can be sequentially shifted by one pixel.

Further, as shown in FIG. 8B, with respect to 4 (= 2 × 2) pixels PXb whose attributes are to be discriminated, image data of a block area BAb for 8 × 8 pixels around the pixel PXb Can be extracted as the image data DMa corresponding to the pixel PXb, and the block area BAb can be shifted by two pixels.

In the above-described embodiment, four types of attribute determinations, that is, a character region, a photo flat region, a photo edge region, and a halftone dot region are performed, but it is configured to perform three or less types or five or more types of attribute determinations. You may. For example, the character area is divided into two areas, that is, a character image area on a black background in which the character itself is drawn and a character background area on a white background around the character, and attribute determination of a total of five types of areas is performed. You may

In the above embodiment, the block area extraction unit 11 is realized by software by a memory storing programs and data and a CPU executing the programs. The neural network 12 is realized by a computer simulator.
Therefore, as described above, the neural network 12 can be realized by a table showing the learned coupling coefficient and response function, and a program for calculating and searching them. Such an aspect is also included in the neural network of the present invention. Further, the neural network may be directly realized by hardware.

In the above embodiment, the neural network NNA having five layers is used, but it may have four layers or less or six layers or more. Four networks A1
Although 4 was used, 3 or less or 5 or more may be used. Although a three-layer network is used as the area discrimination network 12b,
It may have four, five, or more layers. Although one area discriminating network 12b is connected to all the networks A1 to A4, separate area discriminating networks may be connected to the networks A1 to A4. The number of neurons in each layer, the presence / absence of coupling coefficient, the type of response function, the learning method, and the like can be variously changed other than the above. In addition, the configuration, processing content, processing order, etc. of each part or the whole of the attribute determination device 1 can be appropriately changed in accordance with the gist of the present invention.

The present invention can be applied to discrimination of various attributes AT other than the area. Further, it is applicable to various information processing such as other image recognition, voice recognition, voice synthesis, and automatic translation. Even in such a case, since the extraction of the feature amount itself is performed by the neural network 12,
It is not necessary to consider what should be the characteristic amount, and it is not necessary to create a circuit or program for extracting the characteristic amount. Moreover, since the feature extraction network 12a is created by learning the neural network NNA, the creation of the neural network 12 is easy.

[0049]

According to the present invention, it is possible to obtain a neural network capable of directly inputting an input signal to be discriminated without having to give a feature quantity as an input signal.

After learning the neural network, information processing can be performed by multiplying and summing the input data and the coupling coefficient obtained by learning, and searching the table showing the response function. Therefore, the processing speed of the calculation can be improved.

Further, as the coupling coefficient of the feature extracting network, the coupling coefficient of the learned first neural network can be diverted as it is, so that the feature extracting network can be easily constructed, and Since the calculation process can be simplified, the calculation speed can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an attribute discrimination device having a neural network according to the present invention.

FIG. 2 is a diagram illustrating a block area extracted from a document.

FIG. 3 is a diagram showing a configuration of a neural network.

FIG. 4 is a diagram showing a process of creating a feature extraction network.

FIG. 5 is a diagram showing an example of an hourglass-type neural network.

FIG. 6 is a diagram showing an example of an area discrimination network.

FIG. 7 is a flowchart showing a method for creating a neural network.

FIG. 8 is a diagram for explaining another example of a block area allocation method.

[Explanation of symbols]

NNA Neural network (first neural network) 12b Region discrimination network (second neural network) 12a Feature extraction network 12 Neural network

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Tetsuya Ito 2-3-13 Azuchi-cho, Chuo-ku, Osaka-shi, Osaka, Osaka International Building Minolta Co., Ltd. (72) Shin Shin Fu, Azuchi, Chuo-ku, Osaka-shi, Osaka 2-13-3 Machi Osaka International Building Minolta Co., Ltd.

Claims (2)

[Claims] 1. The number of neurons in at least one intermediate layer is smaller than the number of neurons in the input layer, and the number of neurons in the input layer is the same as the number of neurons in the output layer. From the feature extraction network configured from the input layer to the intermediate layer of the first neural network that has been learned so that the output values from the corresponding output layers are equal, A second neural network connected so that the output of is input, and a neural network comprising: 2. A first neural network in which the number of neurons in at least one hidden layer is smaller than the number of neurons in the input layer and the number of neurons in the input layer and the number of neurons in the output layer are the same, and a certain distribution characteristic is provided. The input signal is input to the input layer, and learning is performed so that the input value to each input layer and the output value from each output layer corresponding to each input layer become equal, and the learned first neural The input layer to the intermediate layer of the network are used as the feature extraction network, and the output from the feature extraction network is combined so as to be input to the input layer of the second neural network. An input signal having the same distributive feature as that of the input signal is input to the input layer of the input layer without changing the coupling coefficient of the feature extraction network. Train the second neural network, how to create a neural network, characterized in that.