JP2900427B2 - Neural net - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Overview] Regarding a configuration of a neural network for processing pattern recognition and the like, a multidimensional input signal space is obtained by calculating required weights for setting a neural network to a required state. The purpose of the present invention is to provide a neural network capable of efficiently performing the categorization of a required area having one or a plurality of cells, each of which is provided corresponding to each required island area in an n-dimensional input signal space. An input layer, an intermediate layer, and an output layer, the input layer having n input terminals, each of the input terminals corresponding to each input signal, and the intermediate layer having 2n intermediate layers. A node, wherein every two intermediate nodes are connected to each of the input terminals, each intermediate node comprising:
A value obtained by multiplying an input signal from the input terminal to be connected by each set weight and each set bias are input to a predetermined function, the function value is output, and the output layer outputs a required number of outputs. Each output node connects to all of the intermediate nodes, and each of the output nodes has a value obtained by multiplying the output of each of the connected intermediate nodes by each of the set weights and each of the set biases. As an input of a predetermined function, and outputs the function value as an output signal, and each cell outputs the required output signal for the island area corresponding to the cell, so that each of the weights and the Each bias is set, and the output signals of all the cells are combined to produce a predetermined sum signal such that the sum signal is the input signal in the area that combines the corresponding island areas of all the cells. It is configured to have a required output value for the signal.

[Industrial applications]

The present invention relates to a configuration of a neural network for processing pattern recognition and the like.

[Conventional technology]

As is well known, a neural network has, for example, four inputs, that is, a four-dimensional input signal space, as illustrated in FIG.
In the case of the neural network, the input layer, the intermediate layer, and the output layer, the output of all nodes of the previous layer is output to all nodes of the next layer by multiplying a certain weight determined for each node input, The value of a predetermined function that receives the sum of those inputs as an input is configured as the output of the node. The intermediate layer may have two or more layers, and the number of nodes in the output layer may be two or more depending on the number of required output types.

For this function, for example, a sigmoid function represented by f (x) = 1 / (1 + e− ^x ) is used, where x is the input sum and e is the base of natural logarithm. The fourth node in the input layer and the I-th node in the intermediate layer are nodes that output a constant of 1.0. A bias obtained by multiplying this output by a weight in the same manner as the other nodes is used as one input of the next layer. , And apply the necessary bias to each node input.

When performing pattern recognition of an image or the like by a neural network, for example, a signal of a feature amount extracted from a character image is input to an input terminal of an input layer of the neural network, and the recognition result is obtained at an output terminal of an output layer. To do.

As a feature amount to be input in the case of recognizing an image pattern such as a character, for example, an amount related to the inclination of a line constituting the character image is used as is known. That is, as illustrated in FIG. 6 (a), for example, the character image 25 is scanned by moving the slit 22a or the like having an appropriate length and width, and the ratio of the area occupied by the black line of the character entering the slit frame is determined. For example, there is provided a detector which outputs a value of 0 when the color is all white and 1.0 when the color is all black.

For example, the inclination with respect to a certain reference line is 0 ° (vertical line), −45
° (upward diagonal), + 45 ° (upward diagonal), and 90
The scanning is performed by the slits 22a to 22d defined by ° (horizontal line), and the maximum value of each detected output is taken, thereby obtaining the characteristic amount of the four-element character.

Each element of the feature amount indicates an amount of a line having a slope close to each slit and having the largest amount indicating both the closeness of the slope and the length extending in the direction. When further detailed identification is required, for example, as shown in FIG. 6 (b), the pattern area can be divided into four parts, and the above-described feature amount can be obtained for each area, and can be refined as a total of 16 element feature amounts. .

A set of real values in the range of 0 to 1.0 representing such a feature amount is input to each node of the input layer of the neural network, and for example, when a feature amount of the pattern “A” is input, it is recognized as A. For example, learning is performed so as to output 1.0 as an output indicating that, and to output 0 in other cases.

As is known, neural network learning is performed by inputting an input signal such as a feature value or the like and a teacher signal indicating a desired output signal from the input to the neural network. By a predetermined algorithm such as the back propagation method, the change is made based on the difference between the actual output signal and the teacher signal, and this operation is repeated to gradually reduce the difference between the two signals and converge to a difference within a predetermined value. By doing so, for example, one of the categories "A"
Learn about patterns.

However, when it is desired to recognize a certain character, learning is performed so as to obtain a recognition output by sequentially inputting the feature amounts of a large number of patterns distorted from the pattern to a required range together with the standard pattern. Therefore, in order to increase the recognition rate, it is necessary to collect and learn the above feature amounts for various types of distortion patterns of the category to be recognized. A learning processing method that can reduce the required number of inputs is described in Japanese Patent Application No.
01-46927 "Neural network learning processing method".

According to this method, learning to set the output to 1.0 for an input of a standard signal corresponding to a certain point on the input signal space and learning to set the output to 0 for an input signal around each standard signal are performed. Because of the learning characteristic that the neural network can interpolate and extrapolate the function value, the output of the standard signal is 1.0 for all the inputs in the small island space surrounded by the surroundings.
The output value that gradually decreases from 0 to 0 in the periphery is interpolated, and 0 is extrapolated for the output outside the island area. Fig. 4 shows 1
FIG. 7 is a diagram conceptually illustrating a relationship between x and an output signal y in a case where a standard signal value of an input signal x is 0.8 and a section from 0.7 to 0.9 is an island, taking a dimension space as an example. Therefore,
By filling the required partial space with islands formed by the required number of standard signals set at appropriate intervals, it is not necessary to fill the partial space with points, and the amount of learning processing can be reduced.

[Problems to be solved by the invention]

However, as described above, when the number of elements of the input feature amount is increased in order to refine the feature amount, the number of nodes in each layer of the neural network increases, and the size of each node increases accordingly. In some cases, it is necessary to increase the number of intermediate layers. At the same time, it is known that the convergence time of the learning process rapidly increases, and the number of cases where the learning does not converge due to initial conditions or the like also increases.

In addition, in the above-described conventional neural network, if data to be learned is to be added after the learning is once completed, generally, all learning including the previous data must be performed again.

It is an object of the present invention to provide a neural network capable of efficiently categorizing a required region of a multidimensional input signal space by calculating a required weight for setting the neural network to a required state.

[Means for solving the problem]

 FIG. 1 is a block diagram showing the configuration of the present invention.

FIG. 1 shows a configuration of a neural network, which has one or a plurality of cells 10, each cell 10 being provided corresponding to each required island region of an n-dimensional input signal space, an input layer 11 and an intermediate layer. 12
And an output layer 13. The input layer 11 has n input terminals 14. Each input terminal 14 corresponds to each input signal 8, and the intermediate layer 12
Has 2n intermediate nodes 15, each two intermediate nodes
15 is connected to each input terminal 14, and each intermediate node 15 calculates a value obtained by multiplying the input signal from the input terminal 14 to be connected by each set weight and each set bias by a predetermined function. Outputting the function value as an input, the output layer 13 has a required number of output nodes 16, each output node 16 is connected to all intermediate nodes 15, and each output node 16 is connected to each intermediate node A value obtained by multiplying the output of 15 by each set weight and each set bias are input to a predetermined function, the function value is output as an output signal 9, and each cell 10 corresponds to the cell 10. The weights and the biases are set so as to output a required output signal 9 for the island region, and the output signals 9 of all the cells 10 are combined by the combining unit 17 so as to generate a predetermined sum signal. And the sum signal combines the corresponding island regions of all the cells. It is configured so as to have a required output value with respect to an input signal within the region.

[Action]

When each cell 10 is made to correspond to a required one island region in the input signal space by the neural network configured as described above, for example, a center portion is defined with respect to a set of input signals defining the island region. The values of each weight and bias can be calculated to be close to 1.0 for the input of, and to be close to 0 for the peripheral input, as described below.
Thus, for each of the required one or more island regions, cell 10
Are provided, and the outputs thereof are combined by the combining unit 17 and added, whereby a neural network corresponding to the input signal of the required subspace can be configured.

〔Example〕

FIG. 2 is a diagram for explaining the principle of the present invention. As shown in FIG. 2 (a), a one-dimensional space is provided with an island region centered at x = a, and a neural output is obtained as indicated by y. Considering the net, for example, a result similar to that of FIG. 4 where a = 0.8 and both ends of 0.7 and 0.9 can be realized by a weighted neural network exemplified in FIG. 2 (b). In the figure, the values attached to the lines connecting the nodes are the respective weights, and the node written as 1.0 is a node for generating a bias as described above, and is a node for the intermediate node and the output node. The function is a sigmoid function.

Therefore, when the outputs of the intermediate nodes are z ₁ and z ₂ , in the illustrated neural network, z ₁ = 1 / (1 + exp (−80x + 60)) z ₂ = 1 / (1 + exp (80x−68)) y = 1 / (1 + exp (−8z ₁ −8z ₂ +12)). By substituting the value of x into this equation, it can be confirmed that this neural network satisfies the required condition.

Next, considering the case of a two-dimensional input signal space, assuming that the shape of the island region is a rectangle having sides parallel to the coordinate axes with the point (a ₁ , a ₂ ) as the center as shown in FIG. It can be seen that a neural network having the configuration shown in (d) can be realized by independently considering each axis, applying a one-dimensional net to each of them, and connecting both at the output layer.

When a plurality of such island regions are provided as shown in FIG. 2E, as shown in FIG. 2F, the sum of the outputs of the network shown in FIG. What is necessary is to combine so that it may obtain.

The above configuration can be generally extended to an n-dimensional input signal space. If the space is provided with an n-dimensional hypercube island region whose plane is perpendicular to the coordinate axes, the one-dimensional net shown in FIG. A neural network in which n sets are arranged and connected at the output layer, that is, n inputs have 2n intermediate nodes, and each input is any 2
Connected to the intermediate nodes, and each intermediate node is obtained by a neural network connected to only one input, and the cells of such a neural network are formed by the required number of island regions and added and connected. It is possible to cover a partial space composed of an island region.

With such a configuration, the necessary weights and biases in the neural network can be calculated relatively easily as described below.

FIG. 3 shows the configuration of a cell in the case of a four-dimensional input signal space, where w _i is a weight, θ _i is a bias, and u _i is an output of an intermediate node. In this neural network cell, when the input signal (x ₁ , x ₂ , x ₃ , x ₄ ) is an arbitrary value (a ₁ , a ₂ , a ₃ , a ₄ ), the output of the output node is 1 , And as the input signal moves away from that point, the output gradually decreases,
As far as a certain periphery, it should be as close to 0 as possible.

The input and output conditions for this are as follows: when the input is (a ₁ , a ₂ , a ₃ , a ₄ ), the output is 0.99, and when any one of a _{1 to} a ₄ is apart by Δ, the output is 0.01 In this case, there are an infinite number of sets of weights necessary for this, and one set can be derived as follows.

First, the output u _2i-1 of the intermediate node to which x _i is connected is, for example, 0.9 at input a _i , 0.5 at a _i −Δ / 2, 0.1 at a _i −Δ, and u _2i is a _i
In 0.9, a _i + Δ / 2 0.5, so that 0.1 in a _i + delta, in order to determine the weights and biases of the intermediate node, the value of the input of the sigmoid function linear equation y _2i-1 = k ( x _i − (a _i −Δ / 2)), y _2i-1 is y 0 because x _i is 0 at a _i −Δ / 2
The value of the sigmoid function with _2i-1 as input is 0.5, and a _i
−Δ and a _i are −kΔ / 2 and kΔ / 2, respectively, and only the sign is different.
, On the other hand, it is clear that 1-0.1 = 0.9.

Therefore, k = 4.4 / Δ is obtained from y _2i−1 = −kΔ / 2 = −2.2 by obtaining y _2i−1 that satisfies 1 / (1 + exp (−y _2i−1 )) = 0.1. Therefore, the weight of the input of the intermediate node 2i-1 is 1
Rewrite the following equation, against the FIG. _{3, y 2i-1 = kx i} -k (a i -Δ / 2)) = w 2i-1 from _{x i + θ 2i-1,} w 2i-1 = k, θ _2i−1 = −k (a _i −Δ / 2)).

Similarly, for the intermediate node 2i, k = 4.4 / Δ is obtained by considering the linear expression y _2i = −k (x _i − (a _i + Δ / 2)).

For example, if (a ₁ , a ₂ , a ₃ , a ₄ ) = (0.5,0.8,0.2,0.3) and Δ = 0.1, then y ₁ = 4.4 / 0.1 (x _1- (0.5-0.1 / 2) ) = 44x ₁ -19.8 y ₂ = -4.4 / 0.1 (x _1- (0.5 + 0.1 / 2)) =-44x ₁ +24.2 y ₃ = 4.4 / 0.1 (x _2- (0.8-0.1 / 2)) = 44x ₂ -33.0 y ₄ = -4.4 / 0.1 (x _2- (0.8 + 0.1 / 2)) =-44x ₂ +37.4 y ₅ = 4.4 / 0.1 (x _3- (0.2-0.1 / 2)) = 44x _{3 -6.6 y 6 = -4.4 / 0.1 (} x 3 - (0.2 + 0.1 / 2)) = - 44x 3 +11.0 y 7 = 4.4 / 0.1 (x 4 - (0.3-0.1 / 2)) = 44x 4 -11.0 _{y 8 = -4.4 / 0.1 (x} 4 - (0.3 + 0.1 / 2)) = - becomes 44x ₄ +15.4, weights and bias of the intermediate layer is determined all.

Next, the weight and bias of the output layer are obtained. From the previous results, at the center point (0.5, 0.8, 0.2, 0.3), y ₁ = y ₂ = y ₃ = y ₄ = y ₅ = y ₆ = y ₇ = y ₈ = 2.2 and a point shifted by 0.1 in either axis direction, for example (0.
6,0.8,0.2,0.3) are related to displaced elements, such as y ₁ = 6.6, y ₂ = -2.2, y ₃ = y ₄ = y ₅ = y ₆ = y ₇ = y ₈ = 2.2 Values of 6.6 and -2.2
And the other six are 2.2.

That is, since the output of the intermediate layer has a symmetrical change in the above point shift, the weights w _{9 to} w _{16 of the} output layer
Are the same value w, and the input values for setting the value of the sigmoid function of the output node to 0.99 and 0.01 are 4.
6 and −4.6, the following equation.

(1 / (1 + exp (-2.2))) × 8w + θ ₉ = 0.9 × 8w + θ
₉ = 4.6 (1 / (1 + exp (−2.2)) × 6 + (1 / (1 + exp (−6.
6)) + (1 / (1 + exp (2.2))) w + θ ₉ = (0.9 × 6
+ 0.999 + 0.1) w + θ ₉ = −4.6 As a value satisfying, the weight w and the bias θ ₉ are obtained, and w = 13.1 and θ ₉ = −89.8 are obtained. FIG. 3 shows the above results.

Note that these values are different values by changing the output of the intermediate layer (0.1 and 0.9 in the above example) to different values, etc.
The input and output conditions of the same neural network can be satisfied.

As described above, it is apparent that the calculation of the weight and the bias can be similarly performed not only in the case of the four-dimensional input signal space but also in a space of an arbitrary dimension. Then, the cells can be set in a necessary state so as to correspond to one island area without performing learning by iterative processing, and the outputs of the required number of cells set in this way are added. Thus, a neural network that satisfies input / output conditions for a necessary subspace can be constructed.

By using this neural network and setting, for example, the above-described character image feature amount as input and outputting “1” for a specific character type, a neural network that recognizes one character can be created. By providing the neural network for each required character type, a character recognition device can be configured relatively easily.

〔The invention's effect〕

As is apparent from the above description, according to the present invention, in a configuration of a neural network for processing pattern recognition and the like, a required weight and a bias for setting the neural network to a required state are obtained by calculation. Since a required area of the multidimensional input signal space can be efficiently categorized, there is a remarkable industrial effect that the use of the neural network is facilitated.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of the present invention, FIG. 2 is an explanatory diagram of a neural network, FIG. 3 is an explanatory diagram of weight calculation, FIG. 4 is an explanatory diagram of an island region, and FIG. FIG. 6 is a diagram showing an example, and FIG. 6 is an explanatory diagram of a feature amount. In the figure, 8 is an input signal, 9 is an output signal, 10 is a cell, 11 is an input layer,
12 is an intermediate layer, 13 is an output layer, 14 is an input terminal, 15 is an intermediate node, 16 is an output node, 17 is a coupling section, 22a to 22d are slits, and 25 is a character image.

────────────────────────────────────────────────── ─── Continued on the front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G06F 15/18 G01T 1/40 JICST file (JOIS)

Claims (1)

(57) [Claims] 1. An apparatus comprising one or a plurality of cells (10), each cell (10) being provided corresponding to each required island region of an n-dimensional input signal space, and having an input layer (11) and an input layer (11). , Middle layer (1
2) and an output layer (13), wherein the input layer (11) has n input terminals (14), each input terminal corresponding to each input signal (8), Layer (12) has 2n intermediate nodes (15), each 2
Each of the intermediate nodes is connected to each of the input terminals (11), and each of the intermediate nodes (15) is a value obtained by multiplying an input signal from the connected input terminal (11) by a set weight. And the set biases are input to a predetermined function to output the function value. The output layer (13) has a required number of output nodes (16), and each of the output nodes is Each output node (16) is connected to an intermediate node (15), and each of the connected intermediate nodes (1
A value obtained by multiplying the output of 5) by each set weight and each set bias are input as a predetermined function, and the function value is output as an output signal (9). Each of the cells (10) The weights and the biases are set so as to output the required output signal (9) for the island region corresponding to the cell, and the output signals (9) of all the cells are set to a predetermined value. Combined to generate a sum signal (17), the sum signal being configured to be a desired output value for an input signal in a region combining the corresponding island regions of all the cells. A neural net characterized by the following.