JPH0728768A - Method for obtaining inverse solution by learning of neural network - Google Patents

Abstract

PURPOSE:To provide the method which obtains the inverse solution by learning of a neural network without new development of a system of an inverse model or the like for acquisition of the inverse solution. CONSTITUTION:An auxiliary input layer 111 is provided in the preceding stage of an input layer of a neural network 110, and the input layer 112 and the auxiliary input layer 111 are fixedly coupled, and the neural network 110 is learnt by a learning pattern 120; and after the end of learning, fixed coupling between the input layer 112 and the auxiliary input layer 111 is released, and layers from the input layer 112 to an output layer 114 of the neural network 110 are fixedly coupled, and learning is performed with such learning patterns 120 and 160 that an output pattern 150 has a desired value.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for obtaining an inverse solution by learning a neural network, and in particular, when a certain "evaluation" can be calculated from a "combination of certain components", a "certain evaluation value" is given in reverse. The present invention relates to a method of obtaining an inverse solution by learning a neural network to obtain desired information by using a neural network when the information of “combination of components” is desired.

More specifically, as an example of a field in which it is desired to search for "mixing ratio of materials" from "property of material", the component (wt /%) of glass (soda lime silicic acid) has the following mixing ratio. Then, SiO ₄ … 73, Na ₂ O… 14, CaO… 9, MgO… 3.5, Al ₂ O ₃ … 0.5 When the above components are obtained by learning, the type of glass to be applied is calculated from this component mixture ratio. Requires soda lime silicic acid. In this example, the determined soda lime silicic acid is the inverse solution.

Besides the above examples of glass, it can be applied to metals, resin processing, chemical compounding, food processing, fibers and the like.

In general, the method for obtaining an inverse solution may be applied to a field in which a "determination result" or conversely a "determination factor" is obtained.

[0005]

2. Description of the Related Art As a conventional inverse model learning method, a direct inverse modeling method is applied to learning of motion control by an albus or the like.

In this method, the input of the target system is x,
If the output is y = f (x), and the input is y,
For the input y, the inverse model is learned so that the output becomes x.

In this method, taking the glass component as an example,
Table 2 which is a pattern in which the input and the output of Table 1 are exchanged is learned in the inverted triangular neural network shown in FIG. Table 1 shows each component of glass SiO2 ... Al2O3
Table 2 shows the case where the input is and the output is the refractive index.
The refractive index is input and each component is output.

[0008]

[Table 1]

[0009]

[Table 2]

In this example, different components are obtained as inverse solutions by using the refractive index as an input.

As for direct inverse modeling, there is a method called an output feedback type inverse model in which a target system is linearized and a feedback circuit is added.

According to this method, the linear system Y = Ax performs learning of an inverse model by orthogonal learning. 'When is updated, W _i' W _i representing the inverse model W _i 'in the _{limit, W i' = W i +} ηΣ p (x p -W i y p) y p = W i, also y _p = Ax _p , A′−W _i AA ′ = 0 holds. In the case of n <m (n-dimensional vector <m-dimensional vector), if A is regular, then W _i = A ′ (AA ′) ⁻¹ . In the case of n = m (n-dimensional vector = m-dimensional vector), if A is regular, then W _i = A ⁻¹ . In either case, if the output W _i y of the inverse model is input to the target system, the output of the target system becomes y.

As described above, when the target system is a linear system, the inverse model can be learned by the direct inverse modeling even if there is a redundant degree of freedom. Therefore, by linearizing the target system, a kind of inverse model is obtained by the direct inverse modeling method.

[0014]

However, among the above-mentioned conventional methods, in the learning method of the inverse model by Albus et al., The patterns p1 and p2 shown in Table 2 have the same refractive index 1.
When 458 is input, a component different from the data listed in Table 2 is output, causing a contradiction, or approaching an appropriate intermediate value between the two values, and learning does not proceed, or
There is a problem such that the two values do not converge.

In the direct modeling method, the correspondence between x and y is 1.
In the case of pair 1, there is no problem, but when there are a plurality of inputs x corresponding to one output y, the ideal inverse model learned by the direct inverse modeling method is x corresponding to y.
The average value x _{m of} is output. However, for a non-linear system, y
_m = f (x _m ) does not always match y. When y _m does not match y, the output of the target system does not become y even if the output x _{m of the} inverse model is input to the target system. This method limits the system that can learn the perfect inverse model.

Further, the above-mentioned problems relating to the direct modeling method are solved by an output feedback type inverse model method in which the target system is linearized and a feedback circuit is added, but this method is applied only in the case of a linear system. However, there is a problem that it cannot be applied to nonlinear systems. Also, in the case of a linear system that does not use a sigmoid function, the inverse matrix can be calculated, but there is a problem in practicality because the network performance is limited. As described above, in the method of learning the inverse mapping in which the data of the evaluation result of the mixture ratio is replaced by using the neural network, the learning patterns are inconsistent with each other, and the learning cannot be performed successfully. Further, in order to realize the inverse model, it is necessary to develop a neural network for inverse solution and an inverse solution algorithm, which causes a problem that a system designer is burdened.

The present invention has been made in view of the above points, and solves the above-mentioned problems of the prior art, without developing a new system such as an inverse model for obtaining an inverse solution, and If there is an algorithm to determine the unknown coefficient in
It is an object of the present invention to provide a method for obtaining an inverse solution by learning a neural network that can obtain an inverse solution using the arithmetic method.

[0018]

A method for obtaining an inverse solution by learning a neural network according to the present invention comprises a neural network composed of an input layer, an intermediate layer and an output layer and a learning pattern. An auxiliary input layer having the same number of neuron units as the input layer is provided in the preceding stage, the coupling coefficient between the input layer and the auxiliary input layer is fixedly coupled by an arbitrary coefficient, and the neural network is trained by the learning pattern, and after the learning is completed. , The fixed coupling of the coupling coefficient between the input layer and the auxiliary input layer is released, the coupling coefficient from the input layer to the output layer of the neural network is fixedly coupled, and a predetermined learning pattern is used so that the output pattern has a desired value. Learn the neural network.

[0019]

According to the present invention, in a neural network consisting of an input layer, an intermediate layer and an output layer, an auxiliary input layer is provided in front of the input layer, and in the learning of the neural network, the coupling coefficient between the auxiliary input layer and the input layer is fixed. By performing the first learning, the auxiliary input layer and the input layer are not learned, but the input layer, the intermediate layer, and the output layer are learned. Next, the coupling coefficient between the auxiliary input layer and the input layer is set as free coupling, and the coupling weight coefficient from the input layer to the output layer is set as fixed coupling. Since only the connecting part between the input part and the input part learns, when the learning converges, the inverse solution is distributed and expressed in the weights between the auxiliary input layer and the input layer. As a result, according to the present invention, the causal relationship between the patterns is not exchanged as in the conventional method, so that no contradiction occurs due to the exchange. Further, in learning of the neural network, since the learning algorithm is used as it is without using the algorithm for the inverse solution, another system development is not required.

[0020]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a diagram for explaining the outline of the system of the present invention.

FIG. 3A shows the case where the first learning is performed in the neural network, and FIG.
The case of performing the second learning is shown.

First, the structure of the neural network 110 will be described. Neural network 110
Is a general input layer 112, intermediate layer 113 and output layer 1
In addition to the structure of 14, the auxiliary input layer 111 is provided before the input layer 112. Auxiliary input layer 1
The number of 11 neuron units is the same as the number of neuron units in the input layer 112, and in the case of this embodiment, 5
It is an individual.

In FIG. 3A, the coupling loads of the auxiliary input layer 111 and the input layer 112 are fixedly coupled by a predetermined coupling coefficient, and the input layer 112, the intermediate layer 113 and the output layer 114 are arbitrarily coupled. Freely coupled with a coefficient. When the neural network 110 performs learning in this state,
Only the value input from the auxiliary input layer 111 is transmitted to the input layer 112, and learning is performed in the layers above the input layer 112.

In FIG. 2B, after the learning of FIG. 3A, the coupling weights of the auxiliary input layer 111 and the input layer 112 are freely coupled by an arbitrary coupling coefficient.
2. The intermediate layer 113 and the output layer 114 are fixedly coupled by the coupling coefficient after learning. When the neural network 110 performs learning in this state, learning is performed only between the auxiliary input layer 111 and the input layer 112. Therefore, when learning converges, an inverse solution appears between the auxiliary input layer 111 and the input layer 112.

FIG. 2 is a flow chart showing the outline of the operation of one embodiment of the present invention.

A neural network and a learning pattern are prepared in advance as a premise of the operation described below.

Step 1) Connect the input layer 112 and the auxiliary input layer 111 of the neural network.

Step 2) Input layer 112 and auxiliary input layer 1
The coupling weight coefficient of 11 is fixed as 1.0, for example.

Step 3) Neural network 11
0 is learned by a learning pattern.

Step 4) Neural network 11
When the learning of 0 is completed, the fixed coupling between the input layer 112 and the auxiliary input layer 111 fixed in step 2 is released.

Step 5) The coupling weight coefficients of the input layer 112 to the output layer 114 which have been learned for the first time are fixed.

Step 6) The input pattern is set to 1.0 or the like, and the output pattern is set to the "desired output value (teacher pattern)".
Setting a learning pattern different from the pattern used in step 3.

Step 7) According to the learning pattern set in Step 6, the neural network 110 whose coupling weight coefficient is changed in Step 6 is trained.

Step 8) After the learning is completed, the inverse solution of the desired output value is obtained as the coupling weight coefficient of the input layer 112 and the auxiliary input layer 111 learned in step 7.

Step 9) On the other hand, if another inverse solution is desired, the process proceeds to step 10, and if a desired inverse solution is obtained, the process ends.

Step 10) The coupling weight coefficient of the input layer 112 and the auxiliary input layer 111 of the neural network is changed to another value for fixed coupling, and the learning coefficient is changed to shift to step 9.

FIG. 3 shows a neural network according to an embodiment of the present invention when the coupling weight coefficient changes.

FIG. 7A shows a state in which the coupling weight coefficient of the input layer 112 and the auxiliary input layer 111 is fixed to 10.0, and the neuron data of the auxiliary input layer 111 is kept as it is. Will be input to the neuron unit. At this time, from the input layer 112 to the output layer 1
The coupling load factor of 14 is a free coupling load. In the figure, the solid line between the neurons of the input layer 112 and the auxiliary input layer 111 is the connection weight, and in the example of FIG. 7A, the connection weight is 10.0 and is indicated by the dotted line. Shown is the unbonded load, which has a combined load of zero.
The neural network in this state is learned by a predetermined learning pattern. By making the neural network learn in this state, the input layer 112 to the output layer 114
Will learn.

In the figure (B), the learning of the input layer 112 to the output layer 114 is completed in the state of the figure (A), and the input layer 112 is shown.
Then, the fixed coupling weight coefficient of the auxiliary input layer 111 is released and free coupling is performed, and the weights of the input layer 112 to the output layer 114 are fixed. By learning with a desired pattern in the state of this neural network, learning is performed between the auxiliary input layer 111 and the input layer 112, and an inverse solution is obtained here.

An example in which the present invention is applied to glass data will be shown below.

In the following explanation of the operation, f (x) represents a sigmoid function,

[0043]

[Equation 1]

F '(x) represents the inverse function of the sigmoid. x = f '(y) =-log (1 / y-1) (1) First, it is assumed that there is elementary data on the glass components and the refractive index shown in Table 1 described above. An input / output pattern is created from the raw data shown in Table 1. In this example, the value of the refractive index exceeds 1.0, so the data in Table 1 is normalized and replaced with the pattern shown in Table 3. In Table 3, for convenience, the normalization formula is: refractive index = 1.4 + output / 3.0.

[0045]

[Table 3]

(2) As shown in FIG. 3, the auxiliary input layer 11
1, a neural network composed of the input layer 112, the intermediate layer 113, and the output layer 114 is generated, and the coupling weight between the auxiliary input layer 111 and the input layer 112 is fixed at 10.0.

(3) Table 4 shows the input pattern corrected by the fixed coupling load 10.0 and the inverse sigmoid function f '(x).

[0048]

[Table 4]

As shown in Table 4 above, for example, when the value of SiO ₂ (0.5293) of the pattern p1 is input to the neuron unit of the auxiliary input layer 111, 5.293 is input to the input layer 112.
The output of the neuron unit of the input layer 112 is as follows: f (5.293) = 0.995, which is the same as inputting the raw data of Table 1 directly to the input layer.

(4) The neural network shown in FIG. 3A is trained in the pattern shown in Table 4.

(5) Next, as shown in FIG.
Contrary to (4), the coupling load from the auxiliary input layer 111 to the input layer 112 is set as a free load, and the input layer 112 to the intermediate layer 113 are set.
And the coupling load of the output layer 114 is fixed.

(6) Here, for example, in order to obtain the set of components for outputting the refractive index of 1.55, one inverse solution pattern is generated as shown in Table 5.

[0053]

[Table 5]

(7) The pattern shown in Table 5 is learned by the neural network shown in FIG. 3 (B). (8) When the learning converges, the inverse solution is decomposed into the weights of the auxiliary input layer 111 and the input layer 112, and is expressed as shown in Table 6. In particular, in this embodiment, the value output by the input layer is the inverse solution as it is. This is done in advance from the input layer 112 to the intermediate layer 113.
This is because it is assumed that the output to the will be the ratio% value of Table 2.

[0055]

[Table 6]

In Table 6, the output of the input layer 112 =? ?
= Since the value of the inverse solution is the value through the sigmoid function f (x), the value range is 0 to 1.

In the above (3) of this embodiment,
When the weight is set to 1.0, the range of the correction value by the inverse sigmoid function f ′ (x) is about −10.0 in (4).
Since the value is up to 10.0, which is beyond the range that the simulator can handle, the weight is set to 10.0, but the correction value is not limited to the above example, and even if the weight is 1.0, the correction value can be handled. Other weights may be set in the simulator.

The output value of the neuron unit from the input layer 112 to the intermediate layer 113 is shown in Table 3 as the input value (%).
Although it is necessary to set (sigmoid function f (x) is not passed), in reality, the input layer 112 to the intermediate layer 113 are used.
The sigmoid function f (x) is applied between the two. Therefore, if a value corrected by the inverse sigmoid function f ′ (x) is set in advance in the input layer 112, the result is f (f ′ (x)).
= X, and the influence of the sigmoid from the input layer 112 to the intermediate layer 113 can be apparently eliminated.

Further, in the above example, when the ratio does not become 100% but becomes 50% or 300%, the ratio checking neuron for checking the ratio after learning in the forward direction. Can be added to the neural network as shown in FIG. 4, and the output value for ratio check can be added to the pattern in Table 5 to obtain the inverse solution. In FIG. 4, black circles indicate added check neurons. At this time, the connection weight of the neurons not connected to the added check neuron is set to 0. The coupling load between the auxiliary input layer and the input layer is a free coupling load, and the coupling load between the input layer and the intermediate layer is 1.0. The total number of ratios is collected in the neuron indicated by the black circle at the right end of the intermediate layer, and the coupling weight from the intermediate layer to the output layer is f (total ratio).
Therefore, f (f (total of ratios)) is output from the output layer.

As described above, according to the present invention, in the neural network which is generally used and is composed of the input layer, the intermediate layer and the output layer, the auxiliary input layer is added in front of the input layer, and the auxiliary input is made first. By fixing the coupling weight coefficient between the layer and the input layer, learning is performed between the input layer, the intermediate layer, and the output layer, which are free coupling. Next, the fixed coupling between the auxiliary input layer and the input layer is released, the coupling weight coefficient between the input layer, the intermediate layer, and the output layer is fixed, and the auxiliary input layer and the input layer are learned by learning the auxiliary input layer and the input layer. An inverse solution is obtained between the layers.

[0061]

The results obtained by applying the present invention will be described below.

As a model, in a three-layer neural network, a model without a sigmoid function is set in the first layer, and the backpropagation method is applied as a learning algorithm. The learning pattern in this case is an exclusive OR (XOR) pattern, and x [xor] y = z, where (x, y, z) = (0, 0, 0), (0, 1, 1),
There are four types, (1,0,1) and (1,1,0).

After learning under the above conditions, z = 1, z =
As a result of obtaining a set of inputs for obtaining 0.5, (0, 1,
1) and (0.5,1,0.5) were obtained. (0, 1,
1) is a known pattern, which is a convincing result.
Also, the latter means that an unknown pattern has been discovered.

As another adaptation example, in a three-layer neural network, a model without a sigmoid function is set in the first layer, and the backpropagation method is applied as a learning algorithm to obtain an inverse solution. .

FIG. 5 is a diagram for explaining the effect of the present invention. The figure (A) is an input pattern, and the figure (B) is an inverse solution. As shown in FIG. 6B, the inverse solution often extracts the features of “up”, “down”, “mountain”, and “valley” of the input pattern. Further, as shown in FIG. 7C, when the inverse solutions of "up" and "mountain" were obtained, the inverse solution having both characteristics was obtained.

Therefore, according to the present invention, a desired inverse solution can be obtained by providing a general neural network with an auxiliary input layer and changing the coupling weight coefficient. In the field where a neural network is used for analysis, if a good result is obtained by the present invention, the burden of new development of an inverse solution neural network for obtaining an inverse solution and an inverse solution algorithm is eliminated.

[Brief description of drawings]

FIG. 1 is a diagram for explaining an outline of a system of the present invention.

FIG. 2 is a flowchart showing an outline of the operation of one embodiment of the present invention.

FIG. 3 is a diagram showing a neural network in the case where a coupling weighting coefficient is changed according to an embodiment of the present invention.

FIG. 4 is a diagram showing a case where a neuron for ratio check is added to the neural network according to the embodiment of the present invention.

FIG. 5 is a diagram for explaining the effect of the present invention.

FIG. 6 is a diagram showing a case where learning is performed by an inverted triangular network in a conventional direct inverse modeling method.

[Explanation of symbols]

 100 output pattern 110 neural network 120, 160 learning pattern 111 auxiliary input layer 112 input layer 113 intermediate layer 114 output layer 150 desired input pattern

Claims (1)

[Claims] 1. A neural network (110) comprising an input layer (112), an intermediate layer (113) and an output layer (114) and a learning pattern (120), the input layer of the neural network (110). (11
An auxiliary input layer (111) having the same number of neuron units as the input layer (112) is provided in front of 2), and the coupling coefficient between the input layer (112) and the auxiliary input layer (111) is fixed at a predetermined coefficient. After the learning, the neural network (110) is trained by the learning pattern (120), and after the learning, the input layer (112) and the auxiliary input layer (11)
The fixed coupling of the coupling coefficient of 1) is released, and the coupling coefficient of the input layer (112) to the output layer (114) of the neural network (110) is fixedly coupled, and the output pattern (150) has a desired value. A method for obtaining an inverse solution by learning a neural network, characterized by learning the neural network (110) using an arbitrary learning pattern (150, 160) such that