JPH0476659A - Neural network - Google Patents

Abstract

PURPOSE:To reduce a calculation quantity up to the end of repetitive calculation by setting a train of the update of the states of respective neuron elements by referring to information regarding coupling between respective neuron elements. CONSTITUTION:This neural network consists of the neuron elements 1, coupling parts 5, and a train setting means 7 and when this neural network is in operation, there are inputs I1 - I5 and outputs O1 - O5. Coupling loads wij are defined at the coupling parts 5 information on the coupling loads wij is sent from the coupling parts 5 to the train setting means 7; and order or a series is determined according to the information on the coupling loads wij and the neural network is put in operation according to the information on the order or train.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Field of Industrial Application) The present invention relates to a neural network that can be applied to optimization problems, associative memory, and the like.

(Prior Art) In recent years, the effectiveness of information processing using neural networks has been reconsidered, and research has become active in various places. Applications of neural networks span a wide range of fields, including applications to optimal combinatorial problems that take advantage of their parallel nature, and pattern recognition that utilizes the learning function of neural networks.

The configuration of the neural network is a fully connected type network (so-called Hopfield (l1
(opr Ic1d) type net), a hierarchical net shown in FIG. 4(b), a random connection net shown in FIG. 4(c), etc. are mainly used.

For example, a connection weight W1. is defined.

Specifically, as shown in FIGS. 5(a) and 5(b), when the input signal X l + 2 + ... + xN is given to the neuron element 101, the input/output function f and the threshold value θ are as follows. The arithmetic processing is performed according to y+-f(ΣWIIXI-θ1), as shown in FIG. 5(b).

It has input/output characteristics as shown in (c). These neuron elements output an output 0.0.0 which is obtained by applying the input l shown in FIG. Express solutions to optimal combinatorial problems and identify patterns.

Application of neural networks to optimization problems and associative memory (J, J, flopfield and DJ, T
ank: Old o1, Cyberr+, 52141 (198
5)), a fully connected net or a randomly connected net consisting of mutually connected elements is often used, and the state of each neuron element is updated either synchronously or asynchronously. or

Although the method of synchronizing the update of the state of each neuron element is superior in that it takes advantage of parallelism and is easier to analyze because it can be treated mathematically as a product of matrices, It also has drawbacks such as the fact that convergence is not guaranteed and there is a risk of vibration.

On the other hand, although the method of updating the state of each neuron element asynchronously is inferior in terms of parallelism, it is effective in terms of processing speed when performing the above processing using a general computer that cannot perform parallel calculations. If a computer that can perform parallel processing asynchronously is developed in the future, it is thought that the drawbacks in terms of parallelism will be overcome.

Furthermore, this method of asynchronously updating the state of each neuron element has the advantage that convergence to a minimum solution is theoretically guaranteed.

However, when updating the state of each neuron element asynchronously, in what order to update each neuron element, that is, to perform repeated calculations,
Currently, there are no clear guidelines for this, and usually one selects any one neuron element with equal probability using, for example, random numbers, updates the state of that neuron element according to a set rule, and then selects the next neuron element with equal probability. It is common to move on to selecting a neuron element and repeat the same process.

However, in general, this selection method using random numbers requires many unnecessary calculations because even stable neuron elements are selected with the same probability as other neuron elements. It may happen. Furthermore, even if there is a neuron element that has the characteristic of having an absolute influence on other neuron elements, this neuron element is not treated specially, so if that neuron element is accidentally selected at a late stage, In some cases, previous iterative calculations may be wasted.

That is, when selecting neuron elements whose states are to be updated using random numbers, information from parameters such as connection weights that characterize the neural network is not used at all.

(Problem to be solved by the invention) As mentioned above, in conventional neural networks that update the states of neuron elements asynchronously, when updating the states of neuron elements, information from parameters such as connection weights is used. Since neuron elements are selected using random numbers without being used, unnecessary calculations are sometimes forced.

In order to solve these drawbacks, the present invention sets an appropriate order or sequence of neuron elements according to the nature of the connection weights of the neural network, and updates the neuron elements accordingly, thereby reducing repetitive calculations. This provides a neural network that reduces the amount of calculations required until completion.

[tδ formation of the invention]

(Means for Solving the Problems) The neural network of the present invention is a neural network in which the states of a plurality of neuron elements connected to each other are updated asynchronously, and the neural network is a neural network in which the states of a plurality of neuron elements connected to each other are updated asynchronously. The gist of the present invention is to include a sequence setting means for setting a sequence for updating the state of each neuron element by referring to the information.

(Operation) According to the neural network of the present invention, when updating the states of neuron elements that are updated asynchronously, each neuron is The state of the element is updated.

Note that it is theoretically guaranteed that the property of this neural network to converge to a minimum solution is maintained even if the order of the neuron elements to be updated is manipulated.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings.

FIG. 1 is a conceptual diagram of a neural network according to a first embodiment of the present invention.

This neural network consists of a neuron element 1, a coupling section 5, and a sequence setting means 7, and when the neural network operates, it is accompanied by inputs 11-I5 and outputs 01-05.

Further, a joint load Wll is defined in the joint 5, and information such as the joint load Wll from the joint 5 is transmitted to the series setting means 7, which determines the order or series according to the information on the joint load Wll. , the neural network is operated based on the order or sequence information.

This will be explained in detail below.

First, the energy E of the entire neural network is defined by the following equation.

The same goes for children. ) and the neuron element j, ■ is the state of the neuron element i, and θ1 is the threshold value of the neuron element i.

Vl is ■1-±1...(2) 2
The joint load WII is Wli”o.

W 1wm W 11 However, (i, j-1,..., N)...
It has the property (3).

Now, when Vl changes by ΔV, the amount of change ΔE in energy E is as follows: V, (t+1.) 2+-11-1'-' (1) However, W
, is the first neuron element (hereinafter referred to as neuron element i). If the state Vl (t) (i-1, ..., N) of the neuron element at time t is updated according to other neuron elements, the energy E The amount of change in can always be negative or zero.

This means that if the state of the neuron element is updated according to equation (5), the energy E will always reach its minimum.

In this embodiment, a case will be considered with respect to the joint load, where there are the following properties: W,, 1≧a, a≧1 (6), where W,=Σ 1wzl (7). That is, when the values of Wl are arranged in descending order of decreasing value, when comparing adjacent values, the larger value is equal to or larger than a times the smaller value. Furthermore, since the numbers assigned to the neuron elements can be arbitrarily converted, equation (6) can be assumed without losing the killing effect.

In addition, it is thought that neuron elements that are connected by strong connections are likely to affect other neuron elements, so it is possible to select neuron elements in descending order of Wl value and update their states. This is effective at the initial stage of operation.

Therefore, the initial state is 1-0, T is set to a certain appropriate number, and at the time of 1≦t≦T, neuron elements are selected in descending order of the value of Wl, and the state of the neuron elements is updated. One of the effective methods in this embodiment is to update the value of Vl and to randomly select neuron elements in the subsequent period when t>T.

next, a-1,41421356゜ θ +-0 (t-1,..., N).

The case of -N will be explained as an example. Note that at this time, if a≧1, a similar method can be applied.

FIG. 2 shows a graph of E versus t in the case of N-100. This is a graph in which the initial state and the initial value of the bonding load are randomly selected within a range that satisfies predetermined conditions, and the energy E is averaged over 10 trials. The solid line is based on the method based on this embodiment, and the dotted line is based on the conventional method of randomly selecting neuron elements. As is clear from the figure, the energy E decreases faster and converges faster in the solid line than in the dotted line.

In both cases, when randomly selecting neuron elements, a list array (1ist[!
]-(remainder when i is divided by N) +1 (i-1,-3N
)), the contents of the array were rearranged using random numbers, and neuron elements were selected according to this array. This is superior to the case where neuron elements are selected using random numbers each time, since each neuron element will always have its turn at some point.

Next, a second embodiment of the present invention will be described.

In this second embodiment as well, it is assumed that equations (1) to (7) in the first embodiment are satisfied. However, unlike the first embodiment, the second embodiment does not simply perform operations in descending order of W.

In other words, if the absolute value 1w of a particular connection weight is extremely large, by selecting one of the neuron elements on both sides and updating it once, the neuron elements on both sides can be updated. Since the state of is fixed and will never change again, it is possible to proceed with the iterative calculation by excluding these two neuron elements from then on.

To illustrate the nature of such joint loads, Figure 3 (
Define the symbols in a) and (f), respectively.

First, when the connection to neuron element j has the following properties, a solid arrow [-] is drawn from neuron element i to neuron element j as shown in FIG. 3(a).

In the eighth equation above, Σ represents excluding only -i when calculating the sum of k-1 and .about.N. Therefore, if equation (8) holds true, the state of neuron element j depends only on the state of neuron element i and is not influenced by other neuron elements.

In addition to equation (8), if the following holds for neuron element i, arrows will be drawn in both directions as shown in Figure 3(b), but in this case, between neuron element i and neuron element j forms a cluster consisting of two neuron elements and behaves independently from other neuron elements. Therefore, in the figure, clusters are represented as areas surrounded by thin solid lines.

Next, I W+++ l + I Wl121 +-+ I
Wz.

〉 Σ I w + + l (n < N)
1-11. -・, l* ... (10) When the following holds true, as shown in FIG.
Draw a dotted arrow on the book. There are many different ways to draw this dotted arrow. For example, in this case, the state of neuron element i is determined once the states of neuron elements j~, j0 are determined, and is often not influenced by others.

When the two types of arrows defined above are used to illustrate the state of connection weights between neuron elements, regions where there are no outward arrows form clusters, and the internal state of the clusters becomes stable. For example, in many cases they are no longer subject to external influences.

Therefore, such properties can be utilized in this embodiment. For example, when a raster is formed as a part of a neural network as shown in FIG. 3(d), the states of neuron elements are updated in the order of CI, C2°, ..., C32 in the same figure. If the calculation is performed once at a time, the state of this cluster is determined, and subsequent iterative calculations can be performed by excluding neuron elements included in this cluster.

At this time, only one of C1 and C7 needs to be updated, and the order of C 3 r to C9 may be changed. That is, if the arrows do not form a loop within the cluster, the order of updating can be arbitrarily set so as not to contradict the direction of the arrows.

For example, if there is a raster as shown in FIG. 3(e), CI r C2r . . . C in the same figure
It is sufficient to update once in the order of 7. In this case also C2
The order of + to C7 may be changed.

Furthermore, if there is a cluster consisting of dotted arrows as shown in Figure 3(f), the conventional method is applied until the state within it is determined, but once it is determined, they are excluded. It is possible to operate a neural network using

As described above, according to this embodiment, the properties of the connection weights are investigated before operating the neural network, and an appropriate sequence or sequence of neuron elements is prepared based on the properties of the connection weights. Neural networks can be operated using information from other people. For example, after the first state update, it is clear that the state of a certain neuron element will not change any more, or it is expected that there is a high possibility that the state will not change. For both specific neuron elements, it is possible to take a method of not selecting them once they have been selected, which eliminates unnecessary calculations that were overlooked when conventional equal probability selection was performed using random numbers. can be omitted to save the amount of calculation.

〔Effect of the invention〕

According to the present invention, information regarding connections between each neuron element is referred to when selecting neuron elements in the operation of a neural network, so an appropriate order or series of neuron elements can be set. , by updating the neuron elements accordingly, it is possible to reduce the amount of calculation by =1 until the iterative calculation is completed.

[Brief explanation of drawings]

Figure 1 is a conceptual diagram of the present invention, Figure 2 is a graph comparing the present invention and the conventional method to show how the energy of a neural network converges to the minimum value, and Figure 3 is a diagram of the method according to the present invention. FIG. 4 is a conceptual diagram showing the properties of the neural network connection weight in the second embodiment, FIG. 4 is a connection diagram of various types of conventional neural networks, and FIG. 5 is a diagram explaining the principle of the method of connecting neuron elements. 1...Neuron element 5...Coupling unit 7...Sequence setting means

Claims (1)

[Claims] A neural network in which the states of a plurality of neuron elements connected to each other are updated asynchronously, and a series of updates of the state of each neuron element is set by referring to information regarding the connections between the neuron elements. A neural network characterized in that it has a sequence setting means.