JPH06162232A - Neural network device consisting of quantum neuron - Google Patents

Abstract

PURPOSE:To provide the neural network device using a quantum neuron for taking a quantized random value. CONSTITUTION:This device is provided with a quantum neuron 1 which consists of multiplying means 31-3n for multiplying neuron values which the respective neurons 21-2n have by weight corresponding to its neurons 21-2n, an adding means 4 for adding all of outputs of these multiplying means 31-3n, an arithmetic means 7 for adding and subtracting half of weight corresponding to its quantum neuron itself to and from the output of its adding means 4, a deciding means 8 for comparing the output of its arithmetic means 7 with a threshold determined in advance, and the quantum neuron which takes a quantized random neuron value is provided. By using plural quantum neurons thereof 1, a network is constituted. In such a way, the number of neurons, the number of connections, and the calculation time can be curtailed remarkably, and also, a between approximate solution can be obtained within a short period.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network device composed of quantum neurons.

[0002]

2. Description of the Related Art In recent years, for example, when an item is transported from a plurality of suppliers having an inventory to a plurality of consumers having respective demands at a minimum cost, an approximate solution of combination optimization of integer solutions is obtained. Neural networks using neurons have come to be used for obtaining.

Such a neural network is a network in which a large number of neurons are connected in a complicated manner as shown in the conceptual diagram of FIG. 5, and each neuron value is obtained by multiplying another connected neuron value by a connection weight. It is determined by the input / output function from the sum of the values In the figure,
n _i , n _j , n _k are neurons, x _i , x _j , x _k are the values of the respective neurons n _i , n _j , n _k , w _ji is the weight of the connection from the neuron n _i to the neuron n _j Yes,
The same applies to w _jk , w _ij , w _ik , w _ki , w _kj , w _ii , w _jj , and w _kk . Although there are many variations depending on the connection form of the network and the input / output functions, the symmetric connection neural network handled here is a network in which all neurons are connected bidirectionally, and moreover, from any neuron n _i to neuron n _j . equal weight of the binding of the weights and the neuron n _j binding to neuron n _i of (symmetric), that is, w _ij = w _ji.

Usually, the conventional neuron value takes a binary value of "0" or "1" or a continuous value, and the state transition of the neuron is defined as follows.

Binary model: xi (t + 1) = 1 (Σj wijxj (t) + hi>
0) xi (t + 1) = 0 (Σj wijxj (t) + hi <
0) xi (t + 1) = xi (t) (Σj wijxj (t) + hi =
0) Continuous model: x _i (t + 1) = f [f ^-1 (x _i (t)) + (Σ _j w _ij x _j (t)
+ H _i )] ε [0,1] f (z) = 1 / (1 + exp (−z / T)) (T is an appropriate constant) In the above model, x _i is the value of neuron n _i , and w _ij
Is the weight of the connection from neuron n _j to neuron n _i ,
h _i is the threshold value of the neuron n _i , and f is the input / output function of the neuron.

[0006]

Conventionally, when a combinational optimization problem of integer solutions is solved by using a neural network, it is easy to obtain a good solution by expressing the integer of the solution by the number of fired neurons (counting method). It is said that. However, the conventional counting method requires a large number of neurons that form a symmetric connected neural network,
As a result, the number of connections between neurons also increases. As a result, when simulating with a conventional sequential computer, the calculation time is proportional to the square of the number of neurons, so that there is a drawback that the calculation time becomes long and the storage capacity becomes large.

Therefore, there has been a demand for the construction of a new symmetric connected neural network suitable for the combinatorial optimization problem of integer solutions. The present invention has been made in view of the above situation, and an object thereof is to provide a nerve using a quantum neuron that takes a quantized discrete value instead of a conventional neuron having a binary or continuous value. To provide a network device.

[0008]

In order to achieve the above object, the first aspect of the present invention provides a multiplication means for multiplying a neuron value of each neuron by a weight corresponding to the neuron, and an output of the multiplication means. And an arithmetic means for adding / subtracting half of the (feedback) weight corresponding to the quantum neuron itself to the output of the adding means, and comparing the output of the arithmetic means with a predetermined threshold value. It is characterized by comprising a quantum neuron which takes a quantized discrete neuron value, and a network is constructed by using a plurality of the quantum neurons. According to a second aspect of the present invention, in the neural network device including the quantum neurons according to the first aspect, a vibrating means that obtains a minimum solution by further giving quantum fluctuations to the approximate solution obtained by the network using a plurality of quantum neurons is provided. It is characterized by being provided.

[0009]

According to the neural network system including quantum neurons of the present invention, the number of neurons, the number of connections, and the calculation time can be significantly reduced, and a better approximate solution can be obtained within a short time. Further, when quantum fluctuation is given to the approximate solution by the vibrating means, the accuracy of the approximate solution is further improved and the probability that the minimum solution is obtained is significantly improved.

Next, the basic principle of the present invention will be described. First, a symmetric coupling network (QSNN: Symmetric Networks) including quantum neurons of the present invention.
Quantum Neurons) is defined as follows. That is, the value x _i of an arbitrary neuron n _i takes an integer value from _mi to M _i, and one neuron (for example, neuron n _k ) is sequentially updated at each time as follows. (All neurons other than k, n _j (j ≠
V _j (t + 1) = V _j (t)) for k):

[Equation 1]

Next, the energy E of the network is conventionally defined as follows.

[Equation 2]

From the above definition, a symmetric connection network composed of quantum neurons is used for each lattice point (each neuron value x _i
Can be derived to converge to the minimum value of the energy at {m _i , ..., M _i } (an integer value) (Omit proof: The Institute of Electronics, Information and Communication Engineers, IEICE Technical Report, IEICE Technical Report) Vol.92, No.59 Kiyoshi Matsuda, "Quantized and Careful Neurons and Fluctuations" 19
(See May 27, 1992). Therefore, the symmetric connection network consisting of quantum neurons is guaranteed to stop at the minimum energy value even if the calculation is started from a neuron having an arbitrary initial value.

Next, the quantum fluctuation associated with the quantization will be described. In general, in a symmetric neural network, it is often difficult to obtain a minimum value (approximate solution) and obtain an optimal solution (minimum value) (the optimal solution has not been obtained even in the above simulation). Therefore, attempts have been made to obtain a better approximate solution by probabilistically operating the symmetrical connection neural network such as simulated slow cooling. But,
Simulated annealing requires a lot of calculation time and is not realistic.

Therefore, in the present invention, the probability of reaching the optimum solution is greatly improved by providing a vibrating means for giving a certain kind of fluctuation (quantum fluctuation) accompanying the quantization.

The state transition of a quantum neuron accompanied by quantum fluctuation is defined as follows.

When w _kk <0, for c such that 0 <c <0.5,

[Equation 3] When w _kk > 0, for c <0.5 <c <1,

[Equation 4] Now, if w _kk <0, then for any c such that 0 <c <0.5, Σ _j w _kj x _j + h _k + 0.5 · w _kk / 2>
If 0, then Σ _j w _kj x _j + h _k + c · w _kk / 2> 0
And similarly, Σ _j w _kj x _j + h _k −0.5 · w _kk /
If 2 <0, Σ _j w _kj x _j + h _k −c · w _kk / 2 <
Since 0, state transitions made at QSNN are also made at FQSNN.

However, Σ _j w _kj x _j + h _k + 0.5 · w
_kk / 2> 0 and Σ _j w _kj x _j + h _k −0.5 · w
Even if none of _kk / 2 <0 holds and no state change occurs in QSNN, Σ _j w _kj x _j + h _k + c ·
w _kk / 2> 0 or Σ _j w _kj x _j + h _k −c · w _kk
Either of / 2 <0 may be satisfied, and FQS
A state change can occur in the NN.

Thus, when c = 0.5, it is not allowed, and c
The state transition that is possible when ≠ 0.5 is called “quantum fluctuation”. FQS is a circuit network consisting of quantum neurons with quantum fluctuations.
NN (Symmetric Networks with Quantum Neuro)
ns and Fluctuation). In the case of FQSNN, energy convergence is not guaranteed. The calculation is terminated by detecting the following pseudo convergence. That is, "the energy value does not change for a certain period" or "a specific energy value frequently appears for a certain period".

Many variations are possible with respect to equation (3) or equation (4) of the FQSNN state transitions. For example,
Even if it fits in the definition, make the state transition stochastic,
Alternatively, the value of c may be dynamically controlled (simulated slow cooling).

Since the binary neuron can be regarded as a special kind of the quantum neuron, the quantum fluctuation can be applied and effective for the neural network device including the binary neuron. For example, it can be applied to the traveling salesman problem.

[0021]

Embodiments of the present invention will now be described with reference to the drawings. 1 is a block diagram of an embodiment of the present invention, and FIG. 2 is a functional block diagram of the quantum neuron surrounded by the dotted line in FIG. As shown in FIG. 1, the neural network device composed of quantum neurons of the present invention constitutes a network using a plurality of quantum neurons 1. Each quantum neuron 1 has a neuron 21 as shown in FIG. , 2n, a multiplier 31, ..., 3n, an adder 4, a threshold value setting device 5, a quantum neuron weight setting device 6, a computing device 7, and a judging device 8.

Next, the case of applying the neural network device composed of quantum neurons of the present invention to the case where an item is transported from a plurality of suppliers having an inventory to a plurality of consumers having respective demands at a minimum cost. Will be described. In addition, it is assumed that the transportation cost per item between each supplier and the consumer is different.

Now, f _{ij is} the number of goods transported from the supplier i to the consumer j, c _{ij is} the transportation cost per item from the supplier i to the consumer j, and s _{i is} the inventory quantity of the supplier _i. , Supposing the number of demands of the consumer j is d _j , the following equation (constraint equation) is established.

[0024]

[Equation 5]

In the above equation (5), the first term is that the number of goods transported from the supplier coincides with the number of stocks, and the second term is
The term indicates that the number of goods transported to the consumer coincides with the demand quantity, and the third term indicates that the transportation cost is the minimum.

Therefore, an approximate solution can be obtained from the value of each neuron at the time of convergence as in the conventional case as follows. That is, in the above energy formula (2), if the subscript of the neuron is two-dimensional for convenience (i → pq, j → rs),

[Equation 6] Becomes Therefore, x _ij in equation (6) is regarded as f _{ij in} equation (5),

[Equation 7] Then, the conventional constraint equation (5) and the energy equation (6) of the network consisting of quantum neurons are the same.

When a conventional continuous value (finally converges to a binary value) or a binary neuron is used, the maximum possible number n of neurons (x _ij1 , ..., x) for each f _ij is used.
_ijn ) is prepared in the same way, and f
ij is expressed by (x _ij1 , ..., X _ijn ). That is, in the energy equation (2), if the subscript of the neuron is three-dimensionally for convenience, (i → pqu, j → rsv)

[Equation 8] Becomes Therefore, Σx _ijk (k = 1, ..., n) in the equation (8)
_ij ) is regarded as f _{ij in} equation (5), and in the same manner as above,

[Equation 9] Then, the conventional constraint equation (5) and the energy equation (9) of the network consisting of binary or continuous-valued neurons
Are the same.

From the above, when the network of quantum neurons of the present invention is used to solve problems such as combination optimization of integer solutions, the number of neurons, the number of connections, the amount of memory, and the calculation time are much smaller than those in the conventional counting method. It can be said that an approximate solution can be obtained. Therefore, more simulations than the counting method can be performed in the same time, and the possibility of obtaining a better solution increases. In addition, it becomes easy to express the target combinatorial optimization problem with a neural network. The calculation of the neural network is performed as shown in the flowchart of FIG. 3 except for the updating equation (1) of the neuron value.

(Specific Example 1) Next, in the above embodiment,
The equation for transporting goods from multiple sources with inventory to multiple consumers with their respective demands at the minimum cost is as follows, using the two constraints and the optimization function as follows: Can be converted. That is, the expression A (Σ _j x _ij −s _i ) ² + B (Σ _i x _ij −d _j ) ² + C Σ _ij c _ij x _ij is to be minimized. Here, the following values are inserted into the stock number s _{i of} the supplier _i , the demand number d _{j of} the consumer j, and the transportation cost c _ij per item from the supplier i to the consumer j by adding the following values to the conventional values: The counting method and the quantization method by QSNN of the present invention were simulated and compared. In addition,
The minimum shipping cost is 38.

Stock number of each supplier: s ₁ = 5, s ₂ = 3
s ₃ = 4, s ₄ = 6 Number of demands of each consumer: d ₁ = 2, d ₂ = 7, d ₃ = 3, d
₄ = 2, d ₅ = 4 Transportation cost between each source and each consumer: c ₁₁ = 5, c ₁₂ = 1, c ₁₃ = 7, c ₁₄ = 3, c ₁₅ = 3 c ₂₁ = 2, c _{22 = 3, c 23 = 6} , c 24 = 9, c 25 = 5 c 31 = 6, c 32 = 4, c 33 = 8, c 34 = 1, c 35 = 4 c 41 = 3, c 42 = _{2, c 43 = 2, c} 44 = 2, c 45 = 4

Each f _ij is represented by a single neuron value in the quantization method, and is represented by 7 neurons in the counting method. The weights of the three constraint conditions are A = B = 80 and C = 0.2.
It was set to 3. The simulation program is MS-
Created in C and run on PC98RX. The initial value of the neuron and the selection of the neuron to be updated first are random numbers (M
S-C rand function). To update the neurons, all the neurons were sequentially selected from the ones selected by random numbers, and when the circuit made a cycle, the first neuron to be updated was selected again, and this was repeated. Table 1 shows the simulation results
Shown in.

[0032]

[Table 1]

From Table 1, the conventional counting method is 500
Although it took about 11 hours to perform each simulation, the quantization method of the present invention required 10 times 5000 simulations, which was half, about 5 hours. On the other hand, since the approximate solutions (solutions satisfying the constraint conditions) are obtained at the same rate of 22 to 24% in the simulation, the average time required to obtain the approximate solution is 1/20. Further, the average transportation cost is 52.9, which is the same in both methods, but the obtained minimum transportation costs are 41 and 44, and the quantization method shows a slightly better result. This is probably because the quantization method took half the time, but the simulation was 10 times more effective than the counting method.

(Specific Example 2: Quantum Fluctuation) In the previous specific example 1, when the fluctuation with c = 0.33 is always given, F
The QSNN results are shown in Table 2 along with the QSNN results. For fairness, QSNN simulations include all simulations under the same conditions as FQSNNs.

In the case of FQSNN, the state transition does not strictly converge. In the simulation, if the state (energy) did not change over 5 consecutive rounds of updating of all neurons, it was considered as converged for convenience. If the calculation (update) did not converge even after 50 cycles, the simulation was aborted.

[0036]

[Table 2]

As shown in Table 2, the FQSNN was simulated 500 times, but the performance was significantly improved.
The optimum solution (= 38), which was not obtained even with 5000 simulations in QSNN, was obtained in 141 out of 500 simulations. Average transportation cost is 52.9 to 4
There was a significant improvement of 0.9. Furthermore, not only is the quality of the solution improved, 99% of the simulations converge despite the fluctuation, and the average time per simulation (average number of iterations until convergence) does not double. Convergence is also very good, and all converged simulations obtain approximate solutions (constraints are satisfied).

From the above, the conventional symmetric connected neural network is deterministic, and the solution (convergence point) depends entirely on the initial value of the neuron (initial value sensitivity). Therefore, the selection of initial values became a new theme. However, the F of the present invention
Although the QSNN is deterministic (w _ii ≦ 0 in this example, deterministic, and the added fluctuation was also deterministic), this simulation result shows that the solution of FQSNN (convergence point ) Does not depend much on the initial value.

Since the binary neuron is a special case of the quantum neuron, the quantum fluctuation can be applied not only to the quantum neuron but also to the binary neuron. Also, the FQSN
The calculation of N is performed in the conventional manner according to the flow chart as shown in FIG. 4, except for the updating equation (1) of the neuron value.

[0040]

As described above, according to the present invention,
By applying a neural network device of symmetric coupling consisting of quantum neurons that take quantized discrete values instead of binary or continuous values as in conventional neurons to the combination optimization of integer solutions, etc. Number of neurons, number of connections,
The calculation time can be greatly reduced, and a better approximate solution can be obtained within the same time. Furthermore, by giving a quantization fluctuation to this approximate solution, an excellent effect that the probability of reaching the optimum solution is significantly increased is exhibited.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a neural network device including quantum neurons according to an embodiment of the present invention.

FIG. 2 is the quantum neuron shown in FIG. 1 (a part surrounded by a dotted line)
FIG.

FIG. 3 is a flowchart showing a calculation procedure by QSNN according to the present invention.

FIG. 4 is a flowchart showing a calculation procedure by the FQSNN of the present invention.

FIG. 5 is a conceptual diagram of a conventional neural network composed of neurons.

[Explanation of symbols]

1 ... Quantum neuron, 21, ..., 2n ... Neuron, 3
1, ..., 3n ... Multiplier, 4 ... Adder, 5 ... Threshold value setter, 6 ... Quantum neuron weight setter, 7 ... Arithmetic unit, 8
… Judge.

Claims (2)

[Claims] 1. A multiplication means for multiplying a neuron value of each neuron by a weight corresponding to the neuron, and an addition means for adding all outputs of the multiplication means,
The quantum neuron is composed of arithmetic means for adding / subtracting half of the weight corresponding to the quantum neuron itself to the output of the adding means, and judging means for comparing the output of the arithmetic means with a predetermined threshold value. A neural network device comprising a quantum neuron, which comprises a quantum neuron that takes a neuron value of, and a network is configured by using a plurality of the quantum neurons. 2. A neural network device comprising quantum neurons according to claim 1, further comprising vibrating means for obtaining a minimum solution by further providing quantum fluctuations to an approximate solution obtained by a network using a plurality of quantum neurons. A neural network device consisting of quantum neurons.