JPH01229360A - Learning processing system in boltzmann machine - Google Patents

Abstract

PURPOSE:To acquire the progress state of learning or a normal learning operation, and to search a parameter to arrange an optimum operating state by providing a monitoring part, and monitoring the operating state of a network in the course of the learning. CONSTITUTION:The network 10 is constituted so as to include, at least, a visible unit 11 and a hidden unit 12, and in addition, to be capable of setting weight Wij of each unit S. Then, at the time of the setting of the weight Wij, the probability distribution P<->(Valpha) of a pattern having occurred actually correspondingly to a learning pattern parameter given by the initial setting part 21 of a learning processing part 20 is adjusted so as to be the occurrence probability distribution P<+>(Valpha) of the pattern. Namely, a weight updating part 22 adjusts the weight Wij so as to approach to the probability P(Talpha) of the pattern occurrence of an external environment. Next, the monitoring part 30 monitors the progress state of the learning in the processing by the updating part 22 or the normal learning operation. Thus, since the quality of the operating state and the progress state can be monitored, the parameter to arrange the optimum state can be searched.

Description

[Detailed Description of the Invention] [Summary] This invention relates to a learning processing method in a Boltzmann machine, which is one of the methods for configuring a neurocomputer.

By monitoring the operating status of the network during learning, the aim is to know the progress of learning and enable the search for parameters that will optimize the operating status.

The network includes at least a visible unit and a hidden unit and is configured to be able to set weights W i j between the respective units.

In a learning processing method in a Boltzmann machine including a learning processing section that adjusts the weight Wij by learning.

A monitor section is provided, and the monitor section is configured to have a function of monitoring the progress of the learning and the quality of the operation status.

[Industrial application field]

The present invention relates to a learning processing method in a Boltzmann machine, which is one of the methods for configuring a neurocomputer.

There are some aspects of the operation of the Boltzmann machine that cannot be explained from the principle of learning rules, especially because the procedure for minimizing the measure G is simple.

For fast learning, parameters must be chosen well. Also, depending on the quality of the annealing schedule,
The state transition of the unit does not follow the Boltzmann distribution, and there is no mathematical guarantee of operation. Therefore, it is necessary to monitor the learning process, check the normality of the operating status and the progress of learning, and set appropriate parameters.

[Conventional technology]

a) Overview of Boltzmann Machine A Boltzmann machine is a learning machine with a network structure.

In this machine, by repeatedly presenting a pattern of the external environment that is desired to be stored, the weights of connections between units in the network are made to learn the probabilistic regularity of pattern generation. Furthermore, after learning is completed, the 0 weight is fixed, a part of the pattern is presented, and the remaining part is recalled.

b) Network structure and behavior A network has units that take two states, 0 and 1, which is expressed as 9th order.

Sr = 0.1 (i = 1, 2.-, n)...
・(tl units l and j are 0-weight symmetry (W, = W, i
), and each unit has a threshold value θ.

When the states of all units are viewed as vectors, there are two all-unit state vectors, as shown in FIG. Define energy for each of them as follows.

Eβ=−self w, js, s, +Σθ1S, ...(
2) All units probabilistically transition between all unit states from β=1 to 21. In the "thermal equilibrium state", the probability that all unit states are β at a certain point in time is as follows.

Boltzmann distribution: p, oc e x p (-2)...(3) These can be interpreted as first order. Weight W, is.

When positive, units S4 and Sj mutually support each other to be 1, and when negative, they mutually exclude each other from being 1. Also, the larger the threshold value θ, the larger the threshold value θ.

It is difficult for S to be 1.

The energy E is considered to indicate the degree of antagonism in the interactions between these units. The state of all units with high energy is unstable.

It is easy to move to a more stable, lower-energy, whole-unit state. The thermal equilibrium state is a state in which the antagonistic conditions have settled down, and all units more frequently transition between lower energy all-unit states.

2 Important points here are that when considering expressing some pattern in all or part of the units in the network, if the Boltzmann distribution holds in the thermal equilibrium state, 2
By adjusting the weight W.

The ability to control the probability of pattern appearance.

C) Learning method Consider a virtual external environment with a terminal that takes a state of 0.1. FIG. 4 shows an explanatory diagram for explaining the external environment.

The external environment is defined by the appearance probability p (T(x)) of the vector T& of the pattern that appears as the state of the terminal.

The goal of learning is to repeatedly present patterns and construct a model of the pattern generation mechanism of the external environment by adjusting weights in the network.

The learning method will be described below.

First, there are two types of units in the network, as shown in Figure 5. In Figure 5, 10 represents the network, 11 represents the visible unit, and 12 represents the hidden unit. It has a terminal that connects directly to the ``visible unit (v
isible unit) J 11.

The other is a "hidden unit" that has no direct connection to the outside world and whose status cannot be set or changed from the outside.
+ unit) J 12. Figure 7 shows an explanatory diagram explaining the phases for learning.
,The procedure of learning, i.e. changing one weight, consists of two phases. In phase + shown in the figure.

The visible units 11 of the network are connected to the terminals of the external environment 13 and their states are set and changed according to a predetermined probability P(T), and only the state of the hidden units 12 is changed to make the entire network reach a thermal equilibrium state. At this time, as shown in FIG. 6, which shows a vector representing the state of the visible unit 11, the probability that the vector becomes V is P'' (V
&).

P” (V&)=P (T&)
...(4).

In phase -, no unit in the network has its state set or changed, allowing all units to change state to reach a state of thermal equilibrium. At this time, the probability that the state of the visible unit 11 becomes V is set as P-(V&). The goal of learning is to construct a model of the pattern generation mechanism of the external environment 13 in the network IO, that is, the probability of pattern generation when the network IO is operated freely is P-(V,). The purpose is to adjust the parameter controlling P-(V&), that is, the connection weight W8, so as to get as close as possible to the probability of occurrence of a pattern in the external environment P(T,)=P''(V&).

It can be formulated by defining a linear scale G that represents the degree of difference between two distributions and minimizing it.

This is called the Kullback divergence.

This is a commonly used measure when estimating parameters of probability distributions. Note that P-(V&) is a function of the weight Wij at the relevant time point.

The measure G is non-negative.

P-(V&)-P" (V,) If and only then, the minimum value is 0.

To find the weights W, j that minimize the measure G.

Use steepest descent method.

θG 3 w.

However, it has been proven that it can be determined by the following formula (see references below).

aWij T (i, j= 1 to n) However, P″ij+P−1j are each phase +
.

This is the probability that S and S become 1 at the same time in phase -. This is a value that can be estimated by observing the network IO during operation.

Using the above formula, move in the direction where the scale G is the smallest.

If the amount of change in 0 weight that changes the weight is ΔW8J, then 1
It can be written as follows.

ΔWH=ε (P″t= P−ij)
...(7) However, the parameter ε (>0) must be set appropriately.

d) Weight Updating Algorithm FIG. 8 shows a flowchart for the weight updating procedure. FIG. 8 (A), (B) and (C) are -J& to represent one diagram. Weight updating consists of three cubic procedures.

○ In phase 10, obtain the estimated value of P'', .

○ Obtain the estimated value of +P-ij at phase -.

○ Update the weight using the following formula.

Wth = W, +ε (Pt=P-t=) ...(
8) 1) The difference between the two phases, Phase 10 and Phase -, is as follows.

In phase +, the visible units 11 are set and changed to the learning pattern, and only the hidden units 12 can change their states freely.

In phase -, the state of any unit changes freely without any values being set or changed.

2) In each phase, the primary procedure is repeated the same number of times as the number of learning patterns.

○ Initialize the unit.

The values of units to be set/changed in the learning pattern are set/changed.

Other units are randomly set to 0.1.

○ Set annealing schedule ie.

The state transition procedure is performed according to the way the temperature is changed and the number of recoveries corresponding to the temperature. Once annealing is completed, the Poltzmann distribution in the thermal equilibrium state is realized.

○ Perform the state transition procedure multiple times at the target final temperature, and determine how often S+ and S= become 1 at the same time.

Tally.

Aggregation is an accumulation over all learning patterns.

3) At the end of each phase, convert the frequency to probability and calculate +
Find the estimated value of P"ij+P-ij.

e) State B Transition Procedure FIG. 9 shows a flowchart regarding the state transition procedure.

Corresponds to the given weight and temperature. The algorithm shown in the figure is adopted as the state transition of the unit to realize the Portzmann distribution in the thermal equilibrium state.

O Select a unit at random from a range in which the unit of interest can be freely changed. That's the second thing.

...(9) S with a stone chamber rate of 0 Pk = □.

1 +exp(-ΔEk/T) is set to 1. ...
The procedure αω or more is repeated as many times as each unit is selected on average once in one state transition procedure.

f) Supplementary explanation In a state of thermal equilibrium. The principle of realizing the Portzmann distribution is as follows.

2〇〇 is called a sigmoidal function, and its graph is shown in Figure 10.

The sigmoidal function Pyo is a step function when T=0, and when ΔEk>0, that is, 5k=1, the energy is lower, S is always 1, and on the contrary, when ΔE, is 0, S is always 1. Sk is always set to 0.

This is a deterministic algorithm called hill-climbing, which changes the state in a direction that reduces the energy of the entire network. However, this method of change has the disadvantage that it falls into a local minimum due to the ups and downs of energy in the space where the state extends.

The sigmoidal function Pk is a step function that is blunted by a parameter: temperature T.

The unit has energy fluctuations proportional to temperature T. This allows us to escape from the local minimum.

A Boltzmann distribution is realized with the minimum point as the center of the distribution.

However, in reality, if the temperature is low, the energy fluctuation may be too small to exceed one local minimum potential peak. In that case.

Although it is not impossible to extract from the local minimum, it will take a long time. On the other hand, if the temperature T is too large, the state of the unit fluctuates too much.

extreme case.

If T = Flash, the unit's state changes will be completely random.

Therefore, in the 1-weight update procedure, we use the idea of simulated annealing, which starts from a temperature higher than the target temperature, searches for a roughly low energy point, performs a state transition, and lowers the temperature one step at a time to reduce fluctuations. By doing this, the Boltzmann distribution in the thermal equilibrium state at the target temperature can be determined more reliably and with fewer repetitions than by continuing the repetition at the target temperature. can get.

Reference 1, D.H., Ackley, G.E., Hinton.
, T., J., Sejnowski″A Learning
Algorithm for BoltzmannM
Cognitive 5science 9+1985+
147-1692, D.E., Hinton, T.J.
Sejnowski“Learning and Re
learning in Boltz11annmonac
Parallel Distributed Proc
essing Vol.

1 (The MIT Press), 282-317 [
Problems to be Solved by the Invention] The parameters that control the operation of the Boltzmann machine are the annealing schedule and the weight change width ε.

Since they are interrelated, they cannot be completely separated, but they can be broadly divided into two. The first one is related to the realization of state transition according to Boltzmann distribution in thermal equilibrium state.

The second is related to the search for weights Wij that minimize the 9 scale G.

First, we will consider the realization of a state transition according to the Boltzmann distribution in a thermal equilibrium state. It should be possible to achieve thermal equilibrium at any temperature by repeating the state transition procedure an infinite number of times.

However, in reality, it must be accomplished a finite number of times, and moreover, it must be accomplished as few times as possible. The higher the temperature, the greater the fluctuations, so there is a high possibility that the energy will be extracted from the local minimum, and it will be easier to reach a state of thermal equilibrium. Therefore, we introduced the idea of simulated annealing to start the state transition from a high temperature and let it settle at the desired temperature. Therefore, ideally, we start from a higher temperature and then do more at the same temperature. number of times, the temperature should be lowered more slowly.

After all, the processing speed and the feasibility of Boltzmann distribution.

A compromise will have to be found.

The search for a weight W that minimizes the measure G is as follows.

It is a multidimensional minimization problem. The Boltzmann machine uses a search method that changes the weight in the direction of the gradient of the three scales G. It has not been determined whether the two scales G became larger or smaller as a result of changing the weights.

For this reason, as for the -boat fishing tendency, it is possible to search for the minimum point, but there is also a possibility that the scale G may be increased.

In that case, it will cause vibration. Furthermore, in such a simple search method, the width of change ε of one weight has a large influence on the search efficiency. Generally, the smaller the change width ε, the more reliable the search will be, but the search speed will be slower. Therefore, an optimal value of the change width ε exists dynamically.

Parameters must be determined taking the above into consideration, but there is a strong interaction between the parameters.

It is difficult to search for parameters that will efficiently yield the correct answer.

In conventional Boltzmann machines, the only way to determine whether learning is good or bad is based on whether the output is correct or incorrect in relation to the input. Furthermore, if learning progress was poor, there was no way to determine what was the cause.

The present invention monitors the operating state of the network during learning to know the progress of learning.

The purpose is to enable the search for parameters that provide optimal operating conditions.

[Means to solve the problem]

FIG. 1 shows a basic configuration diagram of the present invention. In the figure, numeral 10 is a network, 11 is a visible unit, 12 is a hidden unit), 20 is a learning processing section, and 21 is an initial setting section.

22 is a weight update unit, 23 is a convergence determination unit, 24 is a learning completion result holding unit, 30 is a monitor unit, G, G1. GB represents a scale used in the monitor section of the present invention.

The illustrated network 10 corresponds to the footwork shown in FIGS. 5 and 7. Then, the learning processing section 20 performs learning as described with reference to FIGS. 7 and 8 for the units S shown in FIG.

Assuming that the respective units are 88 and Sj, the weights W and j are set as described in connection with FIG. 2, ie.

Corresponding to the learning pattern parameters given in the initial setting section 21, the 1-weight updating section 22 adjusts the weight Wij so that p-(v&) approaches P''(V&), that is, P(T&). Convergence. The determining unit 23 checks whether the weight WiJ has converged or not, and if it has converged, the result is stored in the learning completion result holding unit 24.

Regarding the processing by the 8-weight updating section 22, the monitor section 30 monitors the progress of learning and whether or not a normal learning operation is being performed. That is, as a measure in this case.

The next scale G, Gl, or CB is calculated to monitor the quality of the operating status and the progress of learning.

○ Appearance probability distribution of the pattern to be learned: P” (V
, ) and the probability distribution that it actually appears: P-(V,)
Kullback-divergence with: (V6: State vector of visible unit.) ○ Appearance probability distribution of the pattern to be learned: p' (v,) and ideal value of the Boltzmann distribution corresponding to the current weight: PI (
Kullback-divergence with V, ): ○ Ideal Boltzmann distribution corresponding to the current weights: P
Kullback-divergence between I (V/, Uβ) and the probability distribution of the actually generated pattern: P-(U,); [Effect] ○ The measure G is minimized by the Boltzmann machine. It is the evaluation function itself and indicates the progress of learning. Based on the change in this value each time the weight is updated, the weight change width ε is adjusted to an appropriate value. When monitoring using the scale G, especially when this value does not monotonically decrease but oscillates,
It is necessary to reduce the change width ε.

○ Scale Gl is also a scale that shows the progress of learning. In monitoring using the scale Gl.

Unlike the case of using the scale G, calculation is possible without an annealing procedure, but it requires a calculation amount on the order of 2v.

The V-th power O measure GB of 2v→2 is a measure of how close the actually occurring probability distribution is to the Boltzmann distribution. That is, it indicates whether the operating status is good or bad. When performing monitoring using the measure CB, a particularly large value indicates that the state transition deviates significantly from the Portsman distribution, so a gentler annealing schedule is required.

〔Example〕

11(A) and 11(B) are diagrams in separate locations in FIG. 1 illustrating one embodiment flowchart. The phase-- shown in FIG. 11(A) is replaced with the phase-- shown in FIG. 8(B) in relation to FIG. 8 described above.

Furthermore, FIG. 11(B) corresponds to the monitor unit 30 shown in FIG. 1, which calculates one scale G, Gl, or GB, and monitors the quality and progress of the operating status at each point in time. are doing. FIGS. 12(A) and 12(B) are diagrams together showing a flowchart of one embodiment.

The phase shown in FIG. 12(A) corresponds to the process of the phase shown in FIG. 8(B) in relation to FIG. 8 described above. Needless to say, Figure 12 (A
)(B) corresponds to the monitor unit 30 shown in FIG. 1, which calculates two scales G, Gl, or CB, and monitors the quality and progress of the operating status at each point in time. .

〔Effect of the invention〕

As described above, according to the present invention, it is possible to monitor the quality of the operation status and the progress status, and to perform high-speed learning while applying weight changes according to, for example, equation (7).

Furthermore, in the case of the embodiment shown in FIG. 11, the timing of monitoring and the setting of the number of times "a unit becomes 1" frequency measurement for probability estimation can be adjusted independently of the weight update procedure. In addition, in the case of the embodiment shown in FIG. 12, the operating status itself during one weight update can be measured,
It has the advantage of being able to share the annealing procedure with the weight update procedure.

[Brief explanation of the drawing]

Figure 1 is a diagram showing the principle configuration of the present invention, Figure 2 is a diagram showing the constituent units of the Portsman machine network, Figure 3 is a diagram listing the state vectors of all units, and Figure 4 is an abstraction of the external environment. Figure 5 shows the visible unit and hidden unit, Figure 6 shows the state vector of the visible unit, Figure 7 shows the two phases of learning, and Figure 8 shows the weight update procedure. 9 is a flowchart of a state transition procedure, FIG. 10 is a graph representing a sigmoid function, FIG. 11 is a flowchart of one embodiment, and FIG. 12 is a flowchart of another embodiment. In the figure, 10 is a network and 11 is a visible unit. The hidden unit 12 represents a learning processing section, and the hidden unit 30 represents a monitor section.

Claims (3)

[Claims] (1) Contains at least a visible unit (11) and a hidden unit (12), and each unit S_i and S_j
The network (10) is configured to be able to set the weight W_i_j between the visible units (11), and the pattern appearance probability P^+(V_α) corresponding to the learning pattern for setting/changing the state of the visible unit (11) is provided.
A learning processing unit (
In the learning processing method in the Boltzmann machine having the above-mentioned learning processing method, a monitor section (30) is provided, and the monitor section (30) has a function of at least monitoring the progress of the learning and the quality of the operation status. Characteristic learning processing method in Boltzmann machine. (2) The monitor unit (30) performs a Kull analysis of the appearance probability distribution P^+(V_α) of the pattern to be learned and the probability distribution P^−(V_α) in which it actually appears.
back-divergence▲There are mathematical formulas, chemical formulas, tables, etc.▼ (where V_α is the state vector of the visible unit) as the scale G
A learning processing method in a Boltzmann machine according to claim (1), characterized in that the learning processing method is used as a Boltzmann machine. (3) The monitor unit (30) calculates the Kullback-divergence ▲ between the appearance probability distribution P^+(V_α) of the pattern to be learned and the ideal value PI (W_i_j; V_α) of the Boltzmann distribution for the current weight. A learning processing method in a Boltzmann machine according to claim (1) or (2), characterized in that a mathematical formula, a chemical formula, a table, etc. ▼ is used as a scale GI. (4) The monitor unit (30) calculates the Kuhlback-divergence ▲ mathematical formula between the ideal Boltzmann distribution PI (W_i_j; U_β) at the current weight and the probability distribution P^-(U_β) of the actually generated pattern. There are chemical formulas, tables, etc. ▼ (However, U_β is the state vector of all units) as the scale GB
A learning processing method in a Boltzmann machine according to any one of claims (1) to (3), characterized in that the method is used as a learning processing method in a Boltzmann machine.