JP3483154B2 - Pulse generator and signal processing device using the same - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pulse generator used for simulated annealing in a Boltzmann machine or simulated annealing in a pulse density type neural network, and a signal processor using the same.

[0002]

2. Description of the Related Art Conventionally, a back-propagation type learning method which is often used in a hierarchical neural network has a problem that it is trapped by a so-called local minimum (local solution). As a technique for avoiding this problem, for example, simulated annealing in a Boltzmann machine is known. For example, it is described in "Mathematics of Neurodynamics" in the Institute of Electronics, Information and Communication Engineers, Vol.73 No.2 pp.131-136.

[0003]

In order to put this technology into practical use, hardware is a necessary condition, but Boltzmann machines that perform stochastic operations and hardware that realizes simulated annealing that controls the probability are not. It is in a difficult situation.

[0004] Here, with respect to simulated annealing in the Boltzmann machine, it will be described with reference to FIG. 9. The output value of each neuron of the Boltzmann machine is
The value is limited to two values of “0” or “1”, and which value is obtained depends on the internal potential U of the neuron and the parameter T corresponding to the temperature. P (1) = 1 / {1 + exp (−U / T)} ............ (1) is probabilistically calculated (Equation (1) indicates the probability that the output will be "1").

Therefore, the output from each neuron element of the Boltzmann machine becomes a pulse train according to a certain probability.
As shown in FIG. 9 , this probability is such that the larger the internal potential U, the more easily "1" is output, and the smaller it is, the more easily "0" is output. However, when the parameter T is large, the probability is 1 regardless of the internal potential U as shown in the figure.
It approaches / 2. That is, the output value of each neuron is random, and the system is in a random state. On the other hand, when the parameter T is small, nonlinearity close to a step function is exhibited and the system is in a stable state.

Simulated annealing involves increasing the temperature parameter T from a large value to a small value, that is, a system, in the same way that the actual annealing process gradually lowers the temperature from a high temperature state to a stable state at a low temperature. It is a transition from a random state to a stable state.

Therefore, in order to perform the simulated annealing, it is possible to probabilistically determine whether the output is “0” or “1” as in the equation (1), and the parameter T → large , Probability → 1/2, a temperature parameter T is required.

In view of the above, the present invention provides a pulse generator that satisfies the above conditions and can realize simulated annealing in a Boltzmann machine or the like with a simple circuit configuration, and a signal processing apparatus using the same. With the goal.

[0009]

According to a first aspect of the present invention, a plurality of first pulse generating means for generating a pulse and a pulse whose occurrence probability is variably controllable and its output are generated.
Boltzmann's simulated annealing
A common second pulse generating means corresponding to the temperature parameter of the ring, an individual pulse output from the first pulse generating means, and a pulse output from the second pulse generating means are independently logically operated. And a plurality of logical operation means for outputting a pulse.

According to the second aspect of the present invention, a plurality of first pulse generating means for generating a pulse and a pulse whose pulse width is variably controllable are generated, and the pulse width is determined by the Boltzmann system.
Corresponding to temperature parameter of simulated annealing
The common second pulse generating means, the individual pulse output from the first pulse generating means and the pulse output from the second pulse generating means are independently logically operated to output the pulse. It is composed of a plurality of logic operation means.

According to the invention described in claim 1 or 2 , the logic operation means is, in the invention described in claim 3 , an exclusive OR circuit for calculating an exclusive OR of pulses,
The invention according to claim 4 is a logical product circuit for calculating a logical product between pulses, and the invention according to claim 5 is a logical sum circuit for calculating a logical sum between pulses.

On the other hand, in the invention according to claim 6 , a plurality of nerve cell mimicking elements that output a single pulse train signal by performing arithmetic processing with a plurality of pulse train signals as inputs and together with a coupling coefficient represented by a pulse train are variable. In a signal processing device mutually coupled by a coupling means having a coupling coefficient, each of the nerve cell mimicking elements is provided with a first pulse generating means for generating a pulse train signal, and a pulse train signal whose occurrence probability is variably controllable. The probability that the output will be 1
Boltzmann simulated annealing temperature parameter
It is formed by logical operation means for performing a logical operation on the pulse train signal from the second pulse generator common to the meters and the pulse train signal from the first pulse generator.

Similarly, in the invention according to claim 7 , each nerve cell mimicking element generates a pulse train signal for generating a pulse train signal, and a pulse train signal whose generation probability is variably controllable and its pulse width. The Boltzmann simulation
It is formed by a logical operation means for performing a logical operation of a plurality of pulse train signals from a plurality of common second pulse generators corresponding to the temperature parameter of the door annealing and a pulse train signal from the first pulse generator.

[0014]

According to the first or second aspect of the present invention, the plurality of pulses generated by the plurality of first pulse generating means are logically operated with the pulse whose generation probability or pulse width is variably controlled by the second pulse generating means. By doing so, it is possible to obtain multiple pulses with different pulse generation probabilities under the same conditions ,
By be regarded as the temperature parameters of the second Boltzmann simulated annealing the probability of occurrence probability variable pulse by the pulse generating means, thus capable of a temperature of the entire system constant.

In the invention of claim 3, 4 or 5 , since the logical operation means is constituted by an exclusive OR circuit, an AND circuit or an OR circuit, the logical operation suitable for each purpose and use is provided. The resulting pulse can be obtained.

Further, in the invention according to claim 6 or 7 , a nerve cell mimicking element of the signal processing device having a pulse density type neural network configuration is claimed.
Since it is formed by using the pulse generator according to the invention described in 1 or 2, it becomes possible to control the activity of the entire signal processing device, it is less likely to be caught by a local solution, and more complicated control is possible. Becomes

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first configuration example which is a premise of the present invention will be described with reference to FIGS . Also not a pulse generating circuit (first pulse generating means) 1 for generating a pulse is provided at a certain probability that is determined in advance. Further, a pulse generation circuit (second pulse generation means) 2 for generating a pulse having a variable generation probability is provided. Such a pulse generating circuit 2 is, for example, as shown in FIG. 2, an 8-bit linear feedback known as a pseudo-random number generating device that randomly generates an integer P of "1" to "255" every time 1 bit is shifted. The shift register 3, an integer Q (Q0 to Q7) from "0" to "255" input from the outside, and an integer P from the linear feedback shift register 3
(P0 to P7), and a comparator 4 that compares the magnitudes. Here, when P <Q, "1" is output. For example, if "64" is set as the integer Q, a pulse train having a probability density of 64/255 is generated. That is, the probability control can be performed by the value of the integer Q.

The pulses output from the pulse generating circuits 1 and 2 are input to a logical operation circuit (logical operation means) 5 to perform a predetermined logical operation, and finally output as one output pulse. It

Here, as the logical operation circuit 5, for example, as shown in FIG. 3, an exclusive OR circuit 6 (which corresponds to the invention according to claim 3 ) for taking an exclusive OR of pulses.
Alternatively, a logical product circuit that obtains a logical product of the pulses (claim
4 ) (not particularly shown), or a logical sum circuit for taking the logical sum of pulses (corresponding to the invention of claim 5 ... not particularly shown), etc. It may be appropriately selected depending on the effect of the logical operation of (1) and the purpose of the pulse to be output. The same applies to the logical operation circuit 5 in each embodiment described below.

In such a configuration, assuming that the output of the pulse generating circuit 1 is P, the probability of P = 1 is p, the output of the pulse generating circuit 2 is C, and the probability of C = 1 is q,
An exclusive OR of P and C by the exclusive OR circuit 6,
The probabilities of the combinations of P and C outputs are shown in Table 1 and Table 2, respectively.

[0021]

[Table 1]

[0022]

[Table 2]

At this time, the probability E that the output pulse O of the exclusive OR circuit 6 becomes O = 1 is as shown in FIG. That is, by controlling the probability q that C = 1, O = 1
It can be seen that the probability E of In particular, q =
When it is 0.5, the probability E of O = 1 is 1/2 regardless of the probability p. As a result, when the temperature parameter is increased in the Boltzmann machine described above, the output becomes 1
It is possible to realize the property that the probability of becoming asymptotic to 1/2 (see FIG. 9 ). On the other hand, when q = 0, E = p, and
The property of the probability p appears as it is. For example, if there is a non-linear relationship between p and the internal potential of a neuron element in a neural network, the non-linearity is directly E.
Will appear in.

Therefore, according to this configuration example, FIG.
Simulated annealing can be realized by making the probability q that the output C of the pulse generation circuit 2 becomes C = 1 correspond to the temperature parameter with an extremely simple circuit configuration as shown in FIG.

Next, a first embodiment of the present invention will be described with reference to FIG. The same parts as the parts shown in the above-mentioned configuration example are denoted by the same reference numerals (the same applies to the following embodiments).
This embodiment corresponds to the invention described in claim 1 and is provided with a plurality of pulse generating circuits 1a and 1b, for example, two pulse generating circuits 1a and 1b, each of which generates a pulse having a predetermined probability. Logical operation circuits 5a and 5b for independently performing logical operations (exclusive OR) between the pulses P ₁ and P ₂ output from the pulse generator ₁ and the control pulse C of the pulse generation circuit 2 are provided, and output pulses O ₁ and O ₂ is obtained.

According to this structure, the probability q that control pulse C of the pulse generation circuit 2 becomes C = 1 by variably controlling, similarly to the configuration example, the output pulse O _1,
The O ₂ pulse appearance probability can be controlled. In particular,
Since the logical operation is performed with the common control pulse C for the pulses P ₁ and P ₂ , when the probability q that C = 1 is considered as a temperature parameter, the output pulses O ₁ and O ₂ are the same. Probability is changed under the condition (same temperature),
This is equivalent to making the temperature of the entire system constant.

In the present embodiment, for simplification of description, an example in which the number of pulse generation circuits that output pulses with a certain probability is two has been described, but of course, there are three or more pulse generation circuits. Alternatively, each pulse signal may be a common signal C
If the logical operations are independently performed, the same temperature control can be performed.

[0028] Next will be described a second embodiment of the present invention by FIGS. In this embodiment , a pulse generating circuit (pulse generating means) 7 capable of variably controlling the pulse width is provided instead of the pulse generating circuit 2 shown in FIG. This pulse generation circuit 7 is, for example, as shown in FIG. 7 , an 8-bit counter 8 that adds and counts clock pulse signals.
And a comparator 9 which compares the integer Q (Q0 to Q7) from “0” to “255” input from the outside with the integer P (P0 to P7) output from the counter 8.
Composed of and. Here, when P <Q, "1" is output. For example, if "64" is set as the integer Q, a pulse train having a pulse width of 64/255 is generated. That is, depending on the value of the integer Q, the control pulse C
It is possible to control the pulse width of the output pulse.

Also in this embodiment, since the control pulse C from the pulse generating circuit 7 and the pulse P from the pulse generating circuit 1 are logically operated to obtain the output pulse O, The pulse generation probability of the target pulse P can be easily controlled only by controlling the pulse width of the pulse generation circuit 7.

[0030] and those regarding the pulse generating circuit 7 in such a pulse width variable, the use of the pulse generating circuit 7 in place of the pulse generating circuit 2 shown in FIG. 5 corresponds to the invention of claim 2, wherein Become. Also in this case, the same effect as in the case of the first embodiment can be obtained.

Furthermore, explaining the third embodiment of the present invention by FIG. This embodiment corresponds to the invention according to claim 6 , and the pulse generation probability control method according to the first embodiment described above is applied to each neural cell mimicking element constituting a pulse density neural network (signal processing device), that is, , Which is used for the configuration of neuron elements. That is, one neuron element 10a includes the pulse generator 1a and the logical operation circuit 5
a and another neuron element 10b formed by
Is formed by a pulse generator 1b and a logical operation circuit 5b, and is configured so that the control pulse train signal from the pulse generation probability variable type pulse generation circuit 2 is given to each logical operation circuit 5a, 5b.

Here, the pulse density type neural network has a variable coupling coefficient for a large number of neuron elements 10 which outputs a single pulse train signal by performing arithmetic processing with a plurality of pulse train signals as input and a coupling coefficient represented by a pulse train. They are connected to each other by a connecting means provided with, and are known as shown in, for example, JP-A-4-549 and JP-A-4-111185, and the details thereof will be omitted.

Therefore, according to the present embodiment, the activity of the entire network can be controlled by the pulse generating circuit 2, so that it is less likely to be caught by the local solution.

A pulse width variable type pulse generating circuit 7 may be used in place of the pulse generating circuit 2 in FIG. 8 (corresponding to the invention of claim 7 ).

[0035]

Effects of the Invention According to the invention of claim 1, wherein, the second and the plurality of first pulse generating means for generating a pulse, the occurrence probability or pulse width to generate a variable controllable pulse
A plurality of logical operations for independently performing a logical operation on each of the pulse generating means, the individual pulse output from the first pulse generating means and the pulse output from the second pulse generating means to output the pulse. Since a plurality of pulses having different pulse occurrence probabilities can be obtained under the same condition, the probability of the pulse having a variable occurrence probability by the second pulse generating means is calculated by Boltzmann's simulated annealing.
It is possible to make the temperature of the whole system constant by considering it as the temperature parameter of the group.

In the invention according to claim 1 or 2 , the logical operation means is, in the invention according to claim 3 , an exclusive OR circuit for calculating an exclusive OR of pulses,
In the invention according to claim 4, a logical product circuit for calculating a logical product of the pulses is used, and in the invention according to claim 5 , a logical sum circuit for calculating a logical sum of the pulses is used. It is possible to obtain a pulse of a suitable logical operation result.

On the other hand, in the inventions according to claims 6 and 7 , a large number of nerve cell mimicking elements that output a single pulse train signal by performing arithmetic processing with a plurality of pulse train signals as input and with a coupling coefficient represented by a pulse train are freely changeable. In the signal processing device, which is mutually coupled by the coupling means having the coupling coefficient, the neuron-mimicking elements are controlled by the first pulse generating means for generating a pulse train signal and the generation probability or pulse width is variably controlled. Generates a free pulse train signal and outputs it
Boltzmann simulation of the probability or pulse width of 1
The pulse train signal from the common second pulse generating means corresponding to the temperature parameter of the Ted annealing and the first
Since it is formed by the logic operation means for performing a logic operation with the pulse train signal from the pulse generation means, it is possible to control the activity of the entire signal processing device and it is less likely to be caught by the local solution.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first configuration example which is a premise of the present invention.

FIG. 2 is a block diagram showing a configuration of a pulse generation circuit whose pulse generation probability is variable.

FIG. 3 is a block diagram showing a specific configuration example.

FIG. 4 is a graph showing probability characteristics.

FIG. 5 is a block diagram showing a first embodiment of the present invention.

FIG. 6 is a block diagram showing a second embodiment of the present invention.

FIG. 7 is a block diagram showing a configuration of a pulse generation circuit having a variable pulse width.

FIG. 8 is a block diagram showing a third embodiment of the present invention.

FIG. 9 is a graph showing stochastic characteristics for explaining simulated annealing in a Boltzmann machine.

[Explanation of symbols]

1, 1a, 1b First pulse generating means 2, 2a, 2b Second pulse generating means 5, 5a, 5b logical operation means 6 Exclusive OR circuit 10a, 10b nerve cell mimicking element

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-53-138669 (JP, A) JP-A-3-226002 (JP, A) JP-A-59-13415 (JP, A) JP-A-4- 549 (JP, A) JP 4-111185 (JP, A) JP 5-276142 (JP, A) Actual Kai 3-34326 (JP, U) Uesaka Yoshinori, Neurodynamics Mathematics, Electronic Information Journal of the Institute of Communications, Japan, The Institute of Electronics, Information and Communication Engineers, February 1990, Vol. 73, No. 2,131〜136 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06G 7/60 G06F 15/18 H03K 3/84

Claims (7)

(57) [Claims] 1. A a plurality of first pulse generating means for generating a pulse, the occurrence probability is variably controlled freely pulse occurs
The probability that the output is 1 is calculated by Boltzmann's simulation
A common second pulse generating means corresponding to the temperature parameter of the door annealing, individual pulses output from the first pulse generating means and a pulse output from the second pulse generating means are independently logically logic. A pulse generator comprising a plurality of logical operation means for performing an operation and outputting a pulse. Wherein a plurality of first pulse generating means for generating a pulse, the pulse width is variable controllable pulse occurs
The pulse width is calculated by Boltzmann's simulated annealing.
Common second pulse generating means corresponding to the temperature parameter of the ring, individual pulses output from the first pulse generating means, and the pulse output from the second pulse generating means are independently logically operated. And a pulse generator for outputting a pulse. Wherein the logical operation means, the pulse generator of claim 1, wherein it has an exclusive OR circuit for calculating an exclusive OR of the pulse together. The 4. A logical operation unit, also claim 1, characterized in that a logical product circuit for calculating a logical product of the pulse between
Is the pulse generator described in 2 . 5. A logical operation means, also claim 1, characterized in that the logical OR circuit for calculating the logical sum of the pulse between
Is the pulse generator described in 2 . 6. A coupling means having a variable coupling coefficient for a plurality of nerve cell mimicking elements, which receives a plurality of pulse train signals as input and performs arithmetic processing with a coupling coefficient represented by a pulse train to output a single pulse train signal. in the signal processing apparatus coupled to each other by said each neuron mimetic device, respectively a first pulse generating means for generating a pulse train signal, the probability is generated a variable controllable pulse train signal Resona
Boltzmann's simulated probability that the output will be 1
Common second corresponding to the temperature parameter of annealing
And a pulse train signal from the first pulse generator and a logical operation unit that performs a logical operation on the pulse train signal from the first pulse generator. 7. A coupling means having a variable coupling coefficient for a plurality of neuronal cell mimicking elements, which receives a plurality of pulse train signals as input and performs arithmetic processing with a coupling coefficient represented by a pulse train to output a single pulse train signal. generated in the signal processing device coupled to each other, each of said neuron mimetic device, a first pulse generating means for generating a respective pulse train signal, the pulse width is variable controllable pulse train signal by perilla
The pulse width of Boltzmann's simulated annealing
Signal formed by a common pulse train signal from the second pulse generator and the logical train means that performs a logical operation on the pulse train signal from the first pulse generator. Processing equipment.