JPH10340308A - Neuro element - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a neuro element which improves reliability and enables high-speed operation. SOLUTION: Concerning the neuro element for constituting a neural network while using radial basic function, this element is provided with a two-output differential amplifier 3, a multiplier 5 for multiplying these two outputs and a variable voltage circuit 1 for outputting a prescribed voltage, an input to the virtual node of neural network is defined as one input of the differential amplifier 3 at least, and the voltage outputted by the variable voltage circuit 1 is added to the input.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural element for constructing a neural network imitating the nervous system of the human brain, and more particularly to a neural element which can be easily realized by a semiconductor integrated circuit.

[0002]

2. Description of the Related Art An information processing apparatus (neurocomputer) having a neural network imitating the nervous system of the human brain has been receiving attention. Such a neurocomputer can obtain an ambiguous solution in information processing that Neumann-type computers are not good at by changing the connection strength between many neurons according to a predetermined learning algorithm.

As a neural element required to realize a neural network as a device rather than a program, a neural element using an EEPROM has been proposed (Japanese Patent Laid-Open No. 3-144785). In this neuro element, each input signal is multiplied by a coefficient, the sum of the coefficients is input to a threshold function such as a sigmoid function, and the calculated value of the function is used as the output of the neuro element.

On the other hand, unlike a threshold function, a neural network using a radial basis function has been devised. However, as a configuration for realizing this kind of neural network by hardware, a Gaussian function is used for the radial network. Neural networks based on basis functions are known (“A Gauss
sian Synapse Circuit For Analog VLSI Neural Networ
ks "). FIG. 4A is a circuit diagram showing a configuration example of a neural element forming a neural network using the Gaussian function as a radial basis function.
It is a combination of a pair of differential amplifiers using an OS transistor and a current mirror circuit. Two MOS transistors M ₁ are connected to the drain of a MOS transistor M ₃ whose source is applied with the high-side power supply voltage V _DD. , M ₂
Source connected to the gate of the MOS transistor M ₂ is grounded via a capacitor C _1.

The drain of the MOS transistor M ₁ is connected to the drains of two MOS transistors M ₄ and M ₅ whose low-voltage-side power supply voltage V _SS is applied to the source, and the drain and gate of the MOS transistor M ₅ are connected. ing. The sources of the two MOS transistors M ₇ and M ₈ are connected to the drain of the MOS transistor M ₉ to which the high-side power supply voltage V _DD has been applied, and the MOS transistor M
Drain of ₈ is grounded through a capacitor C _2.

The drain of the MOS transistor M ₇ is connected to the drains of two MOS transistors M ₁₀ and M ₁₁ to which the low-voltage-side power supply voltage V _SS is applied to the source, and the drain and gate of the MOS transistor M ₁₁ are connected. ing. The drain of the MOS transistor M ₈ is connected to the drain of the MOS transistor M _1, the drain of the MOS transistor M ₂ is connected to the drain of the MOS transistor M _7.

[0007] The drain of the MOS transistor M ₁₅ which high-voltage power source voltage V _DD is applied to the source, the two MOS transistors low-voltage power source voltage V _SS is applied to the source M
_6, is connected to the drain of M _12, MOS transistor M
The drain of the ₆ to the drain of the MOS transistor M _5,
The gate of the MOS transistor M ₁₂ is connected to the gate of the MOS transistor M _11.

[0008] The drain of the high-pressure side MOS transistor M ₁₃ which supply voltage V _DD is applied to the source is connected to one of the constant current source I _X1, the other constant current source I _X1 is applied low voltage side power supply voltage V _SS Have been. The drain of the MOS transistor M ₁₃ is connected to the gates of the MOS transistors _{M 13, M 3, M 9} , M 15. Low-voltage power supply voltage V
The drain of the MOS transistor M _{14 which SS} is applied to the source is connected to one of the constant current source I _X2, the constant current source I _X2
The other is applied with the high-side power supply voltage V _DD . MOS
The drain of the transistor M ₁₄ is, MOS transistor M
₁₄ , M ₄ , and M ₁₀ .

The positive side of the input voltage V _IN as a pair of differential amplifiers is connected to the gate of the MOS transistor M ₁ , and the negative side is set to MO.
It is applied to the gate of the S transistor M _7. Input voltage V
Positive side of the reference voltage V _W of the _IN to the gate of the MOS transistor M _2, the negative side is applied to the gate of the MOS transistor M _8. Output current I _{OU T} as neuro device
It is obtained from the drain of the MOS transistor M _15.

In the neuro element having such a configuration, the total current I ₁ of the drain currents of the MOS transistors M ₁ and M ₈ according to the input voltage V _IN and the reference voltage V _W in the pair of differential amplifiers and the MOS The transistors M ₂ ,
Total current I ₂ of the drain current of M ₇ flows through the MOS transistors M _4, M ₅ and the MOS transistor M _11, M ₁₀ is a constant current source. Then, among the current flowing through the MOS transistor M ₁₅ of the upper current mirror circuit flows to the ground through the MOS transistor M _6, M ₁₂ the same amount of current as the current I _1, I ₂ is a constant current source,
The remaining current is output as the output current I _OUT .

In this circuit, as shown in equation (1), a synapse weight W _ji corresponding to a shift of the output Gaussian function characteristic with respect to the input voltage and a σ _ji for determining the standard deviation are respectively differentially calculated. Amplifier reference voltage V _W
And the change in transistor size (gate width / gate length) as shown in FIG. In equation (1), Y _j represents an output neuron, and X _j represents an input neuron.

[0012]

(Equation 1)

FIG. 5 (a) shows the new model shown in FIG. 4 (a).
Element, the reference voltage V in the differential amplifier_WTo
FIG. 5 is a graph showing input / output characteristics when changed.
Reference voltage V_WThe characteristic curve differs according to the height change
Input voltage V in a dynamic amplifier _INShift to higher or lower
You. FIG. 5B shows the neuro element shown in FIG.
Of the constant current source I of the current mirror circuit_X1, I_X2Current value
I_XIs a graph showing the input / output characteristics when
Constant current source I_X1, I_X2Current value I_XAccording to increase
Output current I_OUTAlso increase. These basis function characteristic songs
The change of the line causes the neural network
The weighting of the pulses becomes possible.

[0014]

Conventionally, as shown in FIG.
As shown in the figure, the constant current source I_X1,
I_X2Current value I_XOutput current I at the same ratio
_OUTHas not changed. In other words, when changing parameters
Output does not change linearly. This is the output current I _OUTBut
The difference between the current of the upper stage current mirror and the current of the differential amplifier
It is thought that it is determined by pulling, but like this
Characteristic shift is the basis in neural networks.
When performing product-sum operation of functions, the reliability of the output result is greatly increased
To lower.

The shift amount and amplitude with respect to the input in the radial basis function characteristics are determined by changes in the differential amplifier reference voltage V _W and the current values I _X of the constant current sources I _X1 and I _X2 , respectively. If, by controlling the current value I _X of the circuit outside the reference voltage V _W and a constant current source I _X1, I _X2, it is necessary to store the state, there is a problem that the signal processing time for the entire network increases. The present invention
The present invention has been made in view of such circumstances, and has as its object to provide a neuro element having high reliability and capable of high-speed operation.

[0016]

According to a first aspect of the present invention, there is provided a neural element for constructing a neural network using a radial basis function, wherein the differential element has two outputs and the two outputs are multiplied. A multiplier and a variable voltage circuit for outputting a predetermined voltage, wherein an input to a virtual node of the neural network is at least one input of the differential amplifier, and the voltage is added to the input. It is characterized by having.

The neuro element according to the second invention is characterized in that the variable voltage circuit is an emitter follower variable voltage circuit whose base voltage is controlled by a variable resistor and a constant current circuit.

A neuro element according to a third invention is characterized in that the load resistance of the differential amplifier is a variable resistor.

In the neuro element according to the present invention, in a neuro element composed of a differential amplifier and a multiplier for multiplying two outputs of the differential amplifier, a change in an output shift amount with respect to an input in a radial basis function characteristic is measured. This is realized by adding an emitter follower variable voltage circuit whose base voltage is controlled by a variable resistor and a constant current circuit in series with the input external input voltage of the differential amplifier. Also, by using a variable resistor for the load resistance of the differential amplifier,
The amount of voltage drop at the output node is changed, and the amplitude of the output characteristic is changed linearly with respect to the resistance value.

Thus, it is not necessary to control each current source and each voltage source outside the circuit and store the state of the entire network in order to change the amplitude in the radial basis function characteristic and the shift amount with respect to the input. Signal processing time is reduced. That is, each parameter of the radial basis function in the neural network in the learning process can be held by each variable resistor, so that control from outside the circuit becomes unnecessary, and the signal processing speed of the network is significantly improved.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings showing an embodiment. FIG. 1 is a circuit diagram showing a configuration of an embodiment of a neuro element according to the present invention.
This neuro element is a differential amplifier circuit 3 (differential amplifier)
And an input voltage circuit section 2, a reference voltage circuit section 4, a multiplier 5, and an emitter follower variable voltage circuit section 1 (variable voltage circuit).

The differential amplifier circuit 3 includes a transistor TR ₁
Is connected to the constant voltage source V _DD via one variable resistor VR _C (load resistance), and the emitter is connected to the constant current source _IEEE having one end grounded. The collector of the transistor TR ₂ is the other of the variable resistor VR
_It is connected to a constant voltage source V _DD via _C (load resistance), and the emitter is connected to a constant current source _IE . The characteristics of these two transistors TR ₁ and TR ₂ are the same. The variable resistors VR _C are variable resistors whose resistance can be adjusted according to an external control voltage (not shown).

The input voltage circuit section 2 is a closed circuit comprising an input voltage source V _IN as a time-varying input signal source and resistors R _B1 and R _B2 , and is connected to the positive terminal of the variable input voltage source V _IN. The resistor R _B1 is connected to the negative terminal and the resistor R _B2 is
_B1, the common connection point of R _B2 is connected to the base of the transistor TR _1. The emitter follower variable voltage circuit section 1
The collector of the transistor TR ₃ is connected to a constant voltage source V _DD, emitter, and one end of a resistor R _B3 other end of which is grounded,
It is connected to the negative terminal of the variable input voltage source V _IN . Further, a variable resistor VR _{B having} one end connected to the constant voltage source V _DD.
The other end of the (variable resistor), one end of the constant current source I _BB (constant current circuit) connected, the other end of the constant current source I _BB is grounded. The other end of the variable resistor VR _B is also connected to the base of the transistor TR _3. Variable resistor VR _B is a variable resistor capable of adjusting the resistance value in response to an external control voltage (not shown).

The reference voltage circuit section 4 is a closed circuit composed of a reference voltage source V _REF and resistors R _B1 and R _B2.
Resistor R _B1 to the positive terminal of _REF is, the resistance R _B2 is connected to the negative terminal, a common connection point of the resistors R _B1, R _B2 is connected to the base of the transistor TR _2. Also, the reference voltage source V
The negative terminal of the _REF is also connected to the positive terminal of the bias voltage source V _BS, the negative terminal of the bias voltage source V _BS is grounded. The multiplier 5 receives the collector voltages V ₁ and V ₂ of the transistors TR ₁ and TR ₂ individually.

In the neuro element having such a configuration, the equations (2) and (3) representing the characteristics of both transistors TR ₁ and TR ₂ are used.
And differential output currents I _C1 and I _C having sigmoid characteristics corresponding to the input voltage V _IN represented by equations (5) and (6) obtained by calculation from conditional equation (4) in differential amplifier circuit section 3. _C2
Flows.

[0026]

(Equation 2)

[0027]

(Equation 3)

Then, the differential output current I_C1, I_C2And variable resistor
Anti-arms VR_CDifferential output voltage V determined by the resistance value of₁, V_Two
Is multiplied by a multiplier 5 to obtain a final circuit output.
Thus, I having a radial basis function characteristic represented by the equation (7)_C1
・ I_C2Voltage V proportional to₁・ V _TwoIs output from the multiplier 5
Is done.

[0029]

(Equation 4)

Here, I _E1 and I _E2 are transistors T
R ₁ and TR _{2 represent} emitter currents, and V _E1 and V _E2 represent base-emitter voltages of transistors TR ₁ and TR ₂ .
Α is the base ground short-circuit current amplification factor of the transistors TR ₁ and TR ₂ , q is the amount of electron charge, k is the Boltzmann constant, T is the temperature, I _S is the transistors TR ₁ and TR 2.
₂ represents the reverse saturation current.

Here, in the learning process of the neural network, it is necessary to change the connection strength between the neurons in the network. In order to realize this by hardware, the radial basis which is the output of each neuron is required. The function characteristics are represented by Expression (8) and FIG.

[0032]

(Equation 5)

Here, V _OUT = V ₁ · V ₂ . At this time, it is necessary to change the output amplitude by the variable C _{k, the} half width by the variable β _k , and the shift amount with respect to the input by the variable θ _k . f [x (t)] represents the product sum of each neuron output in the network, and x (t) represents the input data of the network.

A method of realizing a change in the output shift amount with respect to the input and the output amplitude in the neuro element will be described below. As shown in FIG. 1, the constant voltage source V _DD side resistor VR _B in the emitter follower variable voltage circuit 1 added in series to the input voltage circuit 2 of the differential amplifier circuit 3, and the differential amplifier circuit 3 The two collector load resistors VR _C are variable resistors, and the amount of shift and the amplitude with respect to the input of the radial basis function characteristic are changed as shown in FIG. 2 by changing each resistance value. In particular, by changing the resistance value of the variable resistor VR _B, the input-side transistor TR ₁
Of the reference transistor TR ₂
, The equilibrium point of the differential amplifier circuit 3 moves by the offset, and the radial basis function output characteristic also shifts with respect to the input voltage V _IN .

At this time, the change in the resistance value of the variable resistor VR _B is 1 kΩ to 1 k in consideration of a practical circuit power supply voltage.
About 00 kΩ is appropriate. Further, by changing the resistance value of the variable resistor VR _C which is the collector load resistance, the voltage drop of the differential output voltages V ₁ and V ₂ changes, and the sigmoid characteristic and multiplication of the output of the differential amplifier circuit section 3 are obtained. The amplitude of the radial basis function characteristic of the output of the detector 5 changes. At this time, the resistance value of the variable resistor VR _C , which is the collector load resistance, is 1 in consideration of the amplification characteristics and the full swing of the element.
00Ω to 10 kΩ is appropriate.

[0036] FIG. 3 (a) is a graph showing input-output characteristics of the neuro device when changing the resistance value of the variable resistor VR _B to 30Keiomega～38keiomega. Where V _OUT =
V ₁ · V ₂ . The resistance value is 30 kΩ, 34 kΩ, 38
As the resistance increases, the output characteristic shifts with respect to the input voltage V _IN while maintaining the same function characteristic.
The shift amount changes linearly with the change in the resistance value. FIG. 3B shows that the resistance value of the variable resistor VR _C is 1 kΩ.
4 is a graph showing input / output characteristics of a neuro element when the voltage is changed to に 3 kΩ. Resistance value of 1 kΩ, 2 kΩ, 3 k
As the value increases, the amplitude of the characteristic curve is also doubled while maintaining the radial basis function characteristic represented by the equation (7).
It is three times, and changes linearly with the change in the resistance value of the variable resistor VR _C.

[0037]

According to the neuro elements according to the first and second aspects of the present invention, the variable voltage from the variable voltage circuit is added to the input voltage. The shift amount can be changed linearly.

According to the neuro element of the third aspect, the amplitude of the radial basis function characteristic, which is the output of the neuro element, with respect to the input voltage can be linearly changed by the variable load resistor of the differential amplifier. Since it is possible, the connection strength between each neuron in the network, which is necessary in the learning process of the neural network, can be accurately changed, and the reliability of the network output can be greatly improved. Therefore, since each parameter of the basis function can be changed by each built-in variable resistor, complete hardware realization of the neural network can be realized, and the signal processing speed is significantly increased.

[Brief description of the drawings]

FIG. 1 is a circuit diagram showing a configuration of an embodiment of a neuro element according to the present invention.

FIG. 2 is an explanatory diagram for explaining the influence of variables C _k and θ _{k on} input / output characteristics of a neuro element.

FIG. 3 is a graph showing input / output characteristics of a neuro element when a resistance value of a variable resistor is changed.

FIG. 4 is a circuit diagram showing a configuration example of a conventional neural element forming a neural network using a Gaussian function as a radial basis function.

FIG. 5 is a graph showing input / output characteristics of the neuro element shown in FIG.

[Explanation of symbols]

REFERENCE SIGNS LIST ₁ Emitter follower variable voltage circuit section (variable voltage circuit) 2 Input voltage circuit section 3 Differential amplifier circuit section (differential amplifier) 4 Reference voltage circuit section 5 Multiplier I _BB Constant current source (constant current circuit) TR _{1 to} TR ₃ Transistor VR _B variable resistor (variable resistor) VR _C variable resistor (load resistor, collector load resistor) V _IN input voltage source

Claims (3)

[Claims] 1. A neural element for forming a neural network using a radial basis function, comprising: a two-output differential amplifier; a multiplier for multiplying the two outputs;
A variable voltage circuit that outputs a predetermined voltage, wherein an input to a virtual node of the neural network is at least one input of the differential amplifier, and the voltage is added to the input. The characteristic neuro element. 2. The neural element according to claim 1, wherein said variable voltage circuit is an emitter follower variable voltage circuit whose base voltage is controlled by a variable resistor and a constant current circuit. 3. The neuro element according to claim 1, wherein the load resistance of the differential amplifier is a variable resistor.