JPH0318985A - Information processor - Google Patents

Abstract

PURPOSE:To reduce the number of elements, to simplify the manufacturing process and to execute the signal processing at a high speed by using a source - drain resistance of a transistor having a charge accumulation gate as a coupling coefficient, and varying the coupling coefficient by an electric signal application time. CONSTITUTION:A sigmoid function generator 101 shown by a voltage driver and a sigmoid function generator 101 shown by a current amplifier correspond to an intermediate layer and an output layer, respectively. A charge accumulation gate type transistor (MNOS element) 11 is a coupling coefficient Wij for connecting NMj and NOi, and the coupling coefficient is constituted of one piece of element by using the source - drain impedance of the transistor. At the time of updating the coupling coefficient, a pulse voltage having pulse width of length being proportional to the necessary change quantity of the coupling coefficient is applied to a control electrode 113 by a voltage/ time converter 13. In this case, whether the coefficient increases or decreases is determined by the polarity of the MNOS element and positive/negative of the pulse, and both positive and negative pulses are used properly. In such a way, the circuit and the manufacturing process can be simplified remarkably, and a high speed operation can be realized.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an information processing device, and more particularly to an information processing device in which a neural network LSI is configured using analog circuits.

Neural network to neural network.

Artificial neural networks (artificial neural networks) attempt to efficiently solve recognition and judgment problems that involve ambiguity and empirical rules using electrical circuits that mimic the neural structure of living organisms. In recent years, there has been an increasing demand for the implementation of neural network LSIs (or neurochips) that implement this on silicon chips using analog or digital methods.

[Conventional technology]

Publicly known examples related to this invention include J.P. Sage
* Thompson and R,S, 11th
ers rAN ARTIFICIALNHLIRAL
NETVORK INTEGRATED CI
RCUIT BASED ONMNO5/CCD
PRINCIPLESJ Digest of
Technical Papers on the Co
nference on Naural Netw
Orksin Computing (Apr, 8
6) ppa81-385 (by Amari
can Institute of Physics)
An example is J.

This known example is an example of an LSI configuration method using analog MO8 technology. The size of the output signal of each neuron is determined by the CCD
The coupling coefficient is expressed as the amount of charge stored in the above, and the coupling coefficient is expressed as the amount of charge stored in the MNO8 element used in EHFROM, etc.
FIG. 11 shows the structure of an MNO8 element. The MNO8 element is a transistor having a gate that can store charge in a non-volatile manner. Charge is accumulated in the nitride layer 114. Accumulated charges (cross symbols in the figure) are put in and taken out by applying a voltage to the control electrode 113. The value of the coupling coefficient is given by its #I accumulated charge amount. The charge amount is read/written as an analog amount using charge transfer and read/write technology using a CCD.
An OD gate is provided, and reading and writing are performed by sequentially intervening the OD gates. That is, a total of three elements, one MNO8 element and two COD elements on both sides, constitute one coupling coefficient.

In addition to the MNO5 element, there are floating gate transistors and the like as transistors having a gate that can nonvolatilely store charge. A floating gate transistor has a floating gate instead of the nitride layer.

Here, we will introduce a hierarchical neural network, which is a common type of neural network, for reference. Further, detailed terms and symbols are defined using FIG. 4, and the symbols in FIG. 4 are used throughout this specification.

FIG. 4 shows the basic configuration of a hierarchical neural network.The operation thereof will be explained below by taking a backpropagation neural network as an example.

At the left end is an input layer consisting of neurons NIk (k = 1 to n), and in the middle is a neuron NMa (j = 1 to m
), with a neuron N O5 (i=1
There is an output layer consisting of ~L). When this circuit is given a set of input signals Xx=X (hereinafter referred to as input vector X), it generates a corresponding set of high-power signals Y1 to YL (hereinafter referred to as output vector Y). The set of teacher signals T1 to TL (hereinafter referred to as the teacher vector T) is the ideal value (model) that the output vector Y should take for the input vector X, and the output of the intermediate layer is expressed as the intermediate layer vector Z (Zx=Z The coupling coefficient connecting the input layer neuron NIh to the hidden layer neuron NM- is called m J h s from the hidden layer neuron NMj to the output layer neuron NO! Let W s a be the coupling coefficient connecting . The input vector X is output as is as the output X+-(k=1 to n) of the input layer. Here vectors X, Y,
Each component of Z and T and the coupling coefficient W tmah
The size of each component is 1 usually 0 (completely off)
to 1 (completely on).

The intermediate layer vector 2 is given by the following equation.

Z−=f(Σ mJh* xm)
...o) k=1 Here, f is the input/output function of the neuron shown in the upper part of FIG. Usually, a sigmoid function in the shape of the curve shown in the figure is used. Further, the output vector Y is given by the following equation.

Yi=f(Σ W i a umbrella ZJ)
−<2) j=square, O≦Xh, Yi* Za+ mah+ W*
As shown above, a≦1, the coupling coefficient m Jh I W
By varying the set of IJ values, different input/output conversion characteristics can be obtained with the same neural network.

In other words, it is possible to realize a recognition machine having various different functions.

Now, a method called ``learning'' has been used to determine the value of the coupling coefficient so as to obtain an output vector converted from the input vector according to the desired conversion rule. One of the methods is the backpropagation method.

First, a plurality of sets of input vectors X having different desired teacher vectors T are prepared. The coupling coefficient is determined so that the output vector Y (Ys to YL) and the teacher vector T (Tl to T L ) match. Usually, a random number is given as the initial value of each coefficient.Then, the coupling coefficient is iteratively modified using the recurrence formula below.The details are omitted and the results are shown.The coupling coefficient at a certain point in the iterative calculation. The amount of one-time correction to be added to WIJ and m a b is ΔW I J , 6m, respectively.
Let Jh be expressed by a linear equation.

Δw*a='? ”(Ti Y arc) Umbrella Y, fist (1-
Yl)"ZJ...(3) i=1 喰'NtJ*Z4 Umbrella (1-Z a) Skewer Xh Co-
(4) i = 1 umbrella Y place, - (i - ZJ) umbrella xhl ... (5
) where YIJ=w-21 (this is the output Yi when the input is only 21), η=constant.

Here, a value of about 0.1 to 0.3 is usually used for η,
Prevent divergence due to overcorrection.

By repeating the above corrections, you can get Y for any of multiple examples.
This is repeated until T becomes approximately equal to T. Once this is completed, this circuit has learned a certain conversion rule. As a result, it is possible to output an almost valid analogy value even for an unknown input vector.

[Problem to be solved by the invention]

The first drawback of the above-mentioned known method is that three elements are required for one coupling coefficient. Second, since charges are used as signals, it is necessary to form three types of element structures on the same chip: an MNO8 element, a CCD element, and a normal MO8 element. Therefore, there is a drawback that the LSI manufacturing process is extremely 111M. Thirdly, MN
Since signal processing is performed by sequentially transferring charges using the OS and CCD, the operation speed is low.

In addition to the publicly known side, neural network L
There are also ways to configure SI. The problem with the digital method is that it requires 5 to 10 times more elements than the analog method.

Therefore, an object of the present invention is to realize an information processing device using a neural network LSI that has a small number of elements, a simple manufacturing process, and a high signal processing speed.

[Means to solve the problem]

In order to achieve the above object, the present invention first uses the source-drain resistance of a transistor having a charge storage gate as a coupling coefficient, thereby realizing one coupling coefficient with only one transistor.

Second, it eliminates the need for a CGDI element and uses only two types of LSI elements: a transistor with a charge storage gate and an ordinary MO8 element.
Enables the manufacturing process to be configured above. Thirdly, it is characterized by realizing high-speed operation by using voltage and current as signals.

[Effect]

The drain-source conductance GDS of a transistor having a gate that can nonvolatilely store charge changes depending on the charge stored in the charge storage gate. At this time, the amount of change in the conductance Gos is approximately proportional to the length of time for which the control voltage is applied, and there is a relationship λ. Further, it is possible to freely select increase or decrease of Gos depending on the positive or negative value of the control voltage. Focusing on this, the above Gos is used as a coupling coefficient. To update the coupling coefficient, a pulse voltage having a pulse width proportional to the required change amount of the coupling coefficient is applied to the control electrode. At this time, whether the coefficient increases or decreases is determined by the polarity of the MNO8 elements and the positive/negative pulse, and both positive and negative pulses are used properly.

Determination of the required variation of the coupling coefficient is performed using an analog circuit or the like.

The present invention will be explained in detail below with reference to Examples.

〔Example〕

FIG. 1 is a partial diagram (conceptual diagram) of the first embodiment of the present invention. Of the structure of FIG. 4, a set of an intermediate layer, an output layer, and a coupling coefficient W1- therebetween is shown. .

The sigmoid function generator 101 shown as a voltage driver is
Corresponds to the middle class. Here, m voltage drivers are included, and the j#th voltage driver shown in the 0 diagram is the middle layer neuron NM-. The output is the hidden layer vector 2
The sigmoid function generator 101, which is the j-th element Z1 of . . . , corresponds to the output layer. Here is L
The first current amplifier shown in the diagram, which includes 1 current amplifiers, corresponds to the output layer neuron N Oi.

The output is the first element Y1 of the output vector Y.

A charge storage gate type transistor (here, an MNO5 element) 11 is a coupling coefficient wiJ that connects NM- and N01.

The coupling coefficient is constructed using one element by using the impedance between the source and drain of the transistor.

All connections over j=1 to m and i=1 to L are configured by a connection coefficient matrix 23. Control electrode 1 of each transistor
13 are connected by wiring shown by dotted lines in the figure, and a voltage is applied to the control electrode of a desired transistor using an address decoder (not shown). Here, as a conceptual diagram of the connection state when updating wl-, the connection state in which the transistor 11 of number (iy J) is addressed and the control voltage is applied is shown in a simulated manner using solid thin arrows. There is.

The value that output Y1 should take is JI T of the teacher vector.
* is given. From the difference between Yi and T, wll
The circuit that calculates the necessary correction amount Δw cliff 1 is the W t J correction amount determination circuit 104 . The calculation method for determining ΔWIJ uses Equation (3) in [Prior Art]. The circuit configuration will be described later with reference to FIG. The output of 104 is a voltage corresponding to the magnitude of Δw1-. The voltage/time converter 13 is a circuit that generates a pulse control voltage having a pulse width τ proportional to this voltage. The pulse width τ and the pulse voltage value are determined so that when the output pulse No. 13 is applied to the control electrode 113 of the transistor 11, the coupling coefficient changes by exactly ΔW r a. Addressing is performed by sequentially advancing i and j, and the coupling coefficient matrix 23 is updated.

A more detailed explanation will be given below with reference to FIG.

FIG. 2 is a partial view of the first embodiment of the present invention.

The amplifier symbols in FIG. 6, which correspond to the specific circuit in FIG. 1, are 1 for the sigmoid function generator in the intermediate layer, and 2 for the sigmoid function generator in the output layer. These do not need to generate a strict sigmoid function and can be achieved with a normal analog amplifier.

The input impedance of these amplifiers is selected to be smaller than the Gos of the charge storage gate type transistor (MNO8I!4 transistors) 11.

Hereinafter, the operation of the circuit will be described for the state in which the (x*J)th coupling coefficient is updated. W i shown earlier in Figure 1
The J correction amount determination circuit 104 connects three connection points 201,
202 and 203 are connected to ZJ, yl, and T.

This connection point is sequentially switched as the value of (le j) changes. The circuit shown in the figure and the formula (3) of [prior art]
The correspondence with this is described below.

First, the subtracter 14 calculates the difference between Yl and T1 taken in from the connection points 201 and 202, and this is multiplied by Y. Also,)
A difference is created by subtracting Yi from the reference voltage vM (corresponding to logic 1) on the light side, and this is further multiplied.

Next, Z- taken in from the connection point 203 is further multiplied. This calculation result is (Ts Yi) umbrella Y, fist (1-Yt
) Kushi Za. If the proportionality coefficient η is adjusted by the gain of the amplifier, this directly shows equation (3), and ΔW t a
is the voltage corresponding to This voltage is converted to voltage/time converter 1
3, it is converted into a control voltage pulse having a desired pulse width τ. Only one pulse is generated. W I J this
is applied to the control electrode 113 of the MNO5 element 11 forming the .

This updating circuit does not switch the connection according to the order of the values of -1*J, but the above-mentioned wtjtl positive amount determining circuit may be provided for each of l+J and updating may be performed at the same time.

FIG. 3 is an overall view of the first embodiment of the present invention.

The coupling coefficient ml connecting the input layer (X1-Xn) and the intermediate layer (Zl-Z-), and the coupling coefficient ml connecting the intermediate layer and the output layer (Y1 to Y+
, ) is provided with a matrix circuit 23 of coupling coefficients W I J. On the outside thereof, mJk and W I J update circuits are provided. The circuit for generating Δw1j is the same as the second time. Generation of Δml (formula (5) of [prior art])
) requires the output Yia when there are only 25 hidden layer vectors, and it is necessary to add the whole with respect to i. This procedure is as follows: #First, from 21 to 2 in the figure. Turn on all the switches shown in Figure 2, and update ΔW i a as described in Figure 2.
While the value of 1J is stored in the sample and hold circuit, only Za is turned on and the others are turned off. Since the output signal Y at this time is YIJ, the analog circuit section shown in the figure is operated in this state. Transfer the result to accumulator A
.

Store in Acc21. The accumulator may be an analog system or a digital system with AD and DA conversion added, and it accumulates from i 1 to L to obtain a value of 4m Jh. Apply this to a voltage/time converter, and
Only one pulse proportional to m is generated. This is MN
This is applied to the O8 element to update m ah , thereby sequentially switching addresses i, j, and learning.

According to this embodiment, a high-speed analog neural network LSI can be constructed with a simple circuit configuration and manufacturing process.

FIG. 6 is a partial view of a second embodiment of the present invention.

The calculation circuit for the yi umbrella (vi<-Yt) in FIG. 2 is simplified using an element circuit named a delta circuit.

The expression Yl'' (VM-Yl) is a simple parabolic function regarding Yi. Its functional form is expressed as a(yi).

(Here, vM corresponds to l in principle), a(yi)
The shape of is an upwardly convex parabola, and the value is 0 when Yi is O and 1.
becomes. Also, it takes the maximum value when yt is 0.5++
Yt varies in the range from O to 1 (see appendix). Therefore, G can be approximated by creating an upwardly convex function that approaches 0 when Y is close to 0 or 1. It is a delta circuit.

FIG. 5 is an elemental circuit diagram of a second embodiment of the present invention. This shows how to configure a delta circuit.

(a) and (b) are principle diagrams. (a) shows the input/output characteristics Vs and v of two amplifiers with slightly different reference voltages.
The single-amplifier shown in the conceptual diagram of No. 2 is of a low gain type to reduce the slope of the characteristic, and the low level and high level of the output are equal, and their values are Vi and VW. (b) Vi-V
A conceptual diagram of the characteristics of z is shown.

Due to the saturation characteristics of the amplifier, the input is 0 and 1 as shown in (b).
When the output is close to , the output becomes almost 0. Characteristics similar to those of G(Yl) can be obtained. A transconductance amplifier is used, the circuit diagram of which is shown in (C). (d) shows a delta circuit.

FIG. 7 is an overall view of the second embodiment of the present invention.

Similarly, the circuit is simplified using a delta circuit.

According to this embodiment, the same effects as in the first embodiment can be obtained with a simplified circuit.

FIGS. 8 and 9 are a partial view and an overall view of a third embodiment of the present invention. Here, the G(Yl) circuit is completely omitted to further simplify the circuit. G(Yl
) does not change much when Yl is near 0.5, so a zero-wire configuration in which a(Vt) is fixed at a constant value (=1) provides a more simplified structure, although the accuracy is slightly lower. is obtained.

FIG. 10 is an overall view of the fourth embodiment of the present invention.

Here, 21 more umbrellas (-ZJ in ■) are added from the circuit in Figure 9.
This is the result of removing the operation. The subtracter 14 that performs the VM-Z- calculation is removed. Since the ZJ term is already included in Y I J, a 0-line configuration in which this term is removed using a divider also results in a more simplified structure, although the accuracy is slightly lower.

〔Effect of the invention〕

As described above, according to the present invention, firstly, one coupling coefficient can be realized with only one transistor having a charge storage gate, and the circuit can be greatly simplified.

Second, a transistor with a charge storage gate and a normal M
Since only two types of O8 elements need be constructed on the LSI, the manufacturing process can be greatly simplified.

Third, high-speed operation can be achieved by using voltage and current as signals. It has these great effects.

[Brief explanation of the drawing]

Figure 1 is a partial diagram (conceptual diagram) of the first embodiment of the present invention;
The figure is a partial diagram of the first embodiment of the present invention, and Figure 3 is a partial diagram of the first embodiment of the present invention.
4 is a basic ellipse diagram of the hierarchical neural network, FIG. 5 is an elemental circuit diagram of the second embodiment of the present invention, ((a) and (b) are principle diagrams, ( (c) is a circuit diagram of a transconductance amplifier, (d) is a circuit diagram of a delta circuit), FIG. 6 is a partial diagram of the second embodiment of the present invention, and FIG. 7 is an overall diagram of the second embodiment of the present invention. , FIG. 8 is a partial diagram of the third embodiment of the present invention, and FIG. 9 is a complete diagram of the third embodiment of the present invention.
- Sigmoid function generator in intermediate layer, 2... Sigmoid function generator in output layer, 11... Charge storage gate type transistor (8 MNO elements), 12... Multiplier, 13.
...Voltage/time converter, 14...Subtractor, 21...
Accumulator, 22... Divider, 23... Coupling coefficient matrix, 24... Sample and hold circuit, 51
. . . Transconductance amplifier, 101 . . . Sigmoid function generator, 104 . . . W I J correction amount determination circuit. 113... Control electrode, 201, 202, 203...
connection point. Ne4 Eye surprise f To N+ (d) 1 Transco〉Turktance 7nA

Claims (1)

[Claims] 1. The source-drain resistance of a transistor with a gate that can nonvolatilely store charge is used as a coupling coefficient of a neural network, and when updating the coupling coefficient, an electric signal is applied to the transistor to calculate the coupling coefficient. What is claimed is: 1. An information processing device for modifying a coupling coefficient by applying the electric signal for a time proportional to a difference between a value of the coupling coefficient after updating and a value before updating. 2. The information processing device according to claim 1, wherein the transistor is formed of an MNOS transistor. 3. A circuit that forms a difference signal by subtracting the value of the corresponding element of the teacher vector from the value of the element of the output vector, and a circuit that forms a product signal by multiplying the difference signal by the value of the element of the intermediate layer vector. 2. The information processing apparatus according to claim 1, further comprising a circuit for applying the electrical signal to the transistor for a period of time proportional to the product signal. 4. A circuit that generates a first difference signal by subtracting the value of the corresponding element of the teacher vector from the value of the element of the output vector, and a circuit that generates a first difference signal by multiplying the first difference signal by the value of the element of the output vector. a circuit that forms a product signal of
a circuit for forming a product signal by multiplying the second product signal by the value of the element of the corresponding intermediate layer vector to form a third product signal; and a time period proportional to the third product signal. 2. The information processing apparatus according to claim 1, further comprising a circuit for applying the electric signal to the transistor. 5. A first amplifier circuit, a second amplifier circuit whose input offset voltage is different from that of the first amplifier circuit, and first and second amplifier circuits;
and a circuit for forming a difference signal obtained by subtracting the output of the second amplifier circuit from the output of the first amplifier circuit. The information processing device according to item 1.