JPH0348376A - Method and apparatus for bringing bond strength as desired to synapse of neural network - Google Patents

Abstract

PURPOSE: To realize high integration by providing a 2-quadrant multiplier, whose output signal is the product of a differential input and the difference of electric charge, and a means which charges a floating gate to a predeterminate level and discharges it. CONSTITUTION: The 4-quadrant multiplier with two pairs of differentially coupled field effect elements 29 and 30 is included which receives the differential input to generate a differential output. This cell includes also a pair of floating gate memory elements each of which has a floating gate 31 which can be programmed to a preliminarily determined electric charge level. Floating gates 31 are extended to control gates of another field effect elements, and drains of these elements are connected to the source of one of the differentially coupled pair. The current flowing through a half, namely, one side of the multiplier becomes a function of electric charge stored in the floating gate 31. Consequently, the obtained output signal is the product of the difference in electric charge between floating gates 31 and the input signal. Thus, high integration is realized.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to semiconductor cells, and more particularly to semiconductor cells having floating gate elements.

[Conventional technology]

Several circuit models have been proposed in an attempt to replicate the logic achieved by the human brain and the brains of other animals. These models provide means for both learning (eg, programming modes) and decision making (eg, recognition/associative memory). These models often required extensive calculations and complex computer programs. Attempts to use custom VLSI resulted in fairly complex circuits. In order to comprehensively consider neural networks, Adva
nced Research in VLSI A1i
+ pector > and r A by Al 1eH
Neuromorphic VLSI Learn
g S)'stem (neural-like VLSI learning system)''
the 1987 stanford conference
It is a good idea to read the paper in Rence (1987 Stamford Conference Bulletin).

Figure 6 of the Pector paper cited above shows a typical neuron or cell.

Neurons consist of dendrites (input end), synapses (connection points), neuron bodies (summing amplifiers), and axons (output end).
including. In order to facilitate the discussion,
Synapses are simply represented by resistors in FIG. 1 of this application.

This application describes these cells and synapses.

These cells in the prior art may include digital registers and digital-to-analog converters to provide weighting factors or functions. Other means are also provided for multiplying this coefficient by the input signal. Since each cell in the network thus requires a kana b amount of circuitry, even with current VLSI technology,
It is not possible to create large numbers of cells (eg 10,000) on a single chip.

Other prior art known to the applicant is the following passage: IEgE Transactions on
R. in Ele1987. r by Howard et al.
An Associative Memory
Based on an Electronic
Neural Network Architecture
J.
r An Artificial N by Sage
ural Network Integrate
dCircuit Based on MNOS/
A paper entitled "CCD Principles (Artificial Neural Network Integrated Circuit Based on MNOS/COD Principles)".

cults+ Mol. SC-20, December
J. er +1985. rA20VFour - Quadrant C by Babanezhad
MOS Analog Multitier (20
A paper entitled "Volt 4-Quadrant CMOS Analog Multiplier)". IF,E on Neural Networks held in San Diego, California, July 24-27, 1988.
F. submitted to the E International Conference. J. Mack and K. K.
Moon and C. T. Yao and J. A- Model
o and r Programmable Anal
og S)'napses for Microel
electronic neural networks
Using a Hybrid Digital-An
alog Approach (Programmable analog synapse for microelectronic neural networks using a hybrid digital-analog approach)
” paper entitled.

The present invention uses a floating gate memory element within the cell to provide a weighting factor or function and to provide the multiplication of an input with said weighting factor. Many floating gate memory devices are well known in the prior art. For example, U.S. Pat. No. 3,728,695;
5,7 No. 21, same No. 3,9 8 4, 8 2 2,
Same No. 4,1 4 2,9 2 6, Same No. 4,203,
No. 158, No. 4,412.310, No. 4,46
No. 0,982, No. 4,519,849, and U.S. Patent Application No. 66 entitled "Low Voltage EPROM Memory," filed November 2, 1984.
There is No. 7,905. These patents and patent applications are
electrically programmable read-only memory (EPROM)
), cells for electrically erasable and electrically programmable read only memories (EEPROMs), cells for flash EPROMs, and related circuitry. Additionally, the present invention uses a four-quadrant multiplier or Gilbert multiplier to perform analog multiplication of the inputs and weighting factors. The use of such multipliers in analog circuits is well known in the art. As an example, John Wi of New York
Published by Ley and Sons in 1977, Pa111
R. GrayS Robert G. Co-authored by Meyer
Anal)'sis and Design of A
nalog Integrated Circuits
(Analysis and Design of Analog Integrated Circuits)”.

In the prior art, there are EPROM cells that have floating gates to extend read capabilities from the device. These are used to improve the read speed from the cell.

[Summary of the invention]

The present invention provides a synaptic cell for a neural network that includes a four-quadrant multiplier having two pairs of differentially coupled field effect elements for receiving differential inputs and producing differential outputs. The cell also includes a pair of floating gate memory elements each having a floating gate that can be programmed to a predetermined charge level. Each of the floating gates extends to the control gate of the other field-effect devices, and the drains of these devices connect to one of the differentially coupled pairs.
connected to the source of one thing. In this way, each of the currents flowing through one half of the multiplier, or one side of the multiplier, is a function of the charge stored on the floating gate. The resulting output signal is thus the product of the difference in charge present on the floating gate and the input signal.

〔Example〕

A semiconductor cell using a floating gate memory element, etc., which is particularly suitable for use in neural networks, will be described. In order to enable a thorough understanding of the present invention, in the following description,
Numerous specific details are mentioned, such as specific conductivity types.

However, it is not necessary to use these specific details in practicing the invention. In other words, descriptions of well-known structures and circuits are omitted to avoid unnecessarily obscuring the present invention.

Prior Art Neural Network FIG. 1 shows a portion of a prior art neural network. Line 1 0, 11, 12.13
is given an input (eg, a stimulus) to the network. These inputs are connected to the current summing amplifier 1 via resistors.
4, Is, 16.17. From the output of each of these amplifiers lines 18, 19,
The output signals provided to 20, 21 are coupled via resistors to the input of a current summing amplifier in the next layer of the neural network. The input signal contributes to the output signal through a resistor similar to a biological two-aeron synapse. As an example, consider the input to amplifier 1T. Signals applied to lines 10, 12, 13 are coupled through resistors to amplifier 1T. To further elaborate for purposes of illustration, the potential at node 24 is across resistor 22
A current is supplied to the input terminal of the amplifier 1T through the terminal. The resistor can be either "short circuit" or "open circuit" in the simplest case, or a resistor (in the case where binary weighting is used). may have a value.

At first glance, the network shown in FIG. 1 appears to be relatively easy to implement, but in reality, it is particularly difficult to realize as an integrated circuit. For one thing, it is difficult to create multiple resistors in a network with different resistance values, and it is particularly difficult to create resistors that can be programmed into the network after the b1 network has been created. be. (The value of each resistor represents "learning" and can be determined in several ways, as the prior art has shown. The values of these resistors in some applications are: 0°, which is kept fixed for the decision-making mode of the network, as opposed to the learning mode) The aim of the invention is to provide a cell in which resistors such as resistors 22, 23 are replaced by others. It is to be. Note that each resistor can be considered to have an input (eg, node 24 on resistor 22) and an output (eg, node 25 on resistor 22). Ideally, the resistor 22 should be able to be programmed to a predetermined value after the network is created, and the value should be non-volatile and stored in the cell.

The operation of the invention is best illustrated in FIGS. 2a and 2b and 5. FIG. The present invention uses well-known prior art memory cells having components that can be electrically charged. A polysilicon floating gate is often used in these cells, and the floating gate is completely surrounded by an insulator (eg silicon dioxide) VC. Charge is generated by electron avalanche injection, channel injection, electron latent escape,
These floating gates are moved to these floating gates by various mechanisms such as. The charge on the floating gate affects the conductivity in the cell. A cell is considered programmed to one binary state if its conductivity is above a certain level, and it is programmed to one binary state if its conductivity is below another level. It is assumed that the other binary state is programmed. Such memory cells have taken various forms in the prior art.
Some are capable of both electrical erasing and electrical programming, while others require, for example, an index line for erasing. This cell would be housed within a prior art memory, in which case the prior art includes an EPROM.
and 1i EPROM and Flash EPROM and the specific cells cited in the prior art portion of this specification.

The present invention utilizes these prior art cells. However, in the present invention, the charged element (ie, the floating gate) extends beyond the prior art memory cell to the second or other element. The charge on the floating gate controls the current flowing through this second element.

In FIG. 2a, an n-type EPROM cell 29 is shown coupled in series with a p-channel device 30. Element 3
0P is connected to Vcc via a p-channel transistor 26, while a transponder 29 is connected to an n-channel transistor 2T.
connected to ground via. As shown in FIG. 2b, device 29 includes a pair of spaced apart n+ regions on a p-type substrate 34. As shown in FIG. floating gate 31
The sections 31c are located above a channel exposed to each other and defined by the fcp-type sacrum. The control gate 35 is disposed above the floating gate 31, and the floating gate 31 is also insulated. Generally floating gate 3
1 (including section 31a131b131c) is formed of polysilicon and completely surrounded by silicon dioxide or other insulating material. The charge is, for example, element 2
9 into the floating gate 31 as shown by arrow 35 in FIG. 2b. The state of charging and discharging of this floating gate is as follows.
It is well known in the prior art.

The second field effect element 30 shown in FIG. 2a is
Generally, the device 30 consists of a field effect transistor with a gate 31 (section 31c) extending above the channel region. As shown in Figure 2b,
P-type device 30 may be formed within an n-type substrate well and includes a pair of spaced apart p-type regions. Gate 31 extends above the channel defined by the p-type region. The electric field created by the charge on floating gate 31 affects the current flowing between the p-type regions of device 30.

An expanded oxide region separates device 29 and device 30. A capacitor 28 is formed on this oxide. This capacitor is connected to section 3 of floating gate 31.
lb and members 36 separated by a layer of oxide. Capacitor 28 provides additional coupling between floating gate 31 and control gate 35a. This coupling is necessary to program the floating gate.

The sections y31m, 31b, 31c of the gate 31 can be formed from a continuous polysilicon layer by several MOS processes. According to another process, sections (or sections) are formed from separate polysilicon layers and interconnected by connecting members. This connecting member is depicted as a lead wire 31d in FIG. 2b. These connecting members are
It is typically formed away from the active channel regions of elements 29,30. In both cases the gate 31 (all sections) is completely surrounded by oxide, similar to floating gates of this type in EPROMs.

For convenience of explanation, it is assumed that a positive potential is applied to the drain region of element 29, and that the source region of element 29 is connected to ground via transistor 2T. As charges (electrons) are transferred into gate 31, the conductivity of element 29 decreases, while the conductivity of element 30 increases. 1. If this floating gate is not charged, the cable 2
When 9 is in a conductive state, the conductivity of element 30 is reduced. Charging of gate 31, ie charging of gate 31, is possible during the period when charge line 33 is positive and capacitor 28 provides coupling to gate 31. If the potential on line 33 is ramped up from zero to Vcc and the current flowing into line 32 is measured, it can be determined how much charge is on the floating gate. During this detection period, S
(one input to the cell) is positive.

The cells shown in Figures 2a and 2b are also useful in neural networks, but may also be used in other applications. For example, if a programmable decoder is used to select redundant rows or columns in a memory having redundant elements such as "rows or columns to replace defective rows or columns," In such a programmable decoder there is a second
Figure a and number ? The cells in figure b may be used. An example of such a programmable row decoder is U.S. Pat.
50, 572.

When the cell of Figures 2a and 2b is used in a neural network, it is used to multiply the input signal rSJ by the charge on the floating gate. The amount of charge on the floating gate and its use as a weighting factor representing coupling strength will be explained with reference to FIG.

The cells of FIGS. 2a and 2b in the neural network are cells to replace the resistors of FIG. 1. The inputs to this cell are "S" and "S" and its output appears on line 32.

Another cell is shown in FIG. This cell is
It has the ability to perform multiplication when the input is other than ``1'' and ``0'', ie, it is possible to multiply an analog input by a weighting factor. The weighting factor in this case is also expressed and defined by the amount of charge on the floating gate. The technique for charging this floating gate to the desired level 1 will be described later in connection with the embodiment of FIG. 4a.

The description applies to all embodiments.

Also used in the cell of FIG. 5 (as shown to the left of line 41) is a floating gate device 37, which may be a memory cell such as an EPROM SFEPROM, flash EPROM, or the like. The floating gate 44 is extended to serve as a gate member for the n-channel device 38 and the p-channel device 39.

Element 3B and element 39 have their gates connected to element 3.
It is the same as an ordinary field effect transistor except that it is an extension of the floating gate of 7. The charge present on the floating gate controls the current flowing through the channels of devices 38 and 39. One terminal of element 38 receives input rUiJ, and one terminal of element 39 receives input r-UiJ. The other terminals of elements 38 and 39 are connected to node 40. Line 41 consolidates the input to operational amplifier 42. Line 41 and the positive terminal of amplifier 42 are held at a reference potential. Input to amplifier 42 (IS
The currents coming from multiple cells are summed in line 41 to provide UM). The contribution to this sum by the illustrated cell is indicated as ID.

The output of operational amplifier 42 is a voltage equal to VREF divided by the product of sUM and R. This output voltage is applied to the signal discrimination circuit 43. As shown in the figure, the signal discrimination circuit 43 provides signals that are directed in positive and negative directions centering on the reference potential and have a magnitude proportional to IlitlM. As circuit 43, ordinary, well-known circuits may be used.

The cell of FIG. 5 could be incorporated into the neural network of FIG. 1, with the resistor of FIG.
, 3B, 39.

For example, if resistor 22 were replaced with these elements, node 25 in FIG. 1 would not be node 40 in FIG. 5, and the total I8UM shown on line 41 in FIG. represents the sum resulting as input to amplifier 17 of . In this example, the signal discrimination circuit 4 of FIG.
The output of 3 is a pair of lines that replaced line 21 in Figure 1. be. This is important because current XD is a function of the charge present on the floating gate of element 3T.

The equation representing the current Iot- is shown in FIG. In the formula, the quantity in parentheses is the weighting coefficient aWji, and vD
s is the drain-source voltage.

In the illustrated embodiment, one of the terminals of element 38 and one of the terminals of cord 39 are held at a reference potential -J? C,
■■ is the input. The sum (netj) on line 41 is equal to the sum of the products of each cell's weighting factor and the input signal. The quantities in parentheses expressing weighting factors include v0. vG is shown in Figure 2b κ "upper goo shown"35,
The potential at the top gate, such as 36. vT is
Is the cell "excited" κ programmed or "inhibited"?
element 38 that depends on whether it is programmed to
or the threshold voltage of element 39. Q7,/Cp
G is the voltage on the floating gate due to the charge stored on the floating gate. Ideally, vT should be close to zero volts in order to make the ID the product of the weighting factor and the input. Of course, the thread is close to zero? Obtaining the hold voltage can be altered by doping the channels of elements 3B, 3g. There is a dead zone in the floating gate voltage whose weight is equal to zero, and in the illustrated embodiment, the dead zone is between the threshold voltage of the p-channel device (negative potential) and the threshold voltage of the n-channel device (positive potential). occurs between In this dead zone, large programming pulses are required when changing the polarity of the weights. Furthermore, linearity is lost when the weights are small or when the input voltage is very large, or both overlap, so there is a limit to the range of inputs and weights over which satisfactory multiplication can be achieved. There is. In other words, there is a limit to the dynamic range of inputs and weights.

The embodiment of FIG. 3 overcomes some of the problems mentioned with respect to the cell of FIG. 5, such as dynamic range limitations. Also the cells in Figure 3! It includes a floating gate rope 73. A transistor for applying charge to the floating gate of this element 73 is connected in series to the floating gate element T3. Another transistor is coupled to the control gate of element 73. By applying an additional potential (in addition to the power supply potential) to the line T1 and one end of the cable element 73, the element 7
The floating gate 72 of No. 3 may be charged to a predetermined level as described below in connection with the embodiment of FIG. The floating gate of element T3 extends into and is part of field effect element T4 in a manner similar to that described with respect to the cell of FIG.

One end of element 74 and one end of element 75 are each coupled to a differential input end of operational amplifier 79.

Specifically for purposes of illustration, line 7T is connected to amplifier 18.
and line 78 is coupled to its positive input. The other ends of field effect elements 74 and 75 are both coupled to field effect element 76 . An input signal is applied to the gate of this element T6.

Operational amplifier 79 also receives inputs from other cells. In this embodiment, lines of other cells equivalent to line 77 are coupled to line 81, and lines of other cells equivalent to line 78 are coupled to line 82. The output of amplifier 79 (line 80) becomes the input to the other cells as already mentioned. Element 74 in the illustrated embodiment,
7S, 76 have the same conductivity type. Amplifier 79 amplifies the difference in current in line 77 and line 78.

Assuming that the charge applied to the floating gate 72 has no effect on the element T4,
5 are exactly the same, this differential current is zero. This represents that the weighting coefficient is zero. When a charge is applied to the floating gate 72, the element 7
The differential current between 4 and element 75 increases (ray y81,
82 are both held at a constant potential), this difference is multiplied by the input signal applied to element 76.

The amplifier output 80 therefore represents the input to this cell applied to the gate of element T6 multiplied by a weighting factor (the charge on the floating gate 72).

Embodiment of FIG. 4 In the prior art, floating gate memory devices generally
It is charged (and erased) by a series of pulses κ.

After this pulse train is applied, the floating gate is charged (
The conductivity of this element is checked to see if it has been removed (or erased).

If the results are not favorable, additional charging (or erasure) can be achieved by supplying more pulses to this element.
will be held. In the present invention, the floating gate is charges will be monitored.

FIG. 4a shows a pulse generator 51 coupled to a floating gate strand 52 and a pulse generator 51 coupled to a floating gate element 55. A pulse train used to charge these elements is applied to the elements via line 53. The applied amount of electricity is indirectly ascertained based on monitoring the threshold voltage of the floating gate element or the threshold voltage of a second element sharing the floating gate. This monitoring is illustrated at line 54. The circuitry used to generate the pulse train and the circuitry used to monitor the voltage to ascertain the charging level of the floating gate may be conventional, well-known circuitry.

The amount of electricity charged on the floating gate determines the value of the resistor, such as resistors 22, 23 in FIG. 1, or in other words the strength of the connections of the synapses in the neural network.

The cell of FIG. 4 operates in a dynamic mode and therefore consumes less power. This is especially useful when the input is a 1 or a 0. In this example, gt(5
Two floating gate devices are used, denoted 2) and I 2 (55).

These elements are also EPROM,! It may be a cell for FROM, flash EPROM, etc. As mentioned above, the Norx generator 51 is used to charge the element 52.
is coupled to this element. Such a pulse generator is also combined with element 55. Input line 50 is coupled to the input of the amplifier via two separate paths.

One path includes elements 56 and 58, and the other includes elements 5T and 59. Similar to FIG. 2, the floating gate of device 52 extends to device 56 and hp,
The floating gate of also extends to element 5T.

There is a capacitor 6o between the element 56 and the element 58,
Similarly, a capacitor 61 is connected between element 57 and element 59.
There is.

For the illustrated n-channel embodiment, positive waveforms are provided on lines 65 and 66. First, line 65 goes high (while line 66 goes low), charging capacitors 60, 61, as shown in FIG. 4b. Line 66 then goes high after the potential on line/65 goes low, allowing the charge on these capacitors to be coupled through elements 5B, 59 to operational amplifier 62. Charges from other cells are also transferred to lines 63, 6
4 to amplifier 62.

The weighting factor in the embodiment of FIG. 4a is determined by the difference in charge stored in floating gate element 52, 55K.

If both floating gates have the same amount of charge, both devices ss and sy will pass the same amount of input current commensurate with the input signal applied to line 50. and,
Capacitor 6061 is charged to the same level i and the same amount of electricity is coupled to amplifier 62. Therefore, in this case,
The difference in the output line of the differential amplifier 62 between the two is also zero. The greater the difference in charges present on the floating gates of the resistors 52, 55, the greater the weighting factor.

If it is necessary to use ordinary circuitry to prepare the input of values to cells in the neural network, the amplifier 6
The output of 2 may be converted to a binary signal.

Embodiment of FIG. 6 The embodiment of FIG. 6 has only one conductivity type element, e.g.
It has the advantage of using only channel elements. This cell is EPRO! [00 and EPROM101].

These elements may be conventional elements, but include floating gates extending into the other cords 104, 105, as described in connection with other embodiments. Pulse generator 10
2 and 103, as in other embodiments, are used to program the gates of elements 100 and 101, respectively, to the desired charge level for the purpose of obtaining weighting factors.

The floating gates of the cables 100 and 101 are connected to the element 1, respectively.
04, 105 over the channel region.

The charges applied to the gates of these elements 1ll}4 and 105 are current IDe (line 108) and current IDl (line 108).
line 109). Elements 104, 1050 control gates are connected to ground, so elements 104, 1050
Before any charge is taken in or taken out from the floating gate of 05, vPoe and vFGi are zero. Element 1
04 and 105 are transistors 106 and 107, respectively.
connected in series with The input J to this cell is provided on line 114.

The threshold voltages of elements 104, 105, 106, 101 are preferably close to zero volts, with v and vT paths preferably being zero. The elements 106 and 107 used in this cascade-type embodiment are elements with a relatively wide range and a small substrate effect.

(2) De (excitation current) and Ioi (inhibition current) are coupled to the differential amplifier 110. The floating gates of stringer 100 and element 101 are charged to different levels to obtain a large weighting function. Suko 10
When a large positive charge is applied to the floating gate of 1, the current factor Di increases, increasing the output of amplifier 110. When a large positive charge is applied to the floating gate of the probe 100,
The output of amplifier 110 decreases. Formulas for determining IDf1 and IDl are shown in FIG. In this case too, the weighting factor is proportional to the charge on the floating gate, which is multiplied by the input voltage U1 applied to this cell from another amplifier.

Summation with signals coming from other cells takes place at the input to amplifier 110. Lines 112, 113 represent input signals coming from other cells.

Embodiment of FIG. 7 FIG. 7 shows a programmable synapse approach using fully differential four-quadrant multipliers.

The multiplier shown in FIG. 7 is a simplified version of the Gilpert multiplier by reducing the number of elements and reducing the Vcc range. The principles of operation of four-quadrant multipliers and Gilpert multipliers are well known in the art. For example, Paul
R. Gray, Robert G. Meyer
Co-author: Analysis and Design
fAnalog Integrated Circuit
ts (Analysis and Design of Analog Integrated Circuits), New York, published by John Wiley and Sons (
(1977), pages 554-603, a description of these devices can be found.

The four-quadrant differential multiplier approach has significant advantages over other programmable synapses, such as wide signal processing bandwidth, high input impedance, and good bias control. Since the cell operates differentially, variations in input level or floating gate voltage do not significantly affect the accuracy of the multiplication. Other features of the four-quadrant differential multiplier approach include good multiplication linearity, especially when the weights and inputs are small, and the ability to adjust the weights without having to precisely adjust the charge applied to the floating gate elements. and the ability to set it to zero. Moreover, this technique doubles the storage capacity of synapses.

The cell of FIG. 7 includes floating gate devices 123, 124. Floating gate devices 123, 124 are each coupled in series with a transistor used to provide charge to the floating gates of those devices. Furthermore, in order to facilitate charging of the floating gate, elements 123 and 124
An auxiliary transistor is coupled to the control gate of. With this, floating gate charging devices 120, 121
The configuration is complete. Other interchangeable circuits using floating gate devices may be used in place of charging devices 120, 121 without departing from the scope and spirit of the invention.

By applying a positive potential (in addition to the normal power supply potential) to the word line ■wLA and the program line VPG during the programming period, the floating gate of the probe 123 is brought to the expected level! can be charged with electricity.

The charge stored on the floating gate of element 123 is then
It is transferred via line 125 to the gate of field effect element 127. Line 125 in this presented example
is an extension of the floating gate of element 123 rather than a separate connection. The charge QFGI on line 125 represents the weight of this synapse. Similarly, line 126 can be charged to a predetermined reference level 1, represented by QFG2, by raising the potential of line VWLII and line VPG. Thus, weights and reference potentials are programmed into the synaptic cell of FIG.

Charging element 1 that is exactly the same as charging element 120
21 is a key design point in the embodiment of FIG. If the bias setting means for establishing a reference potential on line 126 is not aligned with the other,
Variations in temperature, noise, bias settings, and processing all cause device 121 to "follow" device 120. Such variations are common in integrated circuit environments. .Charging element 1
When 20, 121 are aligned, the difference between the weight line and the reference line remains stable, thereby ensuring accurate multiplication and stable weighting by the synapses.

Both cables 127 and 12B operate as current sources.

That is, the charges present on the gates of these elements are the differential pairs 129, 130 and 131, 1, respectively.
32. The drain of element 127 is connected to element 1
29 and the source of element 130. Similarly, the drain of element 128 is coupled to the source of strand 131 and the source of strand 132. element 129
The drains of element 130 and element 131 are coupled to output line 133, while the drains of element 130 and element 132 are coupled to output line 134. This completes the configuration of the four-quadrant multiplier.

In operation, input signals (designated V and VK in FIG. 7) are applied to the gates of each differential pair. In other words, ■
- is applied to the gates of the cable 129 and the element 132, and V
is applied to the gate of elements 130 and 131.

The difference in charge stored on lines 125 and 126 governs the current flowing through the differential pair. Therefore, the differential input voltage (vv) is effectively multiplied by a factor representing the difference in charge stored on lines 125, 126. The differential output for a small signal input [flow is the difference between the direct load at the weight line 125 and the reference line 126 and the
It is precisely proportional to the differential input voltage. The equation is shown in FIG.

IDe (excitation current) and IDi (inhibition current) are shown along lines 133 and 134, respectively, in FIG. 7, and are coupled together to a differential amplifier 137. If the positive charge on the floating gate of element 123 is large (compared to the positive charge of element 124), the current IDi increases;
That increases the output of amplifier 137. Summing with signal currents coming from other cells takes place at the input to amplifier 137, as shown at line 135 and line 136.

Embodiment of FIG. 8 The embodiment of FIG. 8 has a smaller cell size.
This cell has advantages over the cell shown in FIG.

Elements 123, 170 and elements 124, 171 of FIG. 7 have been omitted in FIG. Instead, elements 156, 150, 151 and elements 157, 154,
155 performs multiplication and programming functions.

Referring to FIG. 8, elements 151 and 152 and elements 153 and 154 each form a differential pair of four-quadrant multipliers, as described above in connection with FIG. Differentially coupled. The source of element 151 and the source of element 152 are both coupled to the drain of element 156, while the source of element 153 and element 1540 are coupled to the drain of element 156.
The source is coupled to the drain of strand 157. The source of element 156 and the source of element 157 are both VSa
is combined with coupling the drain of elements 151 and 153 to an output line 159;
By coupling the drains of element 154 to output line 160, the implementation of this Gilpert multiplier is completed. Typically, input V1 is applied to the control gates of elements 151 and 154, while input V+ is applied to the control gates of elements 152 and 153.

Similar to FIG. 7 except that the weights can be changed without requiring the four transistors used to charge and discharge the floating gate elements. To program element 156, word line VWL
The program line VPGI (connected to the control gates of elements 150 and 155) and the program line VPGI (connected to the drain of element 150) are raised to a high positive potential. Due to the high positive voltage present on the control gate of device 156, electrons burrow through the thin oxide region at the drain and are then captured by the floating gate to reach the (programming) threshold of device 156. to rise. Similarly, the potential of line vwL and line V
By increasing the potential of PG2, the b1 electrons are forced to rush into the floating gate of the probe 157. Suko 155
The drain, control gate, and source of VPG2 are respectively
, and the control gate of the VWL% element 157. During programming, line V- and line V-
are connected, or line 159 and line 16G are left floating.

Electrons are removed from the floating gate of device 156 by connecting VWL to a high voltage and grounding VPGI to ground the control gate of device 156. (2) By raising + and R/160 to a high voltage, the drain of element 156 is also brought to a high voltage. This can also be accomplished by placing 1 and line 159 at a high voltage.

By applying a high bias to the drain of device 156 and grounding its control gate, electrons escape from the floating gate to the drain, thereby reducing the (erase) threshold of device 156. Electrons may be extracted from the floating gate of probe 157 in a manner similar to that just described.

As discussed above in connection with FIG. 7, the coefficients or weighting factors for multiplication are the floating gate of element 156 and
It is determined by the difference in charge stored in the floating gate of 57. Typically, elements 156 and strands 157 are made identical to provide the weighting factors (represented by the difference in charge stored on the floating gates) with immunity to variations in temperature, power supply, processing, etc.

Output line 159 and output line 160 in FIG.
Coupled to amplifier 163. Summing with signal currents coming from other cells takes place at the input to amplifier 163, as indicated by lines 161 and 162.

Embodiment of FIG. 9 Conventional 4 included in the cells shown in FIGS. 7 and 8
One of the drawbacks of the quadrant multiplier approach is its inability to handle large differential input voltages.

(Here, a large input voltage refers to an input voltage that is approximately equal to the power supply voltage.) The ability to handle large differential input voltages is important in synaptic cells of neural networks that are applied to certain tasks. This is a long-awaited feature. The embodiment of FIG. 9 can achieve this objective.

The multiplier shown in FIG.
80, 181 are included. Each device 174-181 shown in FIG. 9 is a zero threshold n-channel device. Although zero threshold devices are not a necessary condition, they can improve linearity in multiplication of small input voltages close to zero volts. The gates of element 178 and element 180 are coupled to the V input line, while the gates of element 179 and element 181 are coupled to the V input line. The drains of elements 178 and 179 and the drains of elements 180 and 181 are coupled to output line 182 and output line 183, respectively.

The source of element 178 is coupled to the drain of field effect element 174, while the source of element 179 is coupled to element 175.
It is connected to the drain of the Similarly, the source of field effect element 180 is coupled to the drain of element 176, while the source of field effect element 181 is coupled to the drain of element 177. Elements 174-177 each control the current flowing through the field-effect elements 118-181 that are combined with each other. For example, the current IOA flowing through line 182 is equal to the current flowing through element 178 plus the current flowing through element 1.
79, they are further added to the current flowing through the cord
4 and element 175. Current IDB
occurs along line 183 in a similar manner.

The sources of elements 174-177 are each coupled to vss or ground potential. The control gates of device 174 and device 177 are each coupled to floating gate memory device 187 via line 172 . Similarly, the control gates of device 1750 and device 176 are each coupled to floating gate memory device 188 via line 173. Element 187 and element 188 are used to charge lye 172 and lye/1T3, respectively, to predetermined levels.

The charge present on lines 172, 173 determines the current flowing through each of the differential pairs. The differential pair is coupled to output lines 182, 183.

As explained in connection with FIGS. 7 and 8, line 1
The difference between the charge on line 72 and line 113 represents the factor by which the input signal is multiplied.

Since elements 174-177 are operated in a linear mode, the range of input voltages applied to
It is possible to cover a wide range. As an example of how the range of linearity is extended by this cross-coupling scheme, consider the case where field effect elements 178, 180 are saturated in response to one extreme input. Although elements 178, 180 no longer control current ID^ and current IDB, elements 179, 18
1 remains below saturation to maintain linearity in the current difference between lines 182 and 183. Therefore, even if the other input is extreme and the field effect elements 17g, 180 are in a saturated state, the linearity of the current difference (that is, the linearity of the multiplication) is maintained. in this way,
Current! The differential output current, represented by the difference between DA and current TDB, is the difference between the charge on line 172 and line 173 multiplied by the differential input voltage for any input ranging from zero volts to supply voltage 1. maintained as such.

For further understanding, consider the following example.

If I1, ■+ is 5 volts and V1 is O volts, element 178 and cable 180 will conduct as much current as element 179 and cable 181. Since the potential of line 172 is raised compared to the potential of line 173,
The conductivity of elements 174 and 177 is even higher. Therefore, more current flows through element 17B and element 1, respectively.
81. However, since the conductivity of element 178 is as high as that of element 181, the current increase due to the large charge on line 172 is
This results in a further increase in the current in line 182 compared to the current in line 183.

The floating gate elements 174 to 17 which also house the cell in FIG.
7, it is not necessary to separately manufacture the field effect memory elements 187 and 188. Naturally, this may reduce the size of the cell. Programming is done in the same manner as described in FIG.

As shown in FIG. 9, line 182 and line 183 are coupled to a differential amplifier 186.

The summation with signal currents coming from other cells is performed by amplifier 186, as shown at lines 184 and 185.
This is done in the input to.

〔Effect of the invention〕

Above, several cells particularly suitable for neural network hooking have been described. The weighting factor is expressed by the difference in charge stored on two separate floating gates. This charge typically remains stored on these floating gates over long periods of time and may provide non-volatile storage of the weighting factors. After the circuit is created, each of the floating gates may be charged to a different level. This cell can contribute to the realization of high density VLSI circuits.

[Brief explanation of drawings]

FIG. 1 is a diagram of a typical prior art neural network to illustrate the placement of the cells of the invention within the neural network; FIG. 2a is an electrical schematic diagram of a cell according to the invention; FIG. 2b 2a is a sectional view of a portion of the substrate of the cell of FIG. 2a; FIG. 3 is an electrical schematic diagram of one embodiment of a cell according to the invention; FIG. 4a is a further embodiment of the cell of the invention. 4b is a waveform diagram for the cell of FIG. 4a; and FIG. 5 is a diagram of yet another embodiment of the invention to illustrate the weighting function performed by the cell of the invention. FIG. 6 is an electrical schematic diagram of another embodiment of the cell of the invention; FIG. 7 is an electrical schematic diagram of another presented embodiment of the cell of the invention; FIG. FIG. 9 is an electrical schematic diagram of yet another proposed embodiment of a cell of the invention. FIG. 14, 15, 16, 17, 42.62, 79, 110,
137,163,186 ●●●● Amplifier, 22.
23...Resistor, 28,60. 61...Capacitor, 31m, 3lb, 31c...●Floating gate section, 31d●●●◆ Red wire, 34...
・P-type substrate, 36... member forming a capacitor,
3T, 52.55. T3.100,101,123,1
24,187.188 ●◆●・Floating gate element, 3
8.39, 56.57.74-76, 104,105,
127-132, 150●・●●pulse generator, 1
25 ●●●● weight line, 126 ●●●● reference line. -] Do IG 1 - Summer FINE 8 7INE: 4B 71G 5 IC+ = To-VT Ichimatsu S n●Na, abbreviated ΣwJIul - -■7I [He 1i 1E'IIG 7 ΔI-αX to, G, - OFG21Δ Construction raider ◆1 I
1 Procedural Amendment ζOki) 3. Name of the patent applicant related to the person making the amendment Date of positive order ■ Date of Heibō Z year g month 2′ g day 6 subject to amendment

Claims (1)

[Claims] 1. A program each having a differential input and a differential output that is a weighted function of the differential input, and a plurality of programs are included in a neural network. In a possible synapse: a four-quadrant multiplier for receiving the differential input and producing the differential output, the four-quadrant multiplier being differentially coupled to a first pair of field effect elements; a second pair;
each of the pairs is connected to a floating gate device;
The floating gate device has a floating gate and a control gate, and the current flowing through each differentially coupled pair is determined by the difference in charge stored in the floating gate, so that the output a four-quadrant multiplier whose signal is the product of the differential input and the charge difference; and means for charging and discharging the floating gate to a predetermined level. synapse. 2. In a method for providing desired coupling strength to synapses of a neural network: charging a first floating gate element to a predetermined weight level and charging a second floating gate element to a predetermined reference level; the steps of: sensing a differential input signal; multiplying the input signal by a coefficient represented by the difference in charge between the weight level and the reference level; and multiplying the input signal by the coefficient. The differential output signal is
A method comprising the steps of: providing a differential input to a neuron amplifier. 3. At multiple synapses included in a neural network for accepting large differential input signals and producing differential outputs that are weighted functions of said differential input signals: for separate control. a multiplier device comprising first, second, third and fourth input field effect elements coupled in series to and combined with a field effect element, said first and said second input field effect elements; the third and fourth input field effect elements are connected to one of the differential output lines, and the third and fourth input field effect elements are connected to the other of the differential output lines; The third input field effect element is connected to the first line of the differential input signal, and the second and fourth input field effect elements are connected to the second line of the differential input signal. the control field-effect element has a chargeable gate element for controlling the current flowing through the combined input field-effect element; A differential output is provided between the gate elements of the control field-effect element that is combined with the first and second input field-effect elements or the third and fourth input field-effect elements. A synapse comprising: multiplying a charge difference by said differential input signal; and means for charging said gate element to a predetermined level. 4. a neural network, each having a differential input and a differential output that is a weighted function of the differential input and coupled to at least one of the plurality of neuron amplifiers; in a plurality of programmable synapses included in: a four-quadrant multiplier for receiving the differential input and producing the differential output; a four-quadrant multiplier configured to provide the differential output to at least one of the plurality of neuron amplifiers including a first pair and a second pair; control means for independently controlling the current flowing through each of the pairs; and the control means includes a pair of floating gate elements, each of the floating gate elements being a chargeable floating gate. , such that the difference in charge on the floating gate determines the current, thereby causing the output signal provided to at least one of the plurality of neuron amplifiers to A synapse, characterized in that the input is multiplied by the difference in charge. 5. At a plurality of synapses in a neural network for accepting a large differential input signal and producing a differential output signal that is a weighted function of said differential input signal: a first output; first and second input field effect elements having drains connected to the differential output line; and third and fourth input field effect elements having drains connected to the second differential output line.
A multiplication device including a field-effect element for input,
The gates of the first and third input field effect elements are connected to a first differential input line, and the gates of the second and fourth input field effect elements are connected to a second differential input line. a multiplication device connected to; said multiplication device further including first, second, third, and fourth pulldown field effect elements, each of said pulldown field effect elements The drains are the first, second, and
The source of each pull-down field effect element is connected to the ground or a similar potential, and the source of each of the first and fourth input field effect elements is connected to the ground or a similar potential. The gate of the pull-down field effect element is connected to a first floating gate element having a chargeable floating gate, and the gates of the second and third pull-down field effect elements are connected to a chargeable floating gate element. a second floating gate element having a gate element, said chargeable floating gate element controlling the current flowing through each said pull-down field effect element such that said differential output signal is , the difference in charge between the floating gate elements of the first and second floating gate elements is multiplied by the differential input signal; and means for charging the floating gate element to a predetermined level. A synapse characterized by comprising: and;