JP2003223790A - Synapse element and integrated circuit device including the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To implement a high integration of a synapse circuit with a learning function which is the most important subject in achievement of a neural network associative memory LSI (In a neural network constituting of an associative memory, many synapses are required and the number is almost the square of the number of the neuron, and therefore, high integration of synapses is most effective for high integration of the associative memory). <P>SOLUTION: An A-MOS synapse can implement comparable integration to that of a DRAM because of its simplified circuit configuration and compact circuit size. With the present cutting-edge technology (0.15 μm CMOS), approximately 1 G synapses can be integrated on one chip. Accordingly, it is possible to implement a neural network with approximately 30,000 neurons all coupled together on one chip. This corresponds to a network scale capable of associatively storing approximately 5,000 patterns. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network associative memory LSI, and provides a technique capable of realizing highly integrated synapse elements having a learning function. In a neuro-associative memory, a synapse circuit with a learning function is absolutely necessary to store associative patterns freely and at high speed. Since the neural network associative memory LSI configured by the synapse element according to the present invention can realize high integration like DRAM, it can contribute to the practical application of an associative memory LSI capable of storing thousands of patterns, which has been difficult in the past.

[0002]

2. Description of the Related Art In recent years, research and development of brain-type computers based on the information processing mode of the brain have been actively conducted. Intuitive information processing, such as pattern recognition, contextual association, and combinatorial optimization, that a brain computer is good at is an essential technology for information processing machines to realize natural communication with humans, and machines are It is expected to give a breakthrough so that it can be used and blended in without any discomfort. In order to put a brain-type computer into practical use, the development of dedicated hardware is indispensable. In particular, the development of a neural network associative memory LSI has been strongly demanded as a main component of a brain computer. In order to put the neural network associative memory LSI into practical use, the most important issue is the high integration of the synapse circuit having a learning function. In a neural network that constitutes an associative memory, a large number of synapses is required in proportion to almost the square of the number of neurons, and thus high integration of synapses is most effective for high integration of associative memory. Also,
In order for the associative memory to store associative patterns at high speed and freely, it is necessary for the synapse to have a learning function.

[0003]

As a prior art relating to an associative memory neural network LSI having a learning function, a patent "integrated circuit device with learning function" inventor: Yu Arima, et al. (Application: September 19, 1991) JP 03-80
379 5,148,514 Sep. 15,199
2.5, 293, 457 Mar. 8, 1994. ) Will be briefly described. FIG. 7 shows a block configuration example of a conventional neural network LSI. Neuron circuits arranged in one row are arranged on each of the four sides of the chip, and synapses are arranged and arranged in a matrix in most of the other areas inside the chip. With these arrangements and interconnection wiring, a neural network for associative memory can be efficiently constructed. FIG. 8 shows an example of a synapse circuit having a learning function. Capacitor C1
The synapse load value (Wij) is represented by the accumulated charge amount of. The accumulated charge amount of C1 is determined by a weight correction circuit composed of a charge pump circuit and a learning control circuit that gives a correction signal to the learning rule (ΔWij = ± ηSiSj, where η is a learning coefficient and corresponds to the number of pulses given to ACP +/−. Will be corrected according to Si and Sj respectively correspond to the output signals of neurons i and j that give a signal to this synapse. In this precedent example, a symmetric synaptic connection (Wij
= Wji), two synapse connection arithmetic circuits are mounted on one synapse load value. Figure 9
Shows an example of a neuron circuit. The output signal current from the synapse is added at the common node (Kirchhoff adder), the signal is converted into a voltage, and the threshold value (Vref) of the neuron is compared with the comparator. The two selectors (SEL1 / 2) selectively output the output of the comparator or the teacher data (SR (T)) according to the attribute data in the register (SR (P)) in the present neuron circuit and the learning control signal IselS. . According to this prior art, a synapse circuit having a learning function can be mounted in a relatively highly integrated manner. Actually 0.8 μm CMOS
Using technology, we succeeded in integrating 80,000 synapses and 400 neurons on one chip (Y. Ar.
ima, et al. "A Refreshable
Analog VLSINeural Network
Chip with 400 Neuronsand
40K Synapses, "IEEE, Journal
al of Solid-State Circuit
s, Vol. 27, No. 12, pp. 1854-18
61, Dec. , 1992. ). In addition, using this prior art, the current state-of-the-art technology 0.15 μm CMOS
With, it is possible to integrate about 2 million synapses and about 2000 neurons on one chip. In that case, associative storage of about 300 patterns is possible, but the storage capacity is insufficient for practical use. Therefore, we considered further high integration of the synapse circuit with a learning function,
This time, we devised a synapse circuit using the previously invented A-MOS device.

[0004]

In the present invention, the A-MOS device (Adjustable β-M) previously invented is used.
OS: Abbreviation A-MOS (Patent "semiconductor element" inventor:
Yu Arima Application: January 26, 2001 Japanese Patent Application 2001-
018133), a highly integrated A with a learning function
-Proposing MOS synapses. The A-MOS device has the device configuration shown in FIG. 10, and the gain coefficient β can be analog-modulated by adjusting the voltage of the control gate. The modulation characteristic of the gain coefficient β can be set by the element shape parameter shown in FIG. FIG. 12 is a diagram for explaining the β modulation principle in the A-MOS. The left diagram in FIG. 12 shows a case where the conductance of the control gate channel is made equal to that of the normal gate, and a thin shaded portion shows an effective gate region. 12
The inner right figure shows the case where the conductance of the control gate channel is made sufficiently larger than that of the normal gate. As can be seen from these figures, the A-MOS can modulate the effective gate length L and gate width W in an analog manner by changing the control gate voltage. As a result, the A-MOS realizes analog modulation of the gain coefficient β by the control gate voltage. In the A-MOS, the source / drain current Isd is approximately proportional to the following equation, and therefore, by providing a load having a diode characteristic on the drain side, an integrated value (approximation) of the normal gate voltage Vg and the control gate voltage Vcg can be output. it can. Isd∝Vg ² × Vcg ^{1-2 If} the drains of a plurality of A-MOSs are connected at a common node and currents are added, a product-sum operation can be expressed.

[0005]

The A-MOS synapse according to the present invention has a simple circuit configuration and a compact circuit size, which makes it a DRAM.
It is possible to achieve an average degree of integration. Using the current state-of-the-art technology (0.15 μm CMOS), it is possible to integrate about 1 G synapses on one chip, and a fully connected neural network of about 30,000 neurons can be realized on one chip. This corresponds to a network scale capable of associatively storing about 5000 patterns.

[0006]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention is characterized in that an A-MOS is used in a synapse circuit which constitutes a neural network. In the circuit configuration of the present invention, the synapse weight value is expressed by the potential of the A-MOS control gate. The product of the synapse weight value (control gate voltage) and the neuron signal (input gate voltage) is realized by the β variable characteristic of the A-MOS. The A-MOS control gate is floating, and by adjusting the learning control voltage capacitively coupled to it, the synapse load value can be corrected by injecting hot electrons to realize the learning function. FIG. 1 shows a circuit configuration example of an A-MOS synapse. The output signal Sj of another neuron is connected to the input gate of the A-MOS, and the synapse weight value Wij is expressed by the charge accumulated in the control gate of the A-MOS. As a result of this circuit configuration, the source / drain current Isd flowing through the A-MOS transistor changes corresponding to the product value of the output signal Sj of the neuron and the synapse weight value Wij. As shown in FIG. 2, the control gate of the A-MOS is floated and capacitively coupled with the learning control voltage Vc node.
In addition, the drain node of the A-MOS is connected to a plurality of A-MOSs.
A neural network can be configured by a circuit configuration in which the node is an axon signal line and the sum of currents is an input signal to a neuron, which are commonly connected with each other. As shown in Figure 2,
The comparator that determines and outputs the output signal of the neuron has p-M connected to the threshold Vref and the common node.
OS transistor Tr. The voltage generated at B is given. Further, the common node M has a p-MOS transistor Tr. The drain of R is also connected. Tr. An inverted delay signal (delay time:
td), the neuron signal is turned on, and after a certain delay time, Tr. R is turned on, the potential of the common node rises, and the output of the neuron is forced off. After a certain delay time after that,
Tr. R is turned off, and after a time corresponding to the driving ability of the A-MOS synapse connected to the common node, the output of the neuron i is turned on again, and a series of operations are repeated. That is, the pulse generation frequency of the output signal of the neuron is modulated according to the sum of the currents flowing through the common node. FIG. 3 shows an example of behavior of various signals related to signal output of the neuron. The pulse signal period T of the output signal Si of the neuron is given by the following equation. T = 2td + tr + tw where tr is Tr. Corresponding to the delay time for the common node M by R, tw is the common node M by all synapses.
It corresponds to the delay time for. Therefore, as the total signal from the synapse increases (the current increases), tw becomes smaller and T becomes shorter, so that the frequency of occurrence of the output pulse signal of the neuron becomes larger with the total signal from the synapse. . Next, FIG. 4 shows various signal states at the time of learning the neural network for storing the associative pattern, that is, at the time of correcting the synapse weight value. Modifying the synapse weight value corresponds to modifying the amount of charge accumulated in the control gate of each A-MOS synapse. To correct the amount of charge accumulated in the control gate, the control voltage Vc and Tr. R and the power supply voltage VdH of the delay circuit
By appropriately controlling and, the learning rule (ΔWij = ± ηSi
It is possible to execute the correction of the accumulated charge amount of the control gate according to Sj). First, during learning, the power supply voltage VdH is changed to Vd
Use a higher value. As a result, the A-MOS
Allows hot carrier injection near the drain.
At this time, the threshold voltage Vref of the comparator is also changed corresponding to the control voltage Vc. hebb's learning rule (Δ
Wij = + ηSiSj), the control voltage V
Set c to "Low" (to GND). By doing so, the potential of the control gate shifts to a low value, and as can be seen from the hot carrier injection characteristic example shown in FIG. 5, hole injection by avalanche becomes dominant, and synapse only occurs when both Si and Sj are ON. The load value will increase.
Next, the anti-hebb learning rule (ΔWij = −ηSiSj)
When executing the control voltage, the control voltage Vc is set to "High" (~ V
dH) and by shifting the potential of the control gate to a high value, the injection of channel hot electrons becomes dominant, and the synapse load value can be reduced only when both Si and Sj are ON. The potential states in the synapse element when each learning rule is executed are shown in FIG.

[Brief description of drawings]

FIG. 1 shows an example of a basic element configuration circuit diagram using an A-MOS in a synapse circuit of the present invention.

FIG. 2 shows a circuit configuration example using an A-MOS in the synapse circuit of the present invention.

FIG. 3 shows an example of behavior of various signals.

FIG. 4 shows an example of states of various signals during learning.

FIG. 5 shows an example of hot electron injection characteristics.

FIG. 6 shows correction of synapse load value during learning.

FIG. 7 shows a block diagram of a conventional neural network LSI block.

FIG. 8 shows a conventional synapse circuit configuration diagram.

FIG. 9 shows a conventional neuron circuit configuration diagram.

FIG. 10 shows an example of an A-MOS device configuration diagram.

FIG. 11 shows an example of A-MOS device configuration parameters.

FIG. 12 shows an example of a β-modulation diagram of an A-MOS. .

[Explanation of symbols]

A-MOS: Adjustable β-MOS (Japanese Patent Application No. 2001-018133) Wij: Control gate Sj: Normal gate GND: Ground Vd: Power supply voltage Vc: Control voltage Cv: Capacitor voltage Vref: Neuro threshold value of synapse output signal current Voltage W: Effective gate width L: Effective gate length θ: Angle between normal gate and control gate

Claims (5)

[Claims] 1. An element for realizing the function of a synapse, which is an element of a neural network, wherein an output signal of another neuron is applied to the gate of a MOS transistor capable of analog-voltage-modulating a gain coefficient β for β modulation thereof. A synapse element characterized by a functional configuration in which the potential of the control gate is used as a synapse load value and the source / drain current of the transistor is used as an output signal of the synapse. 2. In a semiconductor integrated circuit which constitutes a neural network, at least one synapse function,
A semiconductor integrated circuit device comprising the synapse element according to claim 1. 3. The semiconductor integrated circuit device according to claim 2, wherein the β modulation control gate in the synapse element is connected to a control signal node common to a plurality of synapse elements via a capacitor. 4. A semiconductor integrated circuit device comprising the circuit configuration shown in FIG. 5. The synapse element according to claim 1, wherein β
A synapse element with a synapse load value correction function, characterized in that the amount of charge accumulated in the modulation control gate is corrected by hot carriers.