JPH02232725A - Neurocomputer having multidummy node - Google Patents

Abstract

PURPOSE:To transmit and receive data with a small number of wirings and to extend the dynamic range by providing a neural network consisting of gathering of analog neuroprocessors, a control pattern memory, etc. CONSTITUTION:Plural analog neuroprocessors (ANP) are used to constitute a hierarchical or feedback neural network 18. The control signal outputted from a control pattern memory 12 to which the access address is given by a sequencer 13 is applied to the network 18. Weight data obtained by learning or the like is supplied to the network 18 from a weight memory 14. The network 18, the memory 12, the sequencer 13, and the memory 14 are controlled and managed by the digital signal of a digital control means 15. A dummy node means 19A and a multidummy node control means 19B are provided to not only extend the dynamic range of weight data in the fixed point system but also quickly transmit and receive data with a small number of wirings.

Description

[Detailed description of the invention]

[Summary] This is a neurocomputer that is realized by connecting analog neurotisubs using an analog time-division transmission line and that expands the dynamic range of weight data expressed in a fixed-point system, and constructs a hierarchical network with a small number of wires. It is possible to exchange data between basic UniSotos,
The aim is to realize a neurocomputer that can expand the dynamic range of a fixed-point system. Analog signals are time-divisionally input from the first analog bus, and the analog signal is converted to the first A neural network consisting of a set of analog gnuroprocessors that output to the second analog bus, a control pattern memory that stores III control information of the neural network, and a signal for accessing the address of the control pattern memory. In a neurocomputer that has a sequencer and a weight memory that is controlled by the sequencer and stores weight data for the analog neuroprocessor, when the output voltage value of the analog bus of each layer exceeds a predetermined dynamic range, Multiplying data multiplied by a fixed voltage from a dummy node means connected to an analog bus on the input side of the multi-channel controller is expressed as the sum of a plurality of weight data, and each divided weight data is stored independently in a weight memory. It is configured to have a dummy node control means. (Industrial Icheon Field) The present invention relates to neuro-computers, and more specifically, the present invention relates to neuro-computers, and more specifically, the present invention is realized by connecting analog neuro-chips through an analog time-division transmission line, and the dynamic range of weight data expressed in a fixed-point system is [Prior Art] With conventional sequential processing computers (Neumann type computers), it is difficult to adjust the data processing function of the computer according to changes in the usage method or environment. As a data processing method, a new parallel distributed processing method using a hierarchical network has been proposed.In particular, a processing method called the pack broadcasting method (D.
B. Rumelhart, G. E. Hinton, a.
nd. R. J. Williaas+ @Learni
ngInLernal Representation
s by Error Propagation +P
ARALLEL DISTRIBUTED PROCE
SS[NG, Vol. 1. 9P. 318-364,
The MIT Press, 1986) is attracting attention because of its high practicality. In the backprovergation method, a hierarchical network is constructed from a type of node called a basic unit and internal connections with weights. FIG. 38 shows the basic configuration of the basic unit 1. This basic Unisoto 1 performs a process similar to the continuous two-aeron model. In other words, this is a large-power single-output system, including a multiplication processing unit 2 that multiplies a plurality of inputs {Yh ) by the respective internal connection weights {wt h I, and an accumulation unit that adds all the multiplication results. A processing unit 3 performs nonlinear threshold processing on this added value to produce one final output X. and a dark value processing section 4 that outputs. FIG. 39 is a conceptual diagram of the structure of a hierarchical neural network. A number of elementary units of the configuration {1-h, 1-i,
1-j) are connected hierarchically as shown in FIG. 39, so that an output signal pattern corresponding to the input signal pattern is output. During learning, the weights of connections between each layer (Wi
h) is determined. Such learning is performed for multiple input patterns and multiplexed. Further, during associative processing, even if the input pattern is incomplete information that is slightly different from the complete information input during learning, an output pattern close to the teacher pattern during learning can be obtained, making associative processing possible. [Problem to be solved by the invention] In order to make a neurocomputer with such a configuration a reality, it is necessary to transmit and receive data between the basic units 1 that constitute the hierarchical network using as few wiring lines as possible. We need to realize this. This means that
In order to realize complex data processing, it is necessary to make the hierarchical network structure more multi-layered and increase the number of basic units, and this is an issue that must be solved by all means. It is one of the. However, in the data transfer method explained earlier, the third
As is clear from the hierarchical network configuration shown in Figure 9, the number of wires between the two eyebrows is extremely large.
When a hierarchical network is divided into chips, there are problems in that it cannot be made smaller and reliability cannot be increased. For example, if the number of basic units in two adjacent layers is the same and we assume a perfect connection in which all basic units 1 are connected to each other, the number of wires will increase in proportion to the square of the number of basic units. become. In this way, the number of wires increases rapidly. Furthermore, when data is expressed using a fixed-point system with a decimal point after the most significant bit, numbers greater than 1 cannot be expressed, so there is a problem that the Guinamin range of the data cannot be widened. SUMMARY OF THE INVENTION The present invention aims to realize a neurocomputer that can realize the exchange of data between basic UniSotos that constitute a hierarchical network with a small number of wires, and that can expand the dynamic range of a fixed-point system. It is. [Means for Solving the Problems] Figure 11A is a system block diagram of the neurocomputer of the present invention. The neural network 18 inputs analog signals in a time-division manner from a common first analog bus on the input side of each layer, performs a sum-of-products operation using digital weight data, and transfers the analog signals to the common first analog bus on the output side of that layer. Analog neuroprocessor (hereinafter referred to as AN) outputs to the second analog bus of
It is a network with at least one or more layers, which is a set of layers (denoted as P). A control pattern memory 12 stores patterns of control signals of the two-aerial network. ! The weight memory 14 stores weight data. Sequencer 13 generates addresses for control pattern memory 12 and weight memory 14. The digital control means 15 is a general-purpose processing device having an MPU and a main memory, and includes a network 18, D/A, and A/
It is connected via D converters 16 and 17, and controls the entire neural network, control pattern memory, sequencer, and weight memory. The present invention configures a neurocomputer system in this manner. By providing the dummy node means 19A and the multi-dummy node iFiIL control means 19B, even if the weight data for the dummy node becomes 1 or more, this weight data is expressed as the sum of a plurality of weight data. [Operation] Inputs an analog input signal to the analog neurochip in a time-division manner, calculates the product of this signal and weight data, adds the product signals, and outputs the product-sum signal obtained through the nonlinear function circuit. This constitutes an analog neurochip. Hierarchical or feedback neural network 18 using multiple analog neurochips
, and a signal is applied to this neural network 18 by the sequencer 13 giving the address to be accessed. Further, the neural network 18 is supplied with weight data obtained through learning or the like from the weight memory l4. The neural network 1B, control pattern memory 12, sequencer 13, and weight memory 14 are controlled and managed by digital signals from the digital control means 15. Further, the MPU in the digital control means 15 particularly executes a learning algorithm and checks output signals. In this way,
An analog neurocomputer system characterized by using time-division analog input signals and time-division analog output signals is realized. Furthermore, one or more weight data for a dummy node can be expressed using a fixed point system by dividing it into pieces of data for a plurality of dummy nodes, and as a result, the dynamic range of the weight data based on the fixed point system can be expanded. (Example) Hereinafter, an example of the present invention will be described with reference to the drawings. FIG. IB is a schematic diagram of a dual in-line package of an analog neuroprocessor (ANP) 11 comprised of the neurochip of the present invention. This is MB44
42 and executes Nieron model processing. The internal threshold processing section is a model replaced by a sigmoid function. a. The analog neurochip is called ANP.
A device that inputs and outputs analog data. Figure 1C is an internal configuration diagram of the ANP of the present invention. As shown in FIG.
Connected between 2. The ANP 11 includes an analog multiplier 22 that multiplies input analog signals and weights, an analog adder 23 that calculates the sum of bonds, a sample/hold unit 24 that holds the sum, and a nonlinear function unit 2 that outputs the value of the sigmoid function.
Consists of 5. Each terminal of ANP II in Figure 1B will be explained. . The inside of the ANP 11 is composed of an analog circuit section and a digital circuit section. The 1-6 volt terminal is the power supply terminal supplied to the operational amplifier of the analog circuit section. D in and D out are terminals for analog input signals and output signals. AGND is the ground terminal of the analog circuit section. Rt+ and R

[One terminal is the terminal of the external resistor R of the integrating circuit in the analog circuit section, and the Ct+ and Ct terminals are the terminals of the external capacitor C of the integrating circuit. DGND is a ground terminal of the digital circuit section. +5 volts is the R terminal of the digital circuit section. R.S.
T is a reset signal terminal for resetting the charge of the capacitor of the integrating circuit. CSI and CSO are input/output terminals for daisy chain control signals, OC is a terminal for offset cancel Il control signals, S/H terminal is a control signal terminal for sample/hold, and SYNC is a synchronization signal for processing of each layer. terminal, DCLK is a basic clock signal terminal for processing analog input signals, WCLK is a clock terminal for taking in digital weight data, W
D is a terminal for digital weight data that is input bit serially. FIG. 2 shows the analog neuroprocessor (AN) of the present invention.
It is a principle block diagram of P). Analog input signals sent in a time-sharing manner from separate ANPs (not shown) are routed from analog bus Bl to ANP l.
The analog input signal and the digital weight data are input to the analog multiplier 22 in the l, and the analog multiplier 22 multiplies the digital weight data WD which is input bit serially through the shift register 27 and then converted into serial/parallel. Obtain a product signal indicating the product of . The next analog adder 23 is a Miller integration circuit consisting of an external resistor R and a capacitor C, and is connected to a plurality of ANPs in the previous stage connected to the analog bus Bl (the place where an ANP exists is called a node). This method calculates the sum of the product signals obtained from the analog input signal sent in a time-division manner and the threshold analog input signal sent from the dummy node. Next, after the output signal is held in the sample/hold section 24 to wait a desired time, the sampled/held output is further converted via the nonlinear function section 25. The output control unit 26 outputs the analog . Output signal D. Output uL to analog bus B2. Note that the sequence generator 28 also generates control signals that are supplied internally. In the phase control section 29, the AN
This is to control the phase of the control signal so that each switch connecting the analog circuit section and the digital circuit section in P is reliably turned on or off. In particular, when the first switch is on, the second switch When turning off the switches, the phase of the control signal is controlled so that the switches do not turn on at the same time. Note that the sequence generator 28 receives a reset signal R.
ST. ．． DCLKSWCLK, SYNC, S/H, QC
, CSI are inputted from a mask control blocker to be described later, and CSO is outputted to generate an internal control signal of the ANP. Neural networks need to perform high-speed calculations through simultaneous processing. Although time-division data is used in the present invention, in a steady state, each ANP performs simultaneous processing in a pipeline manner. An ideal neural network would require wires that interconnect each neuron with each other, but if the system were to be implemented as is, the number of wires would be large. Therefore, in the present invention, since time-division data is handled, the processing time for the sum of products in each ANP increases, but by arranging the chips in parallel in the vertical direction, that is, in the direction of the same layer, the neurochips in the layer can be AN to configure
Simultaneous processing of P improves the processing time. Also,
Each layer can be pipelined, which also reduces processing time. For example, when an input is input to each of three neurochips connected to an analog bus, it is input to all three at the same time, and in parallel, each ANP generates a product of the weight and the analog voltage. and holds it as a charge in the integrator capacitor. Then, in the next time period, for the analog input of the same analog bus, each ANP will form a product with a weight and add it to the product determined in the previous time period in the integrator capacitor. After the sum is generated for the product of the weights on the analog input signals from all previous ANPs, the sum is sampled/held. Thereafter, it is output via a sigmoid function, which is output when the CSI control signal is input. Then, when the output is completed, the CSI falls, and after a certain time delay, the CSO rises, giving the right to use the output bus to the ANP consisting of two adjacent aero chips in the same layer. Figure 3 is a configuration diagram of the first embodiment of the basic unit, which is a neurochip. The multiplication section 32, addition section 33, and threshold processing section 34 in the figure are continuous neuron model execution sections, but in this embodiment, an output holding section 35 is present. Specifically, a plurality of inputs connected to the basic unit 31 are connected to Yi1.
If the weight set corresponding to each connection is Wi, then the multiplier 32 calculates Yi−Wi, and the adder 33 calculates X=ΣY i −W i10. However, θ is a threshold value. If the final output of the threshold unit 34 is Y, then Y=1/(1+exp(-X))...
・11) will be calculated. 1-θ to the value 1+1″ input from the dummy node
”, and the adder 33 outputs the result of “×−θ”. Therefore, in the threshold section 34, only the conversion using the S-shaped curve is performed. The multiplication section 32 is composed of a multiplication type D/A converter 32a, and receives an input of an analog signal (input via an input switch section 37) from the basic unit 31 in the previous layer or from a dummy node circuit described later. , the input is multiplied by weight information of the digital signal to be multiplied (input via the weight holding unit 38 described later), and the obtained multiplication result is processed to be output as an analog signal. The addition section 33 includes an analog adder 33a formed of an integrator and a holding circuit 33b that holds the addition result of the analog adder 33a. Multiplying D/A
The converter 32a inputs an analog input signal to the reference voltage terminal of the D/A converter, and inputs each bit of weight to each digital input terminal as a digital input signal. Generate the product of the log input signal and the weights. The analog H adder 33a is a multiplication type D
A new addition value is obtained by adding the output of the /A converter 32a and the addition value obtained last time and held in the holding circuit 33b.The holding circuit 33b adds the addition value obtained by the analog addition B 33a. At the same time, analog addition W33a is performed using the hold value as the previous addition value.
This is to provide feedback. These addition processes are executed in synchronization with an addition control signal output from the control circuit 39. The threshold unit 34 includes a nonlinear function generation circuit 34a that is an analog function generation circuit, and outputs a nonlinear signal such as a sigmoid function in response to an input. When the accumulation of the multiplication results including the addition of the threshold value (1θ) is completed, the dark value (1θ) is added to the addition value X held in the holding circuit 33b (the sigmoid function of equation 13 The output holding unit 35, which performs arithmetic processing and obtains the analog output value Y, is composed of a sample and hold circuit, and outputs the output value Y of the analog signal of the nonlinear function generation circuit 34a to the basic Unison 31 in the subsequent layer. In addition, 36 is an output switch section, which is turned on for a certain period of time in response to an output control signal from the control circuit 39, and transfers the final output held by the output holding section 35 to the analog bus B2. 37 is an input switch section which is turned ON when an analog output from the final output is sent from the basic unit 31 in the previous layer in response to an input control signal from the control circuit 39. 38 is a weight holding unit, which is composed of a parallel out shift register, etc., and the bit-serial weight signal sent from the weight memory is transferred to the gate of the buffer 38a (weighted by the control circuit 39). When the input control signal is turned on), this weight signal is held as the bit parallel weight required by the multiplier 32.The bit parallel weight is multiplied in parallel when the input control signal is given. A control circuit 39 of the digital circuit section generates an internal synchronization signal from an external synchronization signal, and controls the internal analog processing function. , the input and output of the basic unit 31 that adopts the signal processing configuration shown in FIG. Alternatively, if the weight information is composed of an analog signal, the multiplication type D/A converter 32a (7) may be used. Alternatively, an analog multiplier can be used. Fig. 4 is a specific circuit diagram of an embodiment of the neurochip (ANP) of the present invention of 1@. This unit includes an input section 42, a multiplication section 43, an addition section 43, and an addition section 42. Section 44
, a sample/hold section 45, a nonlinear function section 46, and an output section 47. Here, it is assumed that there is no output holding circuit and that the sample/hold section 45 also has an output holding function. The input section 42 is composed of an offset canceling section 5l and a 1x buffer 49. The 1x buffer 49 is a voltage follower and is configured by feeding back the output of the operational amplifier to one terminal and inputting the input voltage to the tenth terminal. The data input is an analog time-shared pulse signal. OC is an offset control signal, and when this is 1, the analog switch 66 is turned on, and the 1x buffer 4 is turned on.
9 is forcibly set to 0 voltage. On the other hand, when the offset control signal OC is 0, the analog switch 66 is turned off, the other analog switch 65 is turned on, and the data input is input to the 1x buffer 49. That is, when the offset control signal OC is 1, 0 volt is forcibly input to the neuron unit, and the neuron unit performs an operation of canceling the offset voltage generated at the operational amplifier output of the circuit up to the multiplier output. ing. In the figure, the switching of the analog switches 65 and 66 is controlled by the inverted phase and the positive phase of the OC signal, but a phase control circuit prevents them from being turned on at the same time. Hereinafter, this will be referred to as the OC being "phase controlled." The positive/negative switching circuit 52 is constructed by cascading two multipliers. In the multiplier, the input resistance (IOKΩ) and the feedback resistance (10KΩ) form a 10/10, that is, an inverted voltage of 1, which is
The sign of the analog voltage is determined depending on whether only one stage or two stages are passed. The control signal is the sign of the digital weight data.
(SION), and this SIGN bit is connected to the gate of MOS switch 70. This SIGN control signal is also phase controlled. When the sign bit is 1, the input voltage from the input section 42 is inverted by the first stage multiplier, and since the switch 67 is also turned on, the multiplier in the subsequent stage is also passed, resulting in a positive phase. Further, when the sign bit is O, the switch 69 is turned on via the inversion circuit 68. At this time, the switches 67 and 70 are off, so the input voltage from the input section 42 is applied to the switch 69.
The signal is inputted to one terminal of the operational amplifier 71 at the subsequent stage via the . Therefore, a multiplier is formed by the resistor 72 at the front stage and the resistor 73 of the feedback node of the operational amplifier at the rear stage, and the signal is multiplied by 1 and inverted. That is, the input to the input unit 42 is formed as a positive or negative voltage depending on the sign bit, and this becomes a voltage according to the excitatory and inhibitory synaptic coupling. The output from the positive/negative switching circuit 52 is sent to the multiplier 43
First, we will explain the R-2R type D/A converter, which is input to the 74 points of the R-2R resistor network of the D/A converter 53 in the MS, that is, the reference voltage terminal.
The internal switch is turned on or off depending on the digital weight from I3 to LSB. Digital I [α is 1
, the current flows through the right swing 75 to the virtual ground point 7B of the operational amplifier 76. The virtual ground point 78 of the operational amplifier 76 is controlled to have the same voltage as the ten terminal, and since this is the ground, it is a virtual O volt. In the D/A converter 53, R is IOKΩ, 2R is 2
It is 0KΩ. Regardless of the state of the switch, a current flows through the resistor 2R, and it is determined whether the weight current flowing through the 2R flows toward the virtual ground point 78 according to the value of the digital value. Let i be the current flowing through the rightmost 2R. The current of 2R corresponding to the second from the right, that is, LSB, is 1
Since it is the value obtained by dividing the voltage applied to the rightmost 2R by 2R, 2
RXi÷2R becomes i. Therefore, current 21 flows through the rightmost lateral direction R. 2RX is in the third 2R from the right
Since a voltage of i+RX2 i is applied and this is divided by 2R, a current of 21 flows. Same as under pi, 4 as you go to the left.
i, 8f, resulting in a current that increases by a power of 2. It is from MSB to LSB that determines whether or not the weighted current, which is a power of 2, flows toward the operational amplifier. Therefore, a current corresponding to the digital weight flows into the virtual ground 78 in the form of a power of two, and since the input impedance of the operational amplifier 76 is infinite, this current flows into the feedback resistor 78 of the operational amplifier 36. Therefore, the output voltage of the D/A converter ■. 1 is + . is equivalently a value obtained by multiplying the input voltage E by weight.The weighting coefficient is controlled by the digital value input from MSB to LSB.On the other hand, the adder 44 is a time-division multiplexed analog A cumulative addition operation is performed by time-sharingly using a Miller integrator for each product of each voltage of the signal and the digital weight data.
A hold circuit 45 samples/holds the addition result. Next, the adding section 44 will be explained. The adder 44 is an integrator including a resistor R and a feedback capacitor C. The input section of the adder section 44 has a time division addition@control section 55, and when the phase-controlled sample/hold signal S/Hi is 1, the output voltage of the multiplier section 43 is input to the virtual ground point 79 of the operational amplifier. and S
When the /H signal is 0, the inverting circuit 80 causes a switch of 7,000 8]
is turned on and the output of the multiplier 43 is connected to the ground via the resistor R, so that it is not added to the feedback capacitor C of the adder 44. Now, when the S/H signal is 1,
The output voltage of the multiplier 43 is applied to the operational amplifier 10 via a resistor R.
A current obtained by dividing the input voltage by the resistor R is input to the feedback capacitor C via virtual ground. After this, the S/H signal becomes 0 again and the multiplier 43 and the adder 44 are separated, so the multiplier 43 can multiply the next input signal by the weight data. The feedback circuit 82 of the integrating circuit including the capacitor C is provided with an offset canceling function using four switches. If the OFF control signal OC is now 1, switches 83 and 84 are on, and switches 85 and 86 are off. When the offset control signal OC is O,
Data input section 42, data input terminal DATA-IN
An input voltage is given to PUT, and the corresponding multiplier 4
The output of No. 3 is input to the capacitor C via the resistor R. At this time, the switch 85 and 86 are on, and the polarity of the capacitor C is - the side connected to one terminal of the operational amplifier.
, the side connected to the output of the operational amplifier 102 is 10. Next, if the offset control signal OC is 1, the data input is forced to 0. In this case, if there is no offset through the positive/negative switching circuit 42 and the D/A converter 53 of the multiplier 43, the output of the D/A converter 44 will be 0 volts. However, due to the presence of the operational amplifiers 49, 103, 71, and 102, an offset voltage is generated, and the offset voltage is stored in the capacitor C of the adding section 44. In this case, unlike the previous case where the offset control signal OC was O, the offset control signal OC was 83.8.
4 is turned on, and the polarity of capacitor C is reversed. Therefore, by setting the offset control signal OC to 1, the offset voltage that occurs when the input signal is input changes the polarity of the capacitor C, and as a result,
Offcents will be cancelled. In this way, the present invention is configured to equivalently have an offset cancellation function by using the polarity reversal of the capacitor C. The switch 87 is controlled by a reset signal, and when the reset signal is applied at the start of processing, it sets the voltage of the capacitor Ch to zero and forcibly resets the output of the adder to O. It is assumed that this OC signal is also phase controlled. The output of the adder 44 becomes the input of the sample/hold circuit 45. In the sample/hold unit 45, when the phase-controlled sample/hold control signal S/H o+at is 1, the output of the adder 44 is stored in the capacitor Ch via the switch 88. When the S/Hot+t control signal is 1, the control signal of the switch 90 becomes 0 due to the inversion circuit FIs94, one terminal of the capacitor C is not grounded, and the unit is turned on by turning on the switch 91. The final output signal of C is passed through the switch 91 to the capacitor C
It is input to h. That is, the final output signal at that time is fed back from the output terminal of the operational amplifier 96 and applied to the lower side of the capacitor Cb. Therefore, the voltage obtained by subtracting the value of the final output signal from the output of the adder 44 is held in the capacitor C. On the other hand, the S/H 0-t control signal is 0
When , switches 89 and 90 are turned on, the lower side of capacitor C becomes ground, and as a result, capacitor C
The voltage stored in the adder 44, that is, the voltage value obtained by subtracting the final output value from the output of the adder 44, is input to the positive side of the 1x operational amplifier 93 via the switch 89, and this operational amplifier 93 works as a buffer to The output of 93 becomes the input of the sigmoid function. Also, S / H 0-
The switch 88 is turned on when the L control signal is 1, and the switch 92 is turned on when the voltage difference between the output value of the adder and the final output value is stored in the capacitor CI. Therefore, 0 volt is forcibly input to the operational amplifier 93. At this time, the offset voltage Δ■ is input to the lower side of Ch via the sigmoid function 46, operational amplifier 96, and analog switch l00 via the switch 91. Therefore, when the S/HouL control signal is 0, that is, when the switch 89 is on and the switch 92 is off, the voltage stored in Ch, that is, (the output offset voltage ΔV of the adder) is the same as that of the operational amplifier 93 and the sigmoid It becomes the final output via the function 46, but the S/H. ．． ．． When the signal becomes 1, the offset voltage generated at this time is also ΔV
As a result, the offset voltage will be canceled. The nonlinear function section that generates the sigmoid function has a nonlinear circuit selection control section, and the phase-controlled SelSig signal is
When the switch 95 is turned on, the output of the sigmoid function is input to the next stage. However, when the SelSig signal is 0, the control signal for the switch 98 becomes 1 via the inversion circuit 97, turning it on and cutting off the output of the sigmoid function. That is, when the SelSig signal is O, the output voltage of the sample/hold section is directly input to the operational amplifier 96 without going through the sigmoid function. operational amplifier 96
is essentially a 1x operational amplifier that feeds the output directly back to one terminal, acting as a buffer. That is, the output impedance -V
This buffer serves as a buffer to reduce the A time division analog output section 64 and an output control section 63 are connected to the output section 47 . When CSI is 1, switch 99 is on and switch 101 is also on, so the final output value of operational amplifier 96 is DATA - OUTPUT
The operational amplifier 96 functions as a 1x operational amplifier by being outputted to and fed back to one terminal of the operational amplifier. At the same time, the final output value is fed back to the sample/hold section 45. On the other hand, when CSI is 0, switch lOO is turned on via inverter 104, and switch 101.99 is turned off. In other words, the output of the operational amplifier 96 is not output to the DATA-OUTPUT line. However, since a 1x bath sofa is formed by turning on the switch 100, the voltage follower operation of the operational amplifier 96 is performed without being destroyed. The output section 47 is a circuit that determines whether or not to transmit the output pulse voltage based on the output control input signal CSI. This CSI is sent to CSo via delay circuit 105.
, and determines the time timing of the output analog signal to adjacent nuclear chips in the layer. Therefore, in the present invention, the analog signal from the output section 47 is transmitted in a time-division manner, so that it does not compete with analog signals from other chips on the bus. FIG. 5 shows offset cancel OC in FIG. 4, OCO, OC1, sign SION4: PN, -PN, sample/hold SH in SHII, SHIO, sample/hold S/H. ．． tISH2L SH20, sigmoid selection signal SelSig °-S IGM, S EG
Phase control of the M and daisy chain signal CSI is realized using two signals, cs and -cs. In other words, by configuring one control signal as two signals, one in positive phase and the other in reverse phase, and shifting the phases, other switches controlled by the positive and negative phases of these control signals will not turn on at the same time. This is an example in which a signal is generated as shown in FIG. Note that capacitors C, , connected to the output terminal of the D/A converter 53
The resistor R, connects the feedback signal of the operational amplifier 76 to D/
This is to match the calculation speed of the A converter.
The digital input of the D/A converter is applied to the DT terminal. In FIG. 5, the same parts as in FIG. 4 are given the same numbers, and the explanation will be omitted. FIG. 6 is a timing diagram for an integrator based on the weighted error method of the present invention. Datakuguchitsuta DC
LK and the weighted clock WCLK are basic operating clocks, and the high-speed weighted clock WCLK is output during a half cycle of the high state of the data clock DCLK. weight clock W
The CLK signal is a synchronous clock for capturing weighted serial data. The data clock DCLK signal is a basic clock for processing analog input signals. The synchronization signal SYNC is a synchronization signal that synchronizes each analog neuron processor ANP in one layer in each layer. The change in the output voltage of the integrator is shown by the waveform shown by the triangle at the bottom. The integration waveform is controlled by pulses of the sample/hold control signal SH, and while this pulse is high, an integration operation is performed. That is, when charging of the capacitor C of the integrator is started, and while the pulse of the sample/hold control signal SH is high, charge is gradually accumulated in this capacitor and the voltage increases, but the voltage of the sample/hold control signal SH is high. When the pulse becomes low and is cut off, charging operation is stopped. Therefore, only the charge amount within this integration time range is meaningful. The voltage is proportionally distributed depending on the pulse width of the sample/hold control signal, that is, the input voltage is multiplied by the integral gain. That is, when the pulse width of the input sample/hold control signal S/H is P, the voltage charged to the capacitor C is 2, and when the pulse width of the sample/hold control signal S/H is W, the charging voltage is V, '. The sample/hold control signal SH falls, the polarity of the integrator capacitor changes due to switching control, and the integrated output to which the offset has been added is inverted. and,
When the sample/hold Il control signal SH rises again while the offset control signal OC is in a high state, the offset ft pressure V, (Vb') is added to the capacitor, and by the time the SH signal falls, the offset amount is as a result. The canceled integral output value V, -Vb(
V. '-V. ’) with its polarity restored and the sample/
will be held. Next, a hierarchical neural network will be explained. 7th
Diagram A is a conceptual diagram of a hierarchical network. In the hierarchical type, input data input from the input node l10 of the input layer on the left side is processed in only one direction, sequentially toward the right side. Each neuron 112 in the intermediate layer has a dummy node 1l.
The outputs of the previous layer including 1 are received by complete connections within each layer. If there are, for example, four input nodes 110 in the input layer, one dummy node ll1 is added to each of them, and when viewed from each neuron 112 in the intermediate layer, the input layer appears to be five neurons. Here, the dummy node 111 controls the threshold, and is a value f ( X-θ). This is equivalent to changing the weight corresponding to the dummy node 111 within the neuron, but the constant value θ is generated using the max value node circuit described later. In this way, if you use the weight for the dummy node,
The threshold value can be expressed as a weight. It appears that there are four neurons in the intermediate layer starting from neuron 112 in the output layer. The input data applied to the input layer is subjected to a sum-of-products operation using weight data in intermediate layer neurons l 1 2 and il[neuron 112, respectively, to generate output data as a result. When the hierarchical structure shown in FIG. 7A is realized using the ANP of the present invention, as shown in FIG. Independent analog buses B1, B2, and B3 will be provided at the outputs of the layers. ANP in the vertical direction has a structure in which all JPs can be executed in parallel. A sample and hold circuit SH is attached to the output of the output layer. FIG. 8 is a block diagram of a neurocomputer of the present invention that implements a hierarchical neural network. Analog neuron process ANP1-5 from neurochip
are placed in parallel on each layer, and an analog bus (
Bl, B2, B3) are provided. In the same figure, ANPI
, 2.3 forms an intermediate layer, and ANP4.5 forms an output layer. Also, there is no ANP in the input stage, and the input side has a daisy circuit 17 for inputting analog input signals with good timing.
There are 1,172. The circuit indicated by S/H is a sample/
These are hold circuits 173 and 174. ANP l to 5 each require a logic signal for control, so mask control block (MCB) 1 to 8 is required.
Many control signal lines are sent from 1 to each layer. The data clock DCLK is applied to the daisy circuits 17l and 172 on the input side of all ANPs, and serves as a basic clock for analog processing. Weight clock WCLK is also given to all ANPs and input-side daisy circuits 171 and 172, and is a high-speed clock for weight data. Each ANP 4, 5 from the weight memory blocks 185, 186
Weight data is input to ANPI and ANPI, 2.3 in synchronization with the weight clock WCLK. Also, the synchronization signal SYNCI is the layer synchronization clock given to the intermediate layer ANP, and the synchronization signal SYNC2 is the output layer ANP.
This is the layer synchronization clock given to the NP. SHI and O
CI is a sample/hold system for ANP in the middle layer.
Signals and offset control signals, SH2 and OC2, are sample/hold control signals and offset control signals for the output layer ANP. The daisy circuits 171 and 172, which are the blocks on the left, are
This is an input side circuit corresponding to the input layer. In order to realize the input node, that is, the neuron in the input layer, the analog input signal given from the analog input ports O and l must be input into the circuit at the same timing as the ANP outputs the analog signal in time division. It won't happen. In other words, from the perspective of the output layer, the ANP of the output layer is 4.5, which is the ANP of the previous intermediate layer.
The basic operation is to receive analog signals from 1, 2, and 3 in a time-division manner via analog bus B2. This same relationship must exist for the intermediate layer and the input layer. The relationship between the input layer and the intermediate layer must be such that when viewed from the intermediate layer ANP, it appears that the input layer ANP is in front of it. This means that the ANP in the middle layer has the same function as the timing for outputting analog signals to the analog bus B2, and the analog input signals from the analog input ports o and i must also be output to the analog bus Bl according to a fixed rule. There is a restriction that it cannot be done. That is, analog input cover} 0.1
The input signals from the input terminals are time-divisionally routed onto the analog bus B1. The analog signal from analog input port 0 is placed on analog bus B1 at an appropriate timing, but at the next timing after being output there, the next analog input signal from analog input port 1 is placed on the same analog bus B1. In order to achieve this synchronization, the daisy circuit 171 receives an input control signal CSI that is output at a certain timing, and after a certain period of time, an output control signal CSO is output from the circuit. This CSI is output from csoi of the mask control circuit 181. Daisy circuits 1.71 and 172 are a type of delay circuit. When each daisy circuit 171 inputs the input control signal CSI from the mask control 18l, it transmits the CSO signal to the next vertically adjacent daisy circuit 172 so as to output the analog output signal of the analog input port 1. will be handed over to. This operation is called daisy control. When the CSO1 of the mask control circuit miai rises, the switch 175 is turned on, and the analog input signal of the analog input port 0 held in the sample/hold circuit 173 is transferred to the analog bus B1. Since CSO1 is CS1 of the daisy circuit 171, CSO rises after a certain period of time after CSO1 falls. This is the CSI of the daisy circuit 172 and at the same time the switch 17
6 is turned on, the analog input signal of the analog input port I held in the sample/hold circuit 174 is placed on the bus B1. This system, which has a hierarchical structure, requires this daisy control. In other words, the analog input signal is transferred from the analog input port 0 to the analog bus B via the sample/hold circuit 173.
1, the analog input signal is then output from the analog input port +-t to the same analog bus B1 via the sample/hold circuit 174. Looking at each neuron in the intermediate layer, the analog input signal from analog input port O and the next analog input signal from analog input port 1 are input sequentially in a time-division manner. Each daisy circuit 171, 172 outputs an output control signal CSO by delaying the input control signal CSI by a specific time in order to prevent bus contention on the analog bus B1. Also in the middle layer, the mask control block 181
AN that receives the output control signal CSO2 from as CSI
When P1 outputs an analog signal, it passes the CSO as CSI to ANP2, which then outputs it. AN
ANP3, which receives P2's CSO as CSI, will next output an analog signal. In short, here AN
PI, 2.3 are output in this order, and the daisy operation of the intermediate layer is completed. In parallel with this, the mask control block 181 that manages all operations controls the output layer ANP4.
When SO3 is given, ANP4 outputs, and after the output is completed, AN
When P4 gives CSO to ANP5, ANP5 outputs. The output from ANP4.5 of the output layer is the CS03 signal and ANP from the mask control block 181, respectively.
The signals are sampled/held by sample/hold circuits 177 and 178, respectively, by the daisy chain output control signal CSO from 4. This output voltage is output as an analog output signal from the analog output ports 0 and 1, and is also selected by the analog multiplexer 179 and then
/D converter 180 performs A/D conversion, and MPU 182
, a memory 183, and a communication interface 184. And MPU
At step 182, for example, it is compared with a teacher signal stored in the MPU given during learning to check whether it is a desired output signal, and based on this result, weight data in a weight memory, which will be described later, is changed. In the max value node circuit 187, dummy node control signals DCSI and DCS2 are applied from the mask control block 181 to output enables 1 and 2, and output terminals are connected to analog buses Bl and B.
Connected to 2. FIG. 9A is a timing diagram of the hierarchical neurocomputer according to the embodiment shown in FIG. Control signal lines are extracted and written for each layer. First, the data clock DCLK and the weight clock WCLK, which are basic operating clocks, enter all the ANPs in the same layer and the daisy circuits 17l and 172 on the input side at the same time. The weight clock WCLK is a serial synchronization pulse for serially sending digital data of weights, and is a synchronization clock for reading out the weights from the weight memory block. The timing at which input data is taken in is determined by each control signal. First, in the timing chart of FIG. 9A, CSO1 is the daisy chain control signal CS01 output from the mask control block 18, 1, that is, the daisy chain control signal CSI to the daisy circuit 171. Daisy circuit 17
1, the CSI inputs the first analog input signal from the analog input port 0 to the sample/hold circuit SH17.
3 to the analog bus B1. That is, the analog signal is output to the analog bus B1 at point (■) in the timing chart. At this moment, analog bus B1
Voltage is applied to the top, ANPI, ANP2. ANP3 performs a sum-of-products operation on this analog signal in parallel. The CSO passes through the daisy circuit 171, and a predetermined time after CS○ falls, the next CSI rises as shown in ■, and that CSI enters the daisy circuit 172. The next CSI is a control signal that enters the second daisy circuit 172 in the input layer. Then, while CSI is high, the analog input signal is sampled from the analog input port 1 by the sample/hold circuit S H 1
7 4 via ANPI. It is input to ANP2 and ANP3, and sum-of-products calculation is performed here. DCS 1 from mask control block 181 is a control signal to the dummy node. Each layer has a signal from a dummy node in addition to the input, so it has the form of (number of neuron nodes + 1) nodes, and the input layer has 2 inputs, but the ANP of each hidden layer
From the outside, it looks like there are 3 inputs. To explain this in terms of time, two CSIs and one DCS 1 form one control signal block. The input cycle starts with the first CSI and ends with the input to the DCSL dummy. Dummy node is max value node circuit 1
87, the circuit outputs a fixed threshold voltage on the analog bus while DCS1 is input. In other words, as shown by ■, while this voltage is being output after DCS I rises, each ANP in the intermediate layer performs the sum-of-products operation in the same way as normal inputs, and the fixed voltage is applied to the previous two analog inputs. It will be added to the result of the product-sum operation of the signals. In other words, after multiplication, addition is performed. SYNCI is the DC before CSO 1 starts up.
It becomes high at the falling edge of LK, and becomes low at the next falling edge of DCLK after DCSI rises. This is the signal that synchronizes the input layer. While WCLK is being input, the analog input and weight data are multiplied. Two peaks M1 and M2 are output to the sample/hold signal SH1 that enters the ANP of the intermediate layer, and the product is multiplied slightly before the first peak M1, and the sum is generated at the peak and held. Then, at the next peak M2, the offset voltage ■, (see FIG. 6) is subtracted and sampled/held. Such processing is sequentially repeated for all input analog signals, and the calculation of the sum of products is completed. In this case, each ANP in the middle layer, including the dummy, executes the sum-of-products operation three times. This completes the processing of each ANP in the intermediate layer, and ends the addition of products for the three inputs. In addition, in the timing chart, when DCLK is high immediately after DCS 1 falls, the sum of products is calculated for three signals from analog 2 ports 0, 1, and dummy nodes. Ch) in the sample/hold section 45 in FIG. This kind of operation is basically repeated, but A is connected to analog bus B2 between the intermediate layer and the output layer.
When to output the output signal of NP1 is determined by the rise of the signal of CS02 output from the mask control block 181. Offset control control signal O shown below SHI
C1 performs offset cancellation inside the ANP. That is, each ANP is an analog circuit that internally includes an operational amplifier, and since the circuit itself has an offset, the control signal for canceling this offset is the OC signal. As shown in the OCI, one pulse is emitted every time one checkwa calculation is executed, and offset cancellation is executed internally. In the timing chart, as shown by ■, when CSO2 rises, the signal held in ANPI is output from ANPI to analog bus B2, and while CSO2 is high, ANP4 in the output layer performs a sum-of-products operation. CSO indicated by ■
The rising edge of 2 is the timing for outputting the product-sum result of the previous input results. Next, the timing between the intermediate layer and the output layer will be explained using FIG. In addition, in the same figure, the output of the daisy chain control signal from the intermediate layer ■, ■, ■, ■ and the output from the output layer ■,
The analog signal that appears on the analog bus in synchronization with ■ is the output of the daisy chain control signal from the input layer mentioned above ■, ■, and the analog signal that appears on the analog bus in synchronization with ■ is before the processing cycle. The result will appear. The execution of Vibrine processing will be explained later, but
At the rising edge of CSO2, which is indicated by ■ in the timing chart, the output of ANPI is output. Looking at the signal in the timing chart SH2 at the rise of CSO2 shown in (2), two pulses are output. In the block diagram of FIG. 8, the SH2 signal is input to the first ANP 4 of the output layer. That is, one sum operation is performed within the ANPA for the two peak pulses of the SH2 signal. The middle layer has AN as shown in the figure.
It is assumed that there are three hidden layer neurons of PI, 2.3, but one dummy node is added by the masox value node times 1ffl187, for a total of 4 neurons. Therefore, the two peak pulses of the SH2 signal are ■
The analog signal of the intermediate layer is input to the ANP4 using the four sets of peak pulses of the SH2 signal, and the sum of products is calculated. Naturally, this operation is performed at the same time as the intermediate layer ANP is performing the product-sum operation on the input signal, and this is pipeline processing. CSO2
The signal below is the ANPI CSO signal in the middle layer,
This is the CSl for ANP2 in the same middle layer. This is the part indicated by ■. Below that is the CSO of ANP2, and below that is the CSI of ANP3, which is ■. Below that is the CSO of ANP3, and below that is the CSI of the dummy node, which is the signal output from DCS2, that is, the mask control block. CS
Looking at I, the middle layer A is in the order of Φ, ■, ■, ■.
It is input to NPI, ANP2, ANP3, and the max value node circuit 187 of the dummy node. During this time, the SH2 signal outputs four pulse signals having two peaks. That is, the neuron in the output layer of ANP4 adds four products of the input analog signal and the weight. ■ CS to ANPI in the part

When [ is input, is it ANPI?
An analog signal is sent to the analog bar between the intermediate layer and the output layer. vinegar
A signal is output to ANP4, and this signal is input to ANP4. and
At this time, the corresponding weight data is input to ANP4, and
With this, the product is performed and summed on the first peak of the SH2 signal.
sampled and held at the second peak. and this
When a total of 3 das are completed, the CSO signal will be raised from ANPI.
This becomes the CSI of ANP2. This is the state of ■
At this time, the weight data and the data on the analog bus
are multiplied and the sum is calculated. ■ has fallen
After a predetermined time, the CSI to ANP3 becomes high and is indicated by ■.
A product-sum operation is performed in ANP 4 as shown in FIG. like this
The sum of products operation is calculated in ANPA, and the mapping is performed at ■.
Fixed voltage output from the value node circuit 187
is input to ANP4, and this is stored internally.
will be added to the sum of products. The above operations are performed in parallel for ANP5 in the output layer.
It will be done. There is simultaneous processing here. Product determined by ANP4
The result of the sum operation is sent to the analog bus B3 connected to the output layer.
The output timing is mask control block 1
This is the rise of CSO3 issued from 81. Mac
value node circuit 187 outputs to analog bus B2
The control signal for this is DCS2, which corresponds to
respond. Until this DOS 2, the calculation results in the middle layer
This is the operation up to outputting. timing chart saw
The same behavior applies to signals written below this.
, and the middle layer and cascaded output-side
This is the signal pulse that defines the operation. CSO3 stands up
, the result of the product-sum operation calculated by ANP4 is output.
It turns out. In the output layer, two ANPs, ANP 4 and ANP 5, are output. For example, the rise of CSO2 in ■ is caused by ANPI.
This is an incoming signal, and its rising edge lags behind DCLK.
Ru. This is an analog input signal and digital weight data.
f! When performing calculations, digital data is transmitted using WCLK.
Serial when reading, but parallel internally
Converts digital data to analog with loading time
The input signal reaches the D/A converter, that is, the multiplication processing section
The rise of CSO2 is delayed to take into account the time it takes to
This is because it is In other words, there is a misalignment at the first head.
What we are doing is calling data, i.e. serial data.
Includes loading time. data has finished cents
It has been a while since DCLK rose.
That is, after 16 cycles of WCLK. analog squared
The calculation start time is determined by WCLK after CSO2 starts up.
This is after 8 cycles. Figure 10 shows the timing for reading digital weight data.
FIG. In the same figure,
Star clock MCLK, synchronization signal SYNC, weight cross
clock WCLK, data clock DCLK, actual weight
Data WDATA is shown. Weight data WD
A T A is read bit serially from weight memory.
16 bits are input serially. S is signbi
The bits from B14 to BO are numerical bits. In the same figure
B8, B7, B6 portions of weight data WDATA
expands downward in the diagram as a correspondence with weighted clock WCLK.
It is expressed in the form of The weight clock WCLK is
The period is 250 nsec and the duty ratio is 50%.
Ru. The address inside the sequencer starts from the falling edge of WCLK.
Weight memory after counter propagation delay time for address update
is given an address. That is, weight memory (.RAM
) is the address of bit n of m data WDATA.
This is the address of the crawl memory where the bit 7 is stored. child
After the address is confirmed, Binoto 7 reads it after tAA time.
Being left out. The change from bit 7 to bit 6 is weighted
Determined by the change to the next period of the clock, bit 6
is read out in the next cycle. 16 bits of weight data
The output is input to ANP, and the analog voltage input to ANP is
The product with pressure is calculated by internal D/A converter.
Therefore, to start inputting the analog voltage, use the data cross DC
It is input long after the rise from LK. That is,
, the analog input voltage is input to the D/A converter.
Since there is a time until the digit is reached, that time and digital
control the time that the weight data is sent internally, and
The arrival time of digital data and the arrival time of analog data are exactly the same.
It is necessary to input the analog voltage to match the For example, the rise of the analog input voltage is determined by the mmi data.
Start around B7 of , and BO of weight data is input.
After that, when all the MM data has been internally confirmed,
Control the time so that the multiplication with the analog value of starts.
It is necessary to take And the addition is performed during the period when DCLK becomes low next.
. The operating time of ANP depends on the SYNC signal, WCLK, and
It is specified by the data DCLK. And the analog input voltage is
Execute product with digital weight data from ANP input terminal
It takes a considerable amount of time to reach the voltage to the D/A converter.
Since there is an error between the CSI rise and the
starts later than the rising edge of DCLK.
Become. The mliA diagram shows the configuration of the mask control block 181.
It is a diagram. All mask control blocks 181
This is a part that summarizes the Ilm signals of. The main components are an external path interface circuit 200,
Control pattern memory 201 and microprogram sheet
Controller 202 and microcode memory 203, address
This is the creation unit 204. External bus interface circuit 20
0 is an ink face for connecting to MPU etc.
to the response line 205, data line 206 and control signal line 207.
It is connected. External bus interface circuit 200
Upper address comparison circuit 208, D-FF which is a register
209 is the upper address given from MPU etc.
is decoded and the upper address is a predetermined address.
When the lower address and data are respectively D-FF2
Launch signal from timing circuit 214 on 09,211
Set as a trigger. Its address and data are
7 via bus drivers 210 and 212, respectively.
Input internally via address bus and internal data path
. The address is determined by referring to the microcode memory 203.
, the microcode is written from the MPU side via the data path.
It is used in cases such as when reading. Also, lower addresses are bused.
Mic the microcode address via driver 210
It is also passed to the program sequencer 202, and the MPU side
Refer to the control pattern memory 201 at a specific address.
I'm trying to make it possible. Data from the MPU or main memory is connected to the data line 206.
After being latched to D-FF2 1 1 through
Separate memory in microcode memory via 2]
I/ORAM2 1 3 or control pattern memo
Separate I/ORAM 215, 216 in the library 201
Added. Datastore from MPU or memory
The lobe signal is transmitted via the f, lf# signal line 207.
G time I? When applied to 8214, it returns an acknowledge signal.
The communication method used to send data, including addresses and data transmission and reception.
Control takes place. Timing circuit 214 is D-FF21
1. Latch timing and WR signal to D-FF209
Through microcode memory 203, control pattern memo
The timing of writing to the memory 201 is controlled. The neuron as shown in the timing chart in Figure 9
"l""01 pattern of complex control signals given to the chip
is stored in the control pattern memory 201 for one cycle, and
The pattern for one period of -7 microprogram sequence
from the control pattern memory 201 according to the control of the sensor 202.
Generate by continuing. For example, reset signal R
eset, data clock DCLK, weight clock W
CLK, CSOI, CSO2, CSO3 and SYNC
1, SYNC2, SHI, SH2, OCI, OC2, etc.
Control signals are read from I/ORAM 2 1 5
control information, or sequence, that is issued and accompanies the pattern.
The control flag is from the second server} I/ORAM 216
Read out. For example, the control pattern memory 201 is i0
If the pattern 00110001 is stored
is a “1.0” bit pattern, so this '
Control pattern to repeat the pattern of I, 0” bits
By controlling the address of the online memory 201, this pattern
The repetition of is read out from the control pattern memory 201.
That will happen. In other words, the control signal pattern is very complex.
These patterns are rough, so please prepare these patterns in advance.
Store it in separate I/ORAM215, and
Separate I/ORAM2l5 address in micro
It can be specified according to the control of the program sequencer 202.
This creates a structure that outputs the bit pattern sequentially.
ing. Therefore, you will end up repeating some of the same patterns.
The address system determines how to achieve this repetition.
I obey you. This pattern for one cycle is used as an original putter.
I will call it N. repeat the original pattern
In order to
Feedback specific information from turn memory 20l
There is a need to. In other words, the second server} I/OR
Set the sequencer control flag in AM216 as a condition.
input to the microprogram sequencer 202 as input.
By doing so, the microprogram sequencer 202
Original parameters in the first separate I/ORAM2l5
Control to return to the first address containing the turn
. This makes it possible to repeat the original pattern.
will be carried out. That is, microprogram sequencer 2
02 is the general-purpose boat output line 20 until the condition is met.
2-1 to separate I/ORAM 2 1 5
Generate address signals sequentially. This address is usually
is incremented but at the end of the original pattern.
If the condition is met, the original putter
Return to the first address where the file is stored. Conclusion
As a result, a specific pattern is repeatedly separated l/OR
A control pattern is output from AM215. M11. Figure B shows the mask control block 181.
The mutual relationship between the information in the memories 201 and 203 to be controlled
be. In the same figure, lIIfII pattern memory 1 is
Corresponds to the first separate I/ORAM 2 1 5 and controls
Control pattern memory 2 is the second server} I/ORAM
It corresponds to 2 1 6. Inside the microcode memory 203
The control code of the sequencer 202 is stored, and mainly
, Jump instruction and Repeat instruction are stored.
. Reply to a specific address in the increasing direction of addresses
There is an eat command and a control pattern that follows this repeat command.
The number of repetitions of pattern 1 in memory is the control pattern memo.
For example, NOJ is stored in the corresponding address of
If so, IO iterations will be performed. this
In this way, the address increases and the microcode memory
When it comes to the Jump instruction, microcode memory 2
Jump to 500H with the second Jua+p in 03 and Patt
Output ern2. Repeat Pattern 2 5 times
Then, the third Jump in the microcode memory 203
So, jump to rl00HJ again and exit Pattern 1.
It will force you. In this way, the original putter
The pattern is read out from the control pattern memory l.
It will be done. 1 application that refers to this TA full pattern memory 201
WCLK is generated in synchronization with the dress read clock.
and synchronized to WCLK from weight memory 185.186.
The information is read out. To weight memory 185, 186
The address is the address l and address of the address creation unit 204.
Access by address signal output from address 2
be done. Address 1 and 7 Address 2 are respectively the middle layer and
It is separated according to the output layer. For ANP in the middle class
The weight data to be given is specified by address l
Read from weight memory 185 and sent to ANP to output layer
The weight data of is the weight memo specified by address 2.
Each address is the content read from the library 186.
The contents of memory 185 and 186 are each bit of weight data.
in the direction of increasing addresses. are stored in units of
Therefore, the count system for address counters 217 and 218 is
The control signal is given from the microprogram sequencer 202.
It is necessary to be The address counters 217, 21
8, this address is assigned to the bus driver 219, 220.
access to weight memories 185 and 186 one after another via
It is given in increments as a address signal. So
A plurality of weight data are stored in the weight memories 185 and 186.
is read from. WCLK from first separate I/ORAM2 1 5
and the counter from microprogram sequence 202
The control signal is sent to the AND circuit 221 in the address generation unit 204.
, 222 has been added. Counter control signal is high
When the address counter is updated by WCLK,
Address counter 21 is used for bits 1 to 16 of WCLK.
Increment 7,218. And the rest of the WC
For LK17 to 26 bits, the counter control signal is
By setting it low, WCLK can be inhibited and activated.
Stop incrementing the dress counters 217 and 218
do. Then, in synchronization with SYNCI and SYNC2,
Each counter reset signal is sent to the microprogram.
from the can 202 to the AND circuits 221 and 222.
, reset the address counters 217 and 218. child
The addresses of weight memories 185 and 186 can be determined by
Return to the first address. In addition, the mask control block
The mode signal output from the block 181 is transmitted through the weight memory.
Frequently used, i.e., weight memory is disconnected from the MPU data path.
Mode for giving separation weight data to ANP and weight memo
weight memory from the MPU to the MPU data path.
This is to form a mode that refers to the . The mode signal indicates that the lower bit of the data from the MPU is
1 bit of the position address and the write from the timing circuit 214.
The AND circuit 223 generates WR from the input signal.
Set the Frithop flop 224 using the signal as a trigger.
It is formed by When this mode signal is O
Weight memory is used normally. Write signal WR and 1 bit of internal address bus are ANDed.
The clock of the Frisobflovb 224 via the circuit 223
input to the terminal, and the LSB of the internal data path is input to the flip-flop.
It is input to the data terminal of the ROSOB 224. upper address
The mask control block 181 in the comparison circuit 208
is selected, and if it is, the bottom
Load address and data into DFF209, 211
. This type of ink face operation is connected to the MPU.
This is done similarly for other devices, but the weight memory is
Normally, weight data is supplied to AN・P, so M
Connecting directly to the PU's data path will result in bus contention. To prevent this, the LSB of the internal data path is
When it is taken into the flip-flop 224, the mode is set to 1.
and do not select the weight memory as described below.
In this way, data is generated from the weight memory onto the data path.
Make sure not to Internal address bus at specified timing
microcode by an internal address bus.
Either memory 203 or control pattern memory 201
Specify the address and internal
Write desired data from the data bus. This results in
Microprogram sequencer 202 and microcode
Memory 203, Sever} I/ORAM2 1 6
Change the stored program or select Severe} I/
The control pattern stored in ORAM 215 is changed. FIG. 12A shows data storage in this weight data memory 230.
FIG. In the figure, the 8 bits in the column direction have the same address.
This is the information of the 8-bit data included in the address, and each bit
is given to ANPI, ANP2...ANP8 from the bottom.
Ru. The addresses are different in the row direction, with rows to the left as shown in the figure.
The number of addresses increases as the distance increases. weight day
The data is 16 bits including the sign bit, so this
Store from small address to large address
. The MSB is a sign bit, and the other 15 bits are
It is a numeric bit. Micro program sequencer 20
The address is incremented from 2 in synchronization with WCLK.
1 word of weight data, i.e. 16 bits.
will be read out in order from MSB to LSB.
These weight data are passed to 8 multiple ANPs simultaneously.
It will be done. In this way, data is stored in the direction of increasing addresses.
Since the structure is such that the weight data is
A counter for the addresses is required. In other words, one word of weight data data from MSB to LSB.
Once the code has been counted, the weight data for III
Control is required to ensure that the This control
After all, it is done with the micro program sequencer 202.
There is. FIG. 12B shows the specific weight memory blocks 185 and 186.
It is a circuit. The memory 230 is called MB8464A-70.
It is RAM. Output corresponds to ANPI to ANP8
It is 8 bits. Basically, bus signals seen from the MPU bus
line and the area visible from Mask Control Block 181.
Use either dress 1 or 2. Addresses 1 and 2 are mentioned above.
These are addresses 1 and 2 in FIG. 11A. this address
1 and 2 are incremented in synchronization with WCLK.
It is input. 8 bits of data are read simultaneously and each
Bisoto is given to ANPI to ANP8 at the same time
. When the mode signal is 0, the heavy
The memory 230 is chip selected, and at this time, the memory 230 is
Address 1.2 from the black program sequencer 202
becomes valid at the multiplexer 234. Then, the weight
Weight data is sent from memory 230 to ANPs 1 to 8.
. On the other hand, since the output of the inverting circuit 231 is high, the output is high.
Estate bus transceiver 232 is disabled
state, the output of the weight memory 230 is output to the MPU.
Not done. When outputting to MPU, set the mode signal to 1 and
Address decoding is performed by appropriate address information from PU.
chip selects the memory 230 via the reader 235;
An address is given to the memory 230 from the MPU. mode belief
When the number is 1, reading to or from the MPU bus
$lgFl of write to memory 230 or read
The direction of the write is from the MPU through the AND gate 236.
Read Signal of the data line coming from
determined by. Next, the learning algorithm will be explained. Figure 12c shows the pack bag game system used in the present invention.
This is a flowchart of a learning algorithm called . studies
Xi will proceed as follows. Neural network of the present invention
Work, that is, a hierarchy composed of a set of ANPs
Is the MPU the complete information that should be learned from the input of the type network?
The signal is inputted via an input 'MvsE path (not shown). So
Then, the input signal passes through the input side circuit, intermediate layer and output layer.
+ via A/D converter to network output via
1 MPU is given. Stored in main memory on the MPU side.
A learning algorithm exists. Inside the MPU, the teacher signal is the main
Import from memory and connect the network output and teacher signal.
Check for errors. If the error is large, the MPU
, in the direction that the network produces the correct output.
This will change the mmi data, which is the strength of the connection. This weight data is passed through the weight memory 230 to the AN of each layer.
Added to P. When the weight data is updated by the learning algorithm
, the pack propagation learning algorithm in Figure 12C.
will follow the rules. Learning algorithm starts
, the MPU selects the Lth neuron ANP. of the output layer. teaching
Find the error between the master signal YL and the current output YL and calculate it.
Substitute into ZL. The output YL is the output of the neuron ANPL.
For example, we can use a sigmoid function as a nonlinear element.
If used as the output value of this nonlinear function,
It is something. Therefore, in neuron AN p+,
, it is necessary to propagate the error ZL to the input side of the nonlinear function.
be. When performing error propagation, the energy function, i.e.
The energy is the square of the error signal multiplied by 1/2, that is, EL = 1/2 (YL Yt)! The partial with respect to the nonlinear function input χ, that is, is as follows.
It can transform into a sea urchin. = (YL YL) ・f (Xc
). Here, the nonlinear function f(XL) is a sigmoid
If it is a function, then if we transform the differential r'(xt) of this sigmoid function, we get r'(XL) = YL(1-Yt.).
Become. This is shown in 82 of the flowchart.
Ru. Therefore, δ or energy non? Shape function input
, the deviation for , is V,XZL, that is, shown in S2.
The UL will be This energy nonlinear function input
It is necessary to further back-propagate the error amount δ to the intermediate layer.
be. Let A be the Kth neuron in the hidden layer. The output of Ak is assumed to be YII. Output layer neuron AN
The non-cotton function input XL of P t is
The output of the uron (y + ...YE■}
It is expressed as a sum of products multiplied by a weight WLK. Therefore, Xt is divided with respect to the weight WLw. on the other hand
, the variation of weights W and K with respect to energy EL is given by the following equation.
available. aWt m C; IXL o) Wt Kento
Become. In other words, it expresses S3's TL Uharo EL aWLII, and the energy weight vs.? partial distribution
There is. Therefore, let this TLX be the weight change ΔW.
However, in order to speed up the convergence, the first equation shown in 84
Add the second term and modify the weights using the following recurrence formula:
Ru. ΔWLk - αTLk +β・ΔWL, WL. =WLk+Δwt k Here, α and β are constants. Now the output layer has a specific
-I'm paying attention to Ron ANPL, but this person is NPI. inside
Do not assume that all neurons in the interlayer are connected.
Then, for each A N P L, set K from 1 to K Ill
! It is necessary to repeat until X. This is the flowchart
By repeating R1, the number of neurons in the middle layer is K■8.
This will be repeated. When this repetition is finished, the output layer
The backpropagation for the specific new terminal ANPL ends.
It turns out. Therefore, this can be expressed as all output layer neurons (ANP, ,
ANP! ．． ..., ANPL■, }
As shown in R2 of the flowchart,
, L starts from 1 and repeats until I■. Sunawa
What, the number of Nierons in the final output layer? ■Repeat only 8
It becomes. Next, we will learn from the middle layer to the input layer.
Become. The algorithm is almost the same, but the error signal is
It cannot be expressed by the difference between the master signal and the output voltage, and becomes the formula S5.
. That is, Zk is the Kth neuron in the hidden layer, A
This term corresponds to the output error signal of This is calculated by the following formula.
It is clear that ? Become. Therefore, for the index L of Zk in S5
1 to L. Repeat until ■, that is, the number of outputs (R
3) The error signal component Z,l of the intermediate layer is calculated by
Ru. After that, the algorithm between the hidden layer and the output layer is the same.
It is the same. That is, first, the differential value of the sigmoid function■
, and use it to calculate Uks, that is, the non-energy
The amount of change with respect to the linear function input is determined at 86. In S7
No U? Find the output of the input layer, trII with YJ, using
Melt. This is shown in S8 as the main part of the weight change.
∆WkJ is calculated by adding the second term to speed up the convergence.
Therefore, the previous value Waj is given its ΔW1. Add J to create new W
, and so on. This is a weight update. Update this weight
Repeat for the input number J■8 (R4). In other words, j=1?
By repeating from to J thaX, input and intermediate
The weights between layers will be updated. In addition, Z of S5
k corresponds to the error signal of the output of the intermediate layer,
This is the deviation U of the output layer energy with respect to the function input value
It is expressed as backward backpropagation of L, and WLx is the intermediate
It is determined only after the weights of the layer and output layer are determined.
. In other words, the calculation for updating the weights is performed by the neurons in the output layer.
Starting from the neuron ANPL, the middle layer neuron A N P
Moving on to w, the intermediate layer neuron ANPK changes its weight.
The difference ΔW cannot be calculated unless the previous stage ΔW is determined.
It has become a thing. Therefore, calculations can only be made by going back to the maximum input layer.
Because of this, this learning is called backbroadcasting.
It is. Learning with BankBroadcastin uses learning data.
Forward motion that inputs complete information and outputs the result;
The strength of all the connections is such that the error in the result is small.
will be changed backwards. Therefore, this forward movement
It also requires production. In this forward motion, the analyzer of the present invention
The oral cavity is effectively utilized. Also, out
The algorithm for backpropagating force values is executed on the MPU. Note that if it is nonlinear and is not a sigmoid function, then
The nonlinear differential values of are different. For example, tanh(X)
If so, the learning algorithm is nonlinear, as shown in Figure 12D.
The result of differentiation of the shape is ■, = I-IYLI in the output layer.
(S2′), in the middle layer V,=l−lYm l (
s6”).Other parts are given the same reference numerals as in Fig. 12C, and explanations are omitted.
Omitted. Figure 13 shows the configuration of the daisy circuits 173 and 174 on the input side.
It is a diagram. In the figure, 240, 241, and 242 are D type flaps.
It's a rip flop. D at the rising edge of the DCLK signal
Set the data input to the fi child, and set the output Q to the state of 1.
to be jealous The first flip-flop 240
At the falling edge of , the CSI signal is set. And then the next
At the rising edge of the second Frithop flop 241,
cent output signal. Its output is Inl of the third flip-flop 242.
A child. has been entered. The clock that sets that input
The clock signal is the output of the 4-bit counter 243. Coun
243 is triggered at the falling edge of WCLK. Chestnut
It is activated at the falling edge of DCLK. Therefore, D
At the falling edge of CLK, the counter 243 becomes all 0.
, after the falling edge of WCLK is input eight times, the upper bit
The QD signal at the top becomes high, so this acts as a trigger.
Frisov flop 242 outputs a high signal to CSO
. When the output of flip-flop 241 becomes 0, CSO becomes
cleared. This operation causes the CSI to fall.
Therefore, for a predetermined period of time equivalent to 8 pulses of WCLK,
A daisy operation is performed in which CSO is output after
be exposed. Figure 14 shows the mank that forms the dummy node Nieron.
3 is a specific circuit diagram of the value node circuit 187. FIG. same
In the figure, resistor 250, Zener diode 25L 2
52, resistance 253, voltage followers 254 and 255 are constant
This is a circuit that generates voltage. Resistors 250, 253 and
+12 volts via diodes 251 and 252
When a current flows between −12 volts and voltage follower 254,
The 255 inputs have +7 volts and -7 volts, respectively.
It is formed. These voltages are applied to voltage followers 254, 25
It is outputted via the output resistor 256 of 5. one of these two
Analog switch 25 to draw constant voltage in time division
7 to 264. When the T mode signal is 0
, the constant voltage is passed through the analog switch 257 to the next
A voltage follower 265 is provided. T mode is 1
In test mode, the analog switch 258
Since the output is suppressed to analog ground, O
Volts are input to voltage follower 265. test mode
On the bus, the offset on the bus is notified to the MPU.
It becomes. The voltage follower 265 is a switch control of the output section.
enabled by the control. When output enable is 1
When the analog switch 260 is turned on, it becomes a voltage follower.
The output is given by the dummy node.
It is not output to the code output. Conversely, when the output enable is 0
is output to the dummy node output. analog switch 2
Switch control of 60 and its output is output enable or
is controlled by 2 and 0 enabled. i.e.
Dummy node output when output enable 1 or 2 is 0
A constant voltage is output. In addition, the dummy node output is
One side is for the dummy node in the input layer, and the second one is for the middle layer.
This is the output for the dummy node. The output of this dummy node
Since the voltage is fixed at a suitable value, the threshold
Can be used as voltage. In addition, Zener diode
251 and 252 output a constant voltage in reverse bias state.
The fixed voltage is from +7 volts to -7 volts.
It is possible to vary within the range of. Output enable 1
．． Is 2 another ANP connected to the analog bus?
to avoid collision between their output voltage and their analog bus
dummy no from mask control block 181
The enable state is determined by the code control signal DOS. Figure 15 shows the nonlinear E number generation circuit, Figure 16,
Figures 7 and 18 show the digital logic side inside the ANP.
It's hardware. Figure 15 shows a transistor circuit that realizes a sigmoid function.
It's a net. The sigmo-id function here is continuous and monotonous.
refers to non-decreasing functions and specifically excludes linear functions
isn't it. In the same figure, 343, 356, 378, 39
0,298,314 transistors and their paired
Form a differential amplifier with transistors and connect it to the collector side.
Each group of connected transistors forms a current mirror circuit.
It is. The collector of the left transistor of the differential ANP
The collector current that flows is the output current. current mirror
The current is output by changing the direction of the current. The current goes to the output VO
It enters the connected resistor 336. Due to low resistance 336
voltage is converted into current. High impedance output as there is no drive capability
Received at Op Amp Vanofa. transistor 337,
The circuit on the input side of 339 is a bias circuit. Shigumo
A piecewise linear method is used to realize the id function. S
The slope of each section of the gmoid function is
Determined by the ratio of Misota resistance 344 and output resistance 336.
It will be done. At this time, the emitter resistance of L transistor 343 etc.
included. Each differential ANP has a different gain. Each piecewise linear
The breakpoint for the transition of is based on the saturation characteristic.
I am using it. Their saturation characteristics are all different. vO output
At the point, the sum of the currents output from each operational amplifier is
Set the saturation characteristics of each ANP so that the value becomes a sigmoid function.
It's changing. Transistor 345 and resistor R1 are T4 current sources
It is. Transistor 346 and resistor R2, transistor
353 and resistor R3 are all current sources that supply the same current.
It is. In other words, the resistance is determined so that the current value is the same.
being admired. All have the same current source. transistor
345 and 346 collectors are connected, so the sum of
Current flows through the intersection of resistors 344 and 347. transis
When the collector currents of the terminals 343 and 348 are balanced,
Then it will be the same. The L transistor 351 improves the characteristics of the current mirror.
It is for the purpose of Transistor 350 is a diode
It's a connection. Changing the direction of current means changing the output
When the current is drawn in and when the current is sent out,
be. As shown in the figure, the current mirror transistor
Current flows from the collector of the capacitor 351 toward the output.
. There are many transistor arrays on the bottom, but the emitter and
Transistors whose directors are connected to the same point are connected to the same point.
It is a rangister. For example transistors 358 and 360
is the same transistor and this is the same as transistor 345
It is something. Also, 359 and 361 are the same transistor.
This corresponds to 346. 368,369 transistors
The data is the same, which corresponds to 353. Same as below
It is. Therefore, constant current power supplies driven by the same current
An operational amplifier with
A circuit with a total of 6 components that perform different operations
It is. In addition, the transistors 337 and 338 have a level shift
330 and 327 are also level shifts. Re
The bell shift circuit has an operating range based on the positive and negative sigmoid functions.
This is to make them almost the same. transition
The star 352 is connected to the collector current of the transistor 351 and the transistor 352.
Correction to make the collector currents of the resistor 353 equal
It is for use. Transistor 367, 385, 287
, 307 are also similar. Figure 16 shows the pulse signal supplied to the neurotinob.
Sequence generator 28 (FIG. 2) for forming
This is a concrete circuit. 401 and 402 and 404 and 405
is an inverter, and each inverter is an inverter for the cloth
It is ta. Rise of the latch signal of Frisob F, F
I made separate clocks for rising and falling clocks.
Ru. In the same figure, Furi, Buflo, and Pu are standing up and playing with Chronok.
Inverter and F. Stand up with F and raso
Chi's F. Form F. For example, in D C L K,
The one that passes through one inverter 401 falls and is used for lanochi.
This becomes the clock signal. and passes through the inverter 402
The one that rises becomes the clock DCLK for the latch.
. Similarly, the output of inverter 404 is WC for falling.
LK is the output of inverter 405 as the rising clock
It is WCLK. F. In F41.0, SYNC signal
The falling edge of DCLK latches the signal. F.
SYNC signal with F410 and 415 one cycle of DCLK
SYNC and its 1st signal.
17 pulses are created with the clock-delayed signal. S
1τ (one cycle of DCLK) after YNC rises
The pulse discharges the integrating capacitor in the ANP. vinegar
In other words, the signal CRST resets the capacitor.
It's a signal. Another D S H 2 is from the falling edge of SYNC
DCLK(7) A pulse with a length of lτ,
This corresponds to the sample/hold capacitor in the ANP.
sample/hold signal. 411 F. In F
Because the crosshair is WCLK and the data is DCLK
, WCLK latches the DCLK signal. So
After that, the SYNC signal goes high at NAND gate 414.
The first WC that comes with
Trigger LK and F. I am using the F443 clock. Nando Gate 4 1. 4 and inverter 440
It becomes C. F. At F443, the SYNC signal goes high.
The first signal called WCLK that comes in the state where
Imports weight data, that is, code binoto of WD.
. This signal is the weighted digital data coming in serially.
This is the MSB, or sign bit, of the data. That is, F.
F411 and Andgame} (414.44.0) tie
The sign bit is set to F. F443 latches. 4bi
The binary counter 416 in the cutout counts the number of WCLK pulses.
I get it. 16-bit digital weight data comes in.
So count 16 times. When the count is finished, the output goes high.
and enters the inverter 423. This signal has 16 parts.
This is No. C3 indicating that the loading is complete. This belief
The number shifts the weight data that comes into ANP serially.
It is used for controlling input to the register 27 (FIG. 2). Ma
The lowest bit of the counter 416 is sent to the inverter 422.
is input. The output of this inverter 422 is the CSO signal
generate. CSO is a daisy chain control signal.
Therefore, in the analog bus Bl, the two ANPs in the previous stage
In order to prevent conflicts between signals emitted from
After falling, perform a daisy operation to issue the next CS.
It is necessary to form a delay circuit for this purpose. This di
Ray delay time counts WCLK and its counter
It is formed by values. Counter 416 has finished counting
, an instruction to Fritsop flop 433 that it is finished.
The signal is latched through inverter 423, but this
is being played on WCLK. That is, the 17th WCL
I'm hitting K. through inverters 437 and 438
, the latched signal returns to the counter 416 and the counter
Do not allow 416 increment operation any more.
Performs disable control. Inverter 43
When the output of 8 goes low, counter 416 starts counting.
Stop. F. F433 page output is flip-flop
It is in 442. This is the output of shift register 408.
It becomes a power gate signal. That is, 16 digital
Shift register and register 408 receive weight data.
15 of the numerical bits shifted in sequence and excluding the sign bit
Once the bit data is arranged in parallel, these can be output.
Strengthen. No output is output while shifting, and all
The gate signal for outputting when it enters is WR.
Ru. The contents of shift register 408 are fed to the multiplier of ANP.
available. The signal output from F, F433 is branched.
is used as a shift register enable signal. F.
F442 is F. Latch the output of F433 at the rising edge
It is something that Falling edge of WCLK 16fII
The shift is completed at the gate, then stand up to open the gate.
It is possible to use a falling latch, but this is done at the falling edge. F. F
412 generates a pulse signal for selecting a sigmoid function.
It is. F. Using F412, the reset signal is {. t
At this point, W D , the weight digital input
Depending on whether the signal is O or not, decide whether to use the sigsoid or not.
select. This method may not be used in this system.
be. In reality, sigmoid selection 1'11 can be directly accessed from the outside.
to form a contact. The circuit below is a daisy chain circuit. Output of counter 416! i-F. With F434, it is difficult
, and with that De, Ray, the final F. Trigger F445
Stay connected. This causes a 1r shift in DCLK.
In addition, don't just slide around, put your head down.
are doing. That is, the CSI signal itself is D C i-
There may not be enough CSI for one period of K, so the CSI is used as CSO.
In order to
Delete the Icro, delay the front of the waveform, and leave the rear as it is.
creating a signal. Gates 425 and 427 are CSI buffers.
It's a sofa gate. Positive B, F and inverter bass
It is a. Figure 17 shows the sample/hold S/H signal and the 0C signal.
This is a phase control circuit 29 (FIG. 2) that forms a phase control circuit 29 (FIG. 2). S/
H Shinso is the one that goes into the inverter 515, and the one that goes into the inverter 515.
- I'm saying goodbye to those who fall into 5 2 4. OC signal too
Same 1, p. S/'H signal is game 1.524
A: Farewell to Novata 515, Inverter 5 1:i
If you enter game 1.5 2 5 at R mountain, then inverter
There are 8 levels. With the same phase for the S/H signal
It creates two signals with opposite phases. This is an inverter
I decided to combine several stages of cascades and create a cascade.
This prevents two outputs from becoming 1 at the same time. vinegar
That is, two sample/hold S/H signals, S/H
Forms O and S/H 1, which do not both become 1
7. In other words, inverter
The lane is to avoid both S/11 signals being on at the same time.
It is a delay circuit. Delay delay time is invar
Determined by the length of the chain, one side is on.
It is delayed by several steps from ζ and the other one is turned on. The same applies to S/HDO and S/HDI. The circuit related to the OC signal is basically the same, but
The CRST signal is input to gates 528 and 529.
Then, if CRST is 1, both outputs are forced to 1.
, both OCO and OCI should avoid becoming 1 at the same time.
However, in the case of OC, it is simultaneous only when CRST is 1.
I am trying to set it to 1. This allows analog
Integrator capacitor via Swissochi'LQ f3j
It has a reset function of discharging the ffi load.
Ru. Figure 18 shows a 15-bit shift register 27 (Figure 2)
It is. Gates 602, 603 and 6014, and F
．． It is equivalent to 1 bisoto in F627, and I will explain using this.
. The output of the previous time is input to the gate 603.
This is F. This is the output of F628. previous bit
Since it is an input from , it becomes a data signal for shifting.
. Other signals entering gate 603 are SHFT, i.e.
That is, it is a shift signal inverter. This is shift control
When this is enabled for a signal. to give shift instructions
Become. Also, at the gate 602, F. F627 itself
Contains the output of This feeds its own output
This means that you are in a bad mood. The other input of the gate 602 is an input of the S L{ F T signal.
gated phase is included in the same way, but this phase is gated.
It is different from what is included in 603. This causes the shift
When disabled, the current output will be retained.
. The cross signal always comes in regardless of the shift, so
, if the shift is not valid even if the cloth enters, the shift is
Not performed. Only when shift signal S H F T is valid
Shift the previous bit and input it through gate 603.
This results in a shift operation. The WR signal is the gate 632,
It is in the AND of 633 etc. This is the output of each bit
It is a selection signal for outputting or not outputting, and the signal is sent to the multiplier.
Controls whether or not to pass data stored in the field register.
It becomes a signal. Also, in order to take fanout, for example
If the inverter 620 converts 5 out of 151 units ([!
I's F. F recent signal and 10 at gate 626
F. It is designed to take charge of F's recent signal. Fan-out shift register 608 is shift enable
Equipped with LE S H F T and output enable WR functions.
ing. Next, the neurocomputer according to the present invention can be
A case where it is configured with l-work will be explained. FIG. 19A is a conceptual diagram of a feedback type network. Even in the case of feedback-type networks, there is basically an input
However, there is a return path in which the signal that you output also comes back.
It becomes a structure with . This feedback method is based on a hierarchical neural network.
A type that uses time division multiplexing in one layer in a multilayer network.
When it is used as a hop file, it is also used as a so-called hop file.
When used as a field-type neural network
There is a case. In the former case, the ANP input/output signals are time-divided, so
, a certain sequence cycle at the output point of each ANP
The output data of the same ANP is output sequentially,
Layered neural network for each sequence cycle
It operates sequentially as the input layer, intermediate layer, and output layer of the work.
In the latter case, the output of the ANP is
The output voltage is fed back until the voltage stabilizes. are returning
When you get a result, the result is the previous data, that is, you
The state is repeated until the data matches the previous data.
, convergence occurs when a stable solution is reached. According to the fruit example of the present invention, as shown in FIG. 19B,
, the return path will be realized using a common analog bus CB.
There is a U-shaped return section. And I calculated one myself.
The output is output and sent to each ANP through the return path.
Their outputs will be feedbanked. this return
Repeat the action. Figure 20 shows the neurocomputer of the present invention in a hierarchical network.
implemented by a feedback network that operates as a network.
This is an example of this. from analog input boats 1 and 2
ANPI, 2.3 for time-division analog input signals.
Perform the product-sum operation using ANPI, 2.3 as the middle layer.
Operate and send time and minutes from ANP1, 2.3 to analog bus B2.
This output signal is sent to the analog common, which is the feedback path.
Returns to analog bus B1 via bus CB and returns again to analog bus B1.
Perform a product-sum operation on the return signal using ANPI, 2.3.
This allows ANPI,2.3 to operate as an output layer.
2 and 3 create a layered network.
This is the realization of the work. Manx Value Node
Circuit 187 is the DOS output of the mask control block.
In response, a dummy signal is generated on analog bus B2. stop
DCLK and WCL from the mask control block.
K is input to the daisy circuit 171, and the CSl signal is input to the daisy circuit 171.
Specifies the timing of the rise and fall of the signal.
. Figure 21A is the feedback hierarchical network shown in Figure 20.
This is a timing chart. WCLK is generated only while DCLK is rising.
, the analog signal becomes steady after DCLK rises.
, and after the weight data comes in serially,
Mask control block at the timing before the lines are aligned.
CSOl from 181 is input to the daisy circuit 171.
Stand up as shown in ■. At this time, the analog input port
Analyzer held in sample/hold S/H from 1
The log signal is sent to the analog bar via analog switch 175.
Appeared on SBL, ANPI. The product-sum operation is performed in 2.3.
. At the next DCLK input, CS to the daisy circuit 172
When I rises as shown in ■, the analog input port
A sample/hold circuit that holds the input signal from
S/H signal is converted to analog via analog switch
Appears on bus B1 and ANPI. Second Seiwa performance in 2.3
calculation is performed. Furthermore, DCLK is input at the next timing.
After that, open the mask control block as shown in ■.
A dummy signal DOS is generated and sent to ANP 1, 2.3.
Then, a third summation is performed for a fixed voltage. Next
While the SYNC signal of
．． A sum-of-products operation is performed for the three output layers. Access to weight memory
The address count prohibition signal of address 1 signal rises.
The WC counts the address counter only while
LK is enabled, otherwise its counter
is suppressed. Next, CSO2 mask control IJ-
A. from Rubrosok. Given NPl, A. N
P1 sends the result of iii times of product sum to analog bus B2.
output and connect to analog bus B through analog common bus CB.
1 and return to ANPI, 2.3 again as shown by ■.
Then perform the sum-of-products operation. CSO2 is the internal disk of 8NPI.
In the G-chain circuit, after adding a predetermined delay,
Add the input signal CSI to ANP2 as shown in ■, and
When the ANP output signal is transferred to the analog bus B2 and the common bus C
ANP 1 again via B and analog buses A1, B1
is added to , and a sum-of-products operation is performed here. Similarly, ANP
After the CSO from ANP2 is delayed for a predetermined time, the CS from ANP3 is
I signal, and this CSI signal rises as shown in ■.
When the output signal of ANP3 is connected to the analog bus B2,
ANPI again via communication bus CB and analog bus B1,
It is fed back to step 2.3, where a product-sum operation is performed. similarly
As shown in ■, the rise of the signal DCS from the dummy node
Again, ANPI. 2.3 to
Then, a sum operation is performed. Then, the next CSO2 signal
At the rise of ANPl. ．． 2 through S/II
Force is generated as shown in ■ and ■. Note that the analog input button
There is no output from port 2. Here ■, ■, ■ are ANPI, 2.3 operating as the middle layer.
■, ■, ■ are ANP 1, 2.3 are output layer and L7
It works. Therefore, according to this embodiment, A. NP1,
It is possible to configure a floor Z-type network with only one layer of 2.3.
Wear. FIG. 22 shows the 1st model according to the present invention.
The computer is configured with Hosobfield-type feedback network.
Fig. 23 shows the timing chart.
It is. Memo for mask control block 181
The output of the re-address pin and mode pin is the weighted memory block.
This weight memory block 185 is added to the memory block 185.
The BIO which is the data output of A.5 is A.5. NPIXBll is A
NP2 and B12 are connected to ANP3. mask control
Output signal from the CSO1 terminal of roll block 181
is added to the daisy circuit 171 and the Swiss circuit 175,
At the rising edge of this signal, the signal from analog input port 1 is
The output of sample/hold circuit 173 is connected to analog bus B.
Put it on 1. Then, the daisy circuit 171 causes a predetermined time delay.
After that, the output of CSO occurs, which is connected to daisy circuit 1.
72 as CSI and to analog input port 2.
Switch the signal of the connected sample/hold circuit 174.
It is placed on the analog bus Bl via the notch 176. Similarly, the output signal CSO of the daisy circuit 172' is analog
Sample/bold circuit 1 connected to input port 3
74' output switch 176' is opened and the signal is analyzed.
Put it on the log bus B1, as shown in Fig. 23 in AN[)1.
As shown in FIG.
drive the weight clock when the DCLK signal is high,
Digital mmi data that enters in synchronization with the weight crossoku
is multiplied by the analog input signal, and the DCLK f
When il is a low signal, sample/hold signal S 1
1 goes high, and the sum movement occurs on the integrator capacitor.
make a work That is, CSO 1, that is, Din circuit 1
During the period ■ when CSI is high, the address on bus Bl is
ANP1.23 performs a sum calculation on the analog signal. In addition, the OC signal from mask control block 181 is
When the signal goes high, ANPI, 2.3 is off-set.
Cancel, sample/hold, and create a single performance
Finish the arithmetic cycle. Next, the input of the second daisy circuit 172
Since the signal CSI becomes high ■, the next analog input board
ANPI, 2.3 performs the sum-of-products operation on the input signal from
conduct. Then, after the product-sum operation cycle ends, the digital
The CSI signal enters the digital circuit 172', and the sample/home
An output signal is generated from the field circuit 174', as shown by ■.
The third product-sum calculation cycle begins. Next, C302 message from mask control block 181
Issue ■ occurred and was sent from ANPI during the previous product-sum cycle.
The signal that was being formed is returned via the analog bus CB.
Then, ANPI, ANP2 are
．． ANP3 performs product-sum calculations at the same time. then delay for a predetermined time
After that, the CSO output signal of ANP1 is added to ANP2 at ■.
Here, the previous Teiwa cycle is daisy-chained.
ANP2 outputs the signal detected at the time of the signal. This belief
The signal is fed back via analog bus CB to ANPI, A.
NP2 and APN3 drive the product-sum operation with ■. And the same
After a predetermined time delay, the CSO of ANP2 is
It is added to P3, where the output from ANP3 is analog
Returning via Gubus CB, ANPi ANP2,A
In PN3, perform the Teiwa operation with ■. feedback network
As shown in Figures 23A and 23B,
, in three ANPs, six product-sum operation cycles are performed.
The outputs are respectively sent to the sample/hold circuits 177 and 1.
Analog output baud}0,L2 via 78, 178'
is output to. In addition, the sample/hold circuit 177
, 178, 178' output signals are analog multiplexed.
The A/D converter 18 outputs the selected output from the sensor 179.
0 to MP0182, memory 182, and communication interface.
applied to a digital control circuit including face 184.
. The MPU 182 displays the neuron output state at the current time and the previous time.
check whether the neuron output states of
Click. If they are the same, it is determined that they have converged. This way
is carried out via one common analog bus CB.
. A stable solution is reached by repeating the feedback operation.
This will be the final output. Figure 24 shows a feedback network and a hierarchical network.
This is an optimal embodiment of a combination of 8. A digital circuit is provided as an input layer, and an AN
Pl. 2.3 is provided. ANP4 and 5 are in the output layer.
provided. And the output of ANP1, 2.3 in the middle layer is
Analog bus B2 and common analog bus CD
It is returned to log bus B1. In addition, analog bus B1,
B2 has a max value node that acts as a dummy node.
A code circuit 187 is connected. And configure the output layer
The output of ANP4.5 is sent to the sample/hold circuit 17.
7.178 via analog output ba-}O and
output to l. B3 is an output layer analog bus. Neural nesotova shown in Figure 24 using Figure 25
Explain the operation of the park. First, DCLK and WCLK are mask control blocks.
Daisy circuit 171 and ANP 1, 2, 3, 4 from the
．． 5 respectively. mask control broso
As shown in 181 to
When input to the first daisy circuit Vs171, the analog input
The signal from output port 0 is sent to the sample/hold circuit 173.
and switch] 75 to the analog bus B1;
ANPI. In 2.3, the sum of products operation is SHI and CS1
This is done under the control of Next, after a predetermined period of time has passed after CSO1 falls, the second
The CSI signal input to the digital circuit 172 is shown in ■.
When the signal from the analog input port l is
sample/hold circuit 174 and switch 176
ANPI,2 of the intermediate layer by analog bus B2 via
．． 3, the product-sum operation is performed as shown in S H
It will be done. Similarly, after the CSO signal falls,
After the lapse of time, as shown in ■, the CSI to the third daisy circuit is
When the signal rises, the product-sum operation is performed in the middle layer ANPI, 2.3.
calculation is performed. And the output of intermediate iANPl,2,3
CSO2 stands up as shown by ■ and goes up to ANP 1
is applied to analog bus B2 and its output
is returned to analog bus B1 via common analog bus CB.
Since it is returned, the middle layer ANPI, ANP2, ANP3
Then, the product-sum operation is performed again to control SHI and OC1.
A sum-of-products operation is performed in AN. The output of P1 is analog
Since it occurs on log bus B2, ANP4, ANP5
Also, the product-sum operation is performed under the control of SH2 and OC2.
Ru. That is, in this embodiment, the middle layer ANPI
, ANP2, ANP3 and output layer ANP4, ANP5.
At the same time, a product-sum operation is performed. Next, after a predetermined period of time has passed after CSO2 falls, the intermediate
A CSI signal is input to ANP2 of the layer as shown in ■.
The output signals of ANP2 and ANP2 are passed through the common bus CB.
is fed back to the analog bus B1, so ANP1, 2.
In ANP4.3, the product-sum calculation is performed again.
At the same time in 5, Jyowa Chisan is performed in Imingae. Furthermore, the CSI signal is input to ANP 3 as shown by ■1.
Then, ANP3 sends the output address to 'rdress bus B1.
This results in ANPI of 2.3 and output layer ΔNP of 4.5.
A sum of products operation is executed at the same time. Next, Manox value node circuit 187 head dummy
When the signal DSC f is given as ■, it is sent to the analog bus B.
A constant voltage is output to ■, and this voltage (also common bus C B
and is returned via the analog l'I bus B1.
Then, a sum-of-products operation is performed in ANPI, 2.3. that
At the same time, the sum of products operation is also performed in the output layer ANP4.5. SYNCI is the period during which the sum of products is calculated in the middle layer and the middle layer and
is high during the period during which the sum of products is calculated by the output temperature and output temperature.
SYNC2 has a high speed during the sum of products operation between the intermediate layer and the output layer.
Yes. Then, when CS03 is output, ANP4 is
An output is generated at ■, and the CSO3 signal falls.
After a predetermined time, the ANP5 also outputs at ■
So 41-cheating. Note that while the address l and enable signal are low,
WCLK is suppressed. According to the invention, the front consisting of nierotisob of n (W)
The stage layer and the subsequent layer consisting of m multiple neurotisobes.
When you think about it, conventionally the number of wires is Ωm, but with this invention
According to the embodiment, it is possible to use one analog bus.
The number of wiring lines can be significantly reduced, and n
Input analog signals into a layer consisting of several neurochips.
When transmitting data, it is synchronized via an analog bus, similar to the broadcasting method.
Since it can be input at the same time, n l neurotisob in one layer
can perform parallel operations. Sara ζ, even if each N6 gets stuck, bye
Because brine processing is performed, calculation speed can be increased.
. Also, since the neurochip is composed of analog circuits,
, the scale of the circuit is small, and therefore the power is also small.
Because of this, neurocompilation is possible with a large number of neurotisob.
computer can be configured. In addition, two bites
Increasing the number of groups is done by
The control pattern is stored in the control pattern memory and the control pattern is changed.
This can be easily done by Figures 26A and 26B show errors that actual ANP has.
FIG. 3 is a conceptual diagram of the mechanism that generates the difference. Figure 26A shows the case where the input voltage is a known specific value.
, the output voltage of the neural network is shown by the dotted line.
Even if the theoretical value is
Because there are variations in the integral gain in the corresponding ANP,
If an output value that deviates from the theoretical value as shown by the dotted line is generated,
There is a case. Figure 26B shows the case where the input voltage is 0 volts.
In this case as well, the output voltage is output as an offset voltage.
It will be done. How to determine the error voltage at such an output?
The important question is whether it is appropriate to Figures 27 and 28 show this pulse-like error voltage, respectively.
Hierarchical and feedback neural networks that measure
This is a circuit for error measurement. Masokusubali'' - Node episode
The output of line 187 is connected to the analog bus. Also
, sets the data given from the MPU to the port register.
and its control signal E n . ．． T mode, Layermo
Each bit of the code is sent to a mass value node circuit 187.
is given, but it is not given to that Ma Nox value node.
At the same time, the enable signal
It is also used for も's $1 order. For example, enable signal 1
Sometimes the intermediate layer voltage is
Enter MPU. This makes it possible to measure errors.
Ru. T mode is a test mode.
By using the intermediate layer for the O input mentioned above,
The voltage at the output layer analog bus is
It can be edited by the MPU via the data processor. For example, T mode
= 1, it becomes O input in the hierarchical neural network.
The response should be the offset voltage. 0 input
What voltage is sensed through the A/D converter when
monitor whether the Intermediate to monitor each layer
The analog path voltages connected to the layer and output layer are sequentially M
Transferred to PU. If you reverse T mode to low, next time
A high level voltage is always output. DCS 1 and DC
S2 is input to the OR circuit and is a port register.
It is kept low while En output from is high. Po
Set T mode to O by register
If set to 1 and En high, only the input layer will be locked.
It is possible to monitor the output by the result
remember. Next, if you make the layer low, you can create an analog middle layer.
offset on the bus can be sensed. make this selection
This is the port register. In the case of the feedback type shown in Figure 28,
Basically, there is only one bus, so it can be done with one measurement.
Become. If neither sum operation is performed, the offset voltage will be
Pressure measurements and gain errors cannot be measured. As described later, we change the weights for the dummy nodes and
Measure the output voltage when . In this case, each CS
■Controls the CSO and the output status of each layer in T mode.
Sense. This sensed data is converted into an A/D converter.
The data is transferred to the MPU via the . More specific information using the operation timing chart in Figure 29B.
Explain in detail. In Figure 29A, one period of DLCK
At , one cycle of product-sum is performed. That DCLK
When DCLK is high in the first half of one cycle of
At least 16 CLKs are input. This weight cross
WCLK transfers serial data to the shift register.
It is a clock for thinning out. SYNC signal is one
It defines the operation timing of ANP, and is different from DCLK.
rises half a cycle before
Falling down at the point of the cycle. SH is sample/hold
After that WCLK is input as a signal, that is, input analog
After the product of the digital data and the digital data is
The product signal is first sent to a capacitor.
. Then, the first high signal of the sample/hold signal is
At the moment the offset cancel signal OC falls, the offset cancel signal OC
rise and reverse the polarity of the capacitor. At that time
, the input signal to the capacitor is canted to 0,
Again at the next high signal of the sample/hold signal,
When a capacitor is charged, the capacitor has an input voltage of
A voltage equal to the offset voltage at signal O is charged.
It will be done. This reduces the capacitor by the offset.
signal, i.e. the offset has been cancelled.
Signals will be accumulated. However, even with this
If a cut remains, use the error absorption method of the present invention.
Ru. In Figure 27, a test is sent to port register 1 from the MPU.
When a command signal to enter the default mode is input, the En output goes high.
1 or 0 of test mode is enabled.
state. When the layer rises from low to high,
Test the middle layer. And when T mode is 1
, when detecting an offset error signal. layer
is high and the En signal is high, so gate 70
Output enable goes high via 2 and 703. And the input to the middle layer input of the Maxox value node circuit.
To make the fixed voltage 0 volts or high V state,
enable the path. Output enable goes high.
Once it goes up, it stays high until the end of the middle layer test.
. In other words, the offset error is detected for the middle layer.
do. Here, as mentioned above, DCLK becomes a high signal.
Then, one cycle of product-sum is performed. After that, CSI (CS
O2) becomes high and the output from ANPI becomes the analog bus.
(middle layer). Next, after a certain period of time, the CSI signal is also sent to ANP2.
Since it is added in a chain, the same applies from ANP 2.
The analog output signal is output and placed on the output bus of the intermediate layer.
Ru. Furthermore, after a predetermined period of time has elapsed, the CS [signal of ANP3
Since it rises to high, the analog output signal from ANP3
is output on the intermediate layer output analog bus. middle class
Each analog signal output to the operational amplifier 704 and
A/D converter via analog switches 705 and 706
data 707. and A/D converter 707
The outputs Q, are sent to the MPU and temporarily stored in the main memory.
held. Next, as shown in the timing chart of FIG. 29B,
The output of the dummy node with T mode set to 0 as shown in
By setting the voltage high, the MPU outputs the output voltage Q9 of each ANP.
Sense. ~j P tJ is this Q. Adder gain found from and Q9
Find Δ. Also, when the output of the layer of port register 1 is O, the input
Output input via inverter 708 and gates 709 and 710
Since a high signal is added to enable 2, the dummy in the middle layer
- node output terminal, that is, the 21 node times
An output signal is generated from the second output terminal of the output layer AN
P4.5 offset 7} is detected. And its output is
As mentioned above, operational amplifier 7l1, analog switch 71
2.713 to the A/D converter 707.
Ru. Note that the analog switches 714 and 715 are A/D controllers.
Voltage follower when not outputting to inverter 707
It is protected by A. In addition, the output of ANP4 and 5
is temporarily held in sample/hold 715 and 716.
, is output as an analog output signal. The above mentioned
Since the T mode is l, ANP I, 2, 3,
Since input signals of O are added to 4 and 5, respectively, A
The off-set voltage of the operational amplifiers of NP1 to 5 is △/D
The signal is output from the inverter 707 to the 8NP. In addition, as shown in FIG. 29B, when the T mode is 0,
, input dummy node output and dummy node for intermediate layer
A high level analog signal is output from the output.
, A/D converter 707 constitutes ANPI to 5.
The output voltage of the operational amplifier Q. (f&describe) is output
It will be done. Figure 28 shows the error in the feedback neural network.
The measurement circuit is shown, and Figures 30A and 30B show its operating circuit.
Indicates timing. Max value node circuit 187?
In these cases, only the output signal indicating the dummy node output to the input stone is
The signals are output and input to ANPI, 2.3, respectively. A
The output analog bus of NPI, 2.3 is connected via the common bus CB.
and is fed back to the analog bus on the input side. port register
Star 1, gates 702, 703, operational amplifier 704,
Nalogusu Isochi 705, 706, A/D Comparator Ichiyo 70
7. Gates 708, 709, 710 and Anakuchi Gusuino
The operation of chip 714 is similar to that shown in FIG. in one layer
Since it also serves as the output layer, there is no output layer like a hierarchical type.
No dummy node output to the output layer occurs. Dummy to output layer
The second output enable terminal that provides the -node output is connected.
be grounded. And the output signals from ANPI,2,3 are sampled/
Output via hold circuits 718, 719, 720
Ru. Figures 30A and 30B show the error measurement circuit in Figure 28.
Explain in more detail using the operation timing chart of
do. In FIG. 30A, in one period of DLCK
, one cycle of product-sum is performed. One cycle of that DCLK
When DCLK is high in the first half of
At least 16 items are entered. This weight clock WCL
K stores serial data in shift register 16@
It is a clock for The SYNC signal is one ANP
This specifies the operation timing, and is half the size of DCLK.
At the point of half a cycle of DCLK,
Fall down. SH is the sample/hold signal and its W
After CLK is input, that is, the input analog data and
After the product is multiplied with the digital weight data, the product signal is
It charges the capacitor first. And then
The first high signal of the sample/hold signal has fallen.
At this point, the offset cancel signal OC rises,
Reverse the polarity of the capacitor. At that time, the capacitor
The input signal to is canted to O and then sampled again.
/At the next high signal after the hold signal, the capacitor is
When charging, the capacitor has the same value as when the input signal is 0.
A voltage equal to the offset voltage is charged. to this
Therefore, the capacitor has less power by the offset note.
signal, i.e., a signal with offsets canceled, accumulates.
will be done. However, this still leaves an offcent
In some cases, the error absorption method of the present invention is further utilized. In Figure 28, port register 1 is tested by MPU.
When a command signal to enter the default mode is input, the En output goes high.
1 or O in test mode is enabled.
state. When the layer rises from low to high,
Perform a 1-layer test. When the T mode is 1, the offset error signal
This is the case when detecting. layer is high and the En signal
is high, so output enable via gate 702
The first signal in the list becomes high. And Masoxbalino
Connect the fixed voltage to the layer input of the board circuit to O port or high state.
To enable the circuit, enable the circuit. sand
That is, the offset error is detected for one layer. child
Here, as mentioned above, DCLK becomes a high signal, and the sum of products
One cycle of is performed. In the feedback type, the output/enable signal
Since the signal falls immediately, the input bus is disabled.
It becomes a table. However, while the output enable is high,
All ANP offset voltages are already sampled internally.
This means that it is being held. After that, CST
(CSO2) becomes high and output is output from ANPI.
output on the analog bus. Next, after a certain period of time, the CSI signal is also sent to ANP2.
Since it is added in a chain, the host is added from ANP2 as well.
The analog output signal for the offset that was held is output.
output bus and is placed on the output bus of this layer. Furthermore, a predetermined period of time has elapsed.
After that, the CSI signal of ANP3 rises to high.
, the analog output signal from ANP3 is the analog output of the layer
Output on the bus. Each of the offsets output to H
The analog signal Q, is the obeamp 704 and the analog switch.
to A/D converter 707 via switches 705 and 706.
You can call it. And the output Q of the 8/D converter 707
, are sent to the MPU and are temporarily held in main memory.
. Next, as shown in the timing chart of Figure 30B.
In order to
the MPU senses the output voltage Q of each ANP.
Ru. MPO is the adder gain A. obtained from this Q and Q9. Total
Calculate. The above is for the case where T mode is 1, so
, ANPI, 2.3 are each supplied with an input signal of O.
Therefore, the offset voltage of the operational amplifier of ANPI to 5
pressure is output from the A/D converter 707 to the MPU side.
. Furthermore, as shown in Figure 30B, when the T mode is set to O.
In this case, a high level signal is output from the input layer dummy node output.
Since the analog signal is output, the A/D converter 707
From ANPI to 3 is the output power of the operational amplifier that makes 4W.
Pressure Q. (described later) is output. Next, we will discuss the occurrence of calculation errors in the 7-nalog Nieron processor.
Weight correction method using raw model and dummy nodes
explain about. FIG. 31 is an algorithm of the first correction and second correction processing of the present invention.
FIG. The first correction process is the adder gain
Measurement condition settings for estimation and offset voltage measurement processing
It is. In other words, once the flowchart starts, the dummy
-The fixed voltage of the node is set to O, and the weight data for error measurement is
Reset 1−Qr as the weight for the dummy node.
That will happen. The second correction process is a dummy node 0 volts
offset voltage Qr for and lQr of 1 volt.
Result of multiplication as intermediate weight for dummy node
, an offset voltage is added to the output mixture
The correct adder gain is calculated using the information of the error output Q,
A, = (Q.-Qt)/Cl-Qt) is calculated.
That's true. The weight correction method uses a fixed voltage generated from a dummy node.
The weight for each neurotisob (4: weight is as follows)
, by adjusting the weights for the dummy nodes)
This is a method to absorb error voltage. Errors include offset errors and gain errors. In the present invention, dummy nodes logically exist. In other words, from the Maxox value node circuit to the analog bus
Adopts a method that generates a fixed voltage of, for example, 1 volt at the
Therefore, by SUing the weights for dummy nodes,
Input 0 volts to ANP to absorb the error.
How much error will be produced if
The following is an example of how this occurs.
be done. Figure 32 shows the performance in the analog neuron Brosenosa.
J1 model using calculation 2'l difference model and dummy node
It is a conceptual diagram of the positive method. For example, input with a weight of 0.3
In the case of O, the output is theoretically equal to O. JJ++ 3
When the gamma gain is 0.98, set this 0.98 to the input 0.
Even if you multiply it, it will return O and there is no meaning. On the other hand, the input is 11! le
In the case of 3 is the original product value
. 5.If the adder utility 15 is 0.98, then in 1)
The actual multiplied value is 0. 9 8 X O. 3 tona
) 6 Similarly for 1 volt from other nodes IX
0. 4 and <1. 0) X (-0.6)
Calculate all of the values obtained by multiplying each of these products by 0.98.
The sum is 1.274 volts as shown in the figure.
. In addition, an offset of the operational amplifier occurs, and 0.
01 volts is applied, resulting in 1,264 volts.
Becomes root. The actual value is l x 0. 3 + IX
Is it 0.4 + (i.0) X (-0.6)?
and 1.3 volts. However, it should be 1.3 volts.
However, due to gain error and offset error, 1.
It becomes 264 volts. This is the calculation error generation model.
. The figure below is a conceptual diagram of the market correction type force formula, and it shows the calculation error generation.
Offset voltage generated in the raw model -0
．． In order to correct the error of 01 volts, first, in this figure,
The voltage of the dummy node is set to 0.0 volt 1. this
Then, even if the adder {} is 0.98, the output is 0.0
volt, but an offset 1 voltage is applied, and the offset
The voltage Q is -0, Ol volts. and this
The weight for the dummy node is compensated to offset Q.
Set it as positive and change the weight for dummy node from 1 to 1.01.
Ru. This is the first correction, and generally, l. o-Q
It becomes the value of r. However, the adder utility 11 is 1.0.
The force is 0.98, so 1, O I
0.9 8. = 0.9898 vol 1. becomes Towa Prefecture. difference
In addition, the off-set of A bare bu is -0.01 volts as well.
Then, this is added and the output is 0.9798 volts.
This is defined as the mixed error output value Q9. offset
The voltage is -0.01 volt and the voltage is adjusted only by the first correction.
Q, which is 0.9798 of the mixed error output voltage
Multiply the second-order correction from the two pieces of information to calculate the additional gain A, W
do. The second-order correction follows the following equation as shown in the figure. A9 = (Qv Qt) ÷ (1.Qi, that is,
In the current example, A9 is 0.98, so find Ag.
This is the first stage of the second correction. because,
A fixed voltage of 1 volt is applied to the first correction W (1-Qr).
Offseno is multiplied by the adder gain A1l.
The sum of voltage Qr is Q. Therefore, Q9 = (I Qt) ×l ×Ag” Qt
This is because the weight W, must satisfy o=w,XA,+Qf, so ゛ζ must be obtained.
This is the second stage of the secondary correction. The reason for this is that
This is true when all output signals from all ANPs in the layer are 0.
In the future, the output of each ANP in the later layer will also be offset to the ANP.
Even if there is a note, it should be O. i.e. dummy
Since the output of the node is always ■, the output of the dummy node
, a dummy node such that the output signal later becomes O
The weight for the input signal to each ANP of the 11t layer from the
This is because it would be sufficient if it were determined. This W. 0
．． 010204 as the second-order corrected weight of l volt.
Multiply the dummy node and adder gain no\, = 0.9
Multiplying by 8 gives 0. 0 0 9 9 9 9 9
ri, almost 0. Ol Bolt. becomes. This
Since pressure 10,01 is added, the result is O volt.
Become. In this way, gain is also related to offseno1.
This means that Both offset 1 and gain at the same time
It would be better to correct it independently, but since that is not possible, the first
The method of correction is a mixture of both. This way
to generate a fixed voltage of 1 volt from the dummy node.
When the input of A N P is O volts, the output is
The offset voltage that is generated is canceled, resulting in the output
It would be a good idea to correct it so that it becomes 0 vol l. Such correction processing is performed by setting the correction enable flag En to 1.
It is done by doing. Error in the floor JQ neural network in Figure 27
When the T mode is set to 1 in the measurement circuit, the masox value
- The fixed voltage generated in the node circuit is forced to
Become Ruto. For analog bus ■ or ■, this
0 volts is forcefully input. For example, input layer analog
If you shift the bus ■ to 0 volts, each ANPI of the middle layer, 2
．． The output of No. 3 is output as an offset voltage. this
It is input to the MPU side via the A/D converter 707. The MPU side stores this offset voltage as Qr. M
The PU uses this Q to perform the first correction. Q, is given
Once obtained, use this to calculate the weights for the dummy nodes.
, it is necessary to correct the weights. In the second half of the first amendment
assumes that the fixed voltage of the dummy node is 1 volt. for example
, if we target the l-th ANP 1 in the middle layer, this dummy
-The weight data for the node voltage is set to 1.01.
control. The adder gain of that ANP 1 in the middle layer is 0
．． 98, the output of the adder will be 0.9898.
, and an offset voltage of -0.01 volt is added to Q
g is 0. 9 7 9 8 volts are output. child
This is connected to the output analog bus of the intermediate layer for error measurement.
Q is sent back to the MPU via the A/D converter inside the circuit.
, given as information. And the MPU calculates A9 and
Calculate W1 from 1/A9. As a result, W1 is the second complementary
Weight for a dummy node that generates 1 volt as a positive quantity
data, that is, 0.010204. Konoto
Since the adder gain in ANP is 0.98, the addition
The output of the device is 0.01 volt, and the offset is added.
0. Become a bolt. If the MPU confirms this, the data
The weight for me node is 0. 0 1 0 2 0 4
Since the value is found to be correct, the MPU assigns more weight to W1.
stored in memory. The above operation will be performed for all ANPs. Figures 33A to 33D are for hierarchical networks
This is the weight data correction algorithm. First, maskco
Set the control block to error measurement mode. Testmo
In other words, T mode is set to 1, and the layer is set to 1 of the middle layer.
, En is l. Then, the rigidity of the dummy node in the input layer is
Since the constant voltage is O volt voltage, each ANP in the intermediate layer
It becomes impossible to measure the off-cent voltage. Perform the first correction
To do this, the MPU starts interrupt processing.
ffl primary correction and secondary correction are performed. First, the middle class
Interrupt count counter variable that counts the number of ANPs being processed
Set to O and start interrupt processing. Then, ANP calculates the output voltage for O volt input voltage.
When output as Qf, it is passed through the A/D conversion of the measurement circuit.
data Q, t-MPU reads into internal register,
It then moves it from that register to main memory. So
, increment the interrupt counter variable, and add this
The number of interrupt counters becomes the number of measurement targets, that is, the number of ANPs.
Check whether they match. If they match
If not, it enters a wait state while sensing the count number.
Ru. If the count variable is equal to the number of objects to be measured, interrupt
After completing the processing routine, the first
The next correction process begins. That is, for each ANP, a dummy
The voltage at the node is 1 volt, and the first correction value is 1.0.
- Qf is first set as a weight, and all ANs are stored in the main memory.
Store it as a weight for P. In other words, T mode
Set it to O. Then, when 1゛mode=0, from the dummy node
produces a fixed voltage of 1 volt, whereas
Simultaneously measure the effects of offset voltage and gain error, and
Perform preprocessing for secondary correction. The interrupt count counter changes again.
Set the number to O and start interrupt processing. i.e. for measurement
Q, will be measured through the 8/D conversion of the circuit. The voltage of the dummy node is 1.0, and by the first correction
For example, the weight for ANP is 1.01, so add
The output is 0.9898 for the gain of 0.98, and the output is 0.9898.
offset voltage 10. 01 volt is added and Q9 is 0
．． 9 7 9 8 volts is calculated. This Q, is A
The MPU reads it through the /D converter and writes the memory from the register.
and apply this to all ANPs in the middle layer.
Increment the count variable as you would do. This i
Check whether the increment number has scattered the measurement target.
If they do not match, perform further measurements.
. If the number of counters is the number to be measured, an interrupt is generated.
After completing the correction processing routine, the second correction processing begins. sand
In other words, W1, which is the reciprocal of A multiplied by -Q, is applied to each ANP.
and ask. This W. Tamino that produces l volts
It is given as weight data for the code. In other words, it enters a routine to modify the weights for dummy nodes.
, for example, A. Weight data for N P 0.010
204 is determined. These weights are stored in weight memory.
It will be done. The above operation can also be performed for all ANPs in the output layer.
It becomes. Therefore, move to ■, set T mode to 1, and layer
Set to 0 to set the output layer mode, set En=1, and do the same
Start processing. By the way, set the interrupt counter to 0.
, enters the interrupt handling routine and is processed by the measurement circuit.
Offset voltage to the gummy node of O volts, Qf
is calculated for the ANP in the output layer, and these data are
Store in a harpoon. This process is performed on the number of measurement targets (output often ANP
number), exits the interrupt processing routine and returns to the first
Next correction processing will be performed. In other words, dummy no 1.
Set intermediate weight data for a fixed voltage of 1 port.
will be determined. Move on to ■. Set T mode to 0, layer to 0, En to
Set it to 1. Then, preprocessing for the second correction begins, and Q
Calculate 9 by 3. i.e. fixed voltage from dummy node
Since 1 volt is generated, the offset voltage and gain
The effect of errors will be measured simultaneously. Interrupt count
Set the count variable to O, start interrupt processing, and execute Q. of
It is read through an A/D converter and transferred to memory. And the discount
Increment the input count variable and add A in the output layer.
After performing the same number of steps as NP, exit from the interrupt handling routine.
, for the second correction, and use it for the dummy node.
Store it in weight memory as a weight. In the case of hierarchical type, W, obtained by the second correction is originally a dummy
It needs to be added to the weight given to the node. Moreover,
, all electricity needs to be applied with 1/AI.
Ru. Weight data correction algorithm for feedback type net work
Figures 34A and 34. Shown in Figure B. this
The same applies to the case. In the feedback type network, there is only one layer.
be. Set mask control block to error measurement mode
Then set T mode to 1, layer to 1, En to 1,
First, it is turned off by the 0 volt ffi pressure output of the dummy node.
Measure the set voltage. Set the interrupt counter variable to O.
, enters the interrupt processing routine and goes to the output side of the measurement circuit.
Read Q through some A/D conversion and transfer it to memory.
vinegar. Repeat this process for the number of ANP, and
completes the process and begins the first correction process. return net
17 at work! ! , the first correction and the second
The next correction only needs to be performed once. Next, perform the second correction
To do this, set T mode to O, leave layer at 0, and set En to 1.
Make it. Then input the fixed voltage l volt of the dummy node.
and simultaneously measure the influence of offset m pressure and gain error.
That will happen. Set the interrupt count counter variable to O, and
Start import processing. Via the A/D conversion of the measurement circuit
Q. Read for each ANP. i.e. dummy no
The m diameter determined by the first correction for 1 volt from the
When given the data, A. N P is the internal adder gain 0
．． The voltage is multiplied by 98 and the offset voltage is added.
and outputs Q. This process covers all 7 of 11 minutes.
't N P, and the Q corresponding to each AN})
, and Q9 information are written to memory, the interrupt processing starts.
The routine ends. Then, the MPU performs secondary correction processing. In other words, M
PU is the reciprocal of A, to W. is calculated for each ANP,
This will be stored in the weight memory as the weight correction amount.
Ru. Next, we will explain the weight correction method using multiple dummy nodes of the present invention.
I will clarify. In the fixed-point system, the -E bit is the sign bit,
If you place a decimal point after this upper binoto, it becomes a positive number.
is between O and less than 1. Therefore, the weight data is expressed in 16-pin fixed-point format.
In this case, 1 or more cannot be expressed. First, to explain hierarchical networks, dummy
Assume that there is one node for each layer. 35th
At the top of diagram A, where the adder gain should be 1
0.98, and furthermore, the offset voltage is 0.98. Ol existence
The output should be 1.3 because there are 1. 2
6 4, resulting in an error. Threshho
The weight data for the field is offset, and the weight for the gain error is
The weight data plus the weight data is the weight for the dummy node.
It is given as a weight and stored in the weight memory. i.e.
, as shown in the left side of Figure 35A, the dummy node and the input
In the two nodes showing the output of the ANP of the force layer,
, the weight applied to the voltage of 1 volt at the dummy node is
0.316324 in the form of the sum of hold and error correction
Assume that This value is represented by a fixed point
Although it is possible, if this is divided into two weight data,
think. Then, the weight for normal ANP is set to 0. 4
0 8 2, weight for other APNs - 0.6122
Suppose that In this way, the ANP output will be +1.30.
Become. Furthermore, the weight data of that dummy node is fixed.
It may not be possible to express it with a decimal point. Therefore, this city
If data is expressed as the sum of two or more weight data, each weight
For example, the data is a value less than or equal to 1, and is a fixed-point data format.
This means that it can be expressed as a digital quantity. On the right side of the ffi35A diagram, one is the threshold
mm 0. 3 0 6 1
2 and the other one is the offset and correction damino mentioned above.
The weight for the code is 0.010204. That is,
The voltage for the threshold is greater than the offset voltage.
Therefore, there is a possibility that the error will be large.
, a dummy node for threshold hold and a dummy node for offset.
By separating the menode, the offset value is fixed.
It is almost unavoidable that it cannot be expressed using the constant-point method.
It will be done. The voltage generated from the dummy node is a fixed voltage]
Since the weight is divided into two in this way, l
X0.30612 volts +W. The voltage corresponding to the weight
It is the same as the product when . Each weight data
is the separate weight memory for the weight memory dummy node.
stored in the storage area of , and time-shared to specific ANPs in the middle layer.
is given as weight data for dummy nodes. difference
Furthermore, if the weight data for threshold is large
If we divide this into two, we get the result shown on the right side of Figure 35B.
In this case, there are three pieces of weight data for the dummy node. each weight
Data is represented as a digital quantity using a fixed-point system.
It is something that In this case, a dummy node for weight memory
There are three storage spaces for storing weight data for time-sharing.
is given to the ANP in the middle layer. However, each weight data is divided for each ANP in the middle layer.
are different from each other. The same thing applies to the output layer as well.
Done to the menode. Regarding return-type networks, there are also hierarchical networks and
Similarly, the threshold dummy node and offset
Dummy node for data and further threshold
By dividing the dummy nodes for use as necessary,
Weight data when weight data is expressed using constant point method
can expand the dynamic range of However, the feedback type
In this case, the output of the dummy node is connected to only one layer. Figures 36A and 36B show the hierarchical structure shown in Figure 8.
In a network, if controlled by multiple dummy nodes,
FIG. The dummy node outputs DCSI and DOS2 are consecutive
The multi-dummy
Indicates that two weight data are controlled corresponding to the nodes.
are doing. Figures 37A and 37B show the feedback type cable shown in Figure 20.
In notebook work, if you control I1 with multiple dummy nodes
FIG. Dummy node output DCS3 has two consecutive high pulses.
have. [Effects of the Invention] According to the present invention, the front layer consisting of n neurochips
Considering the subsequent stage consisting of m multiple neurochips,
When the number of wires is nm in the original case,
According to Akira's example, it is possible to use one analog bus.
Since the number of wires can be reduced significantly, the number of wires can be significantly reduced.
, an input analog signal to a layer consisting of n 221 rotisob
When inputting
and can be input at the same time, n. (In
New l: Jna, Bu can perform in parallel. Furthermore, each
Since the layer is also subjected to B/F B/L processing, the performance
Increase calculation speed. In addition, Neurosennopu is composed of analog circuits.
Therefore, the circuit size can be small, and therefore the power consumption is also small.
Since the neural network is
You can configure Nenotwork. Furthermore, since we used multi-dummy nodes, weight data etc.
When expressed in fixed-point format, its dynamic range is
You can expand the image.

[Brief explanation of the drawing]

FIG. IA is a Zoronok diagram of the principle of the neurocomputer of the present invention, and FIG. 1B is an analog neuroprocessor (A) of the present invention.
A schematic diagram of a bathocage composed of chips of NP), Fig. C is an internal configuration diagram of the ANP of the present invention, and Fig. 2 is a
3 is a block diagram of an embodiment of the basic UNISOTO of the present invention; FIG. 4 is a specific circuit diagram of an embodiment of the basic UNISOTO of the present invention; Fig. 5 is a specific circuit diagram of another embodiment of the basic unisoto of the present invention, Fig. 6 is a diagram explaining the operation timing of the integrator used in the basic unisoto of the present invention, and Fig. 7A is Figure 7B is a conceptual diagram of a layered neural network. Figure 7B is a conceptual diagram of a hierarchical bilayer network according to the present invention. Figure 8 is an example of the neural computer of the present invention realized using a hierarchical neural network. Specific circuit diagrams, Figures 9A and 9B are timing diagrams of the signal processing shown in Figure 8, Figure 1O is a diagram showing the timing of reading digital weight data, and Figure 11A is a mask control diagram. The specific circuit diagram of Bronok, ffii IB diagram is a diagram showing the structure of control 11L pattern memory and microsecond memory, Figure 12A is a diagram showing the method of filling data into the weight y data memory, and Figure 12B is The figure shows a concrete configuration diagram of the weight data memory, Figures 12c and 120 show the learning algorithm of the 7Lonaya-1, and Figure 13 shows a concrete circuit diagram of the digital circuit, ml41'.
4 is Ma Noxuba I! 15 is a specific circuit diagram of a gmoid function generation circuit, 16 is a specific circuit diagram of a sequence generator,
7 Fig. 17 is a specific circuit diagram of the phase control circuit, the 18th leap is a specific circuit diagram of the shift register, Fig. 19A is a conceptual diagram explaining the feedback network, and Fig. 1.13 is a specific circuit diagram of the shift register. FIG. 820 is an explanatory diagram of the case where a feedback network is formed using the neurocomputer of the present invention; FIG. 1 A. 82 and 2iB are timing diagrams showing the signal processing of the embodiment shown in FIG.
23. Specific project diagram of an embodiment in which a second feedback type network is constructed using the neurocomputer of the present invention.・The ζ diagram and FIG. 23B are timing diagrams showing the processing (No. 3) of the embodiment shown in FIG. The block diagram of the example, FIGS. 25A and 25B, are timing diagrams, FIGS.
Figure 6B is a conceptual diagram of the mechanism that generates errors in an actual ANP, and Figures 27 and 28 are error measurement circuits in PIH type and feedback type neural networks that measure this pulse-like error voltage, respectively. Figure shown, second
Figure 9A is a control sequence for measuring offset voltage in the middle layer in a hierarchical network, Figure 29B is a control sequence for measuring gain error in the middle layer in a hierarchical network, and Figure 30A is a control sequence for measuring offset voltage in a feedback network. Control sequence, FIG. 30B is a control sequence for measuring gain error in the feedback network, FIG. 31 is an explanatory diagram of the first and second correction processing used for error measurement of the present invention, and FIG. 32 is: Figures 33A to 33D are flowcharts showing a weight data correction method in the case of hierarchical network work. and FIG. 34B are flowcharts showing a weight data correction method in the case of feedback type net work; FIG. 35A and FIG. Figures 36A and 36
Figure B shows a control sequence for controlling a hierarchical network using multiple dummy nodes, Figures 37A and 37B show a control sequence for controlling a feedback network using multiple dummy nodes, and Figure 38 shows a neuron model. Fig. 39 is a conceptual diagram of the configuration of the NN type neural network. 12... Control pattern memory, 13... Sequencer, l4... Weight memory, L5... Digital control means, l6... D/A converter, l7... A/D converter, 18...・Neural network composed of ANP, 19A... dummy node means,

Claims (1)

[Claims] 1) A neural network consisting of a set of analog neuroprocessors that time-divisionally input analog signals from a first analog bus, perform product-sum operations, and output the analog signals to a second analog bus. (18), a control pattern memory (12) that stores control information of the neural network (18), and a sequencer (13) that generates a signal for accessing an address of the control pattern memory (12); In a neurocomputer that is controlled by a sequencer (13) and has a weight memory that stores weight data for the analog neuroprocessor, when the output voltage value of the analog bus of each layer exceeds a predetermined dynamic range, the input side of that layer A multi-dummy system that expresses the weight data multiplied by the fixed voltage from the dummy node means (19A) connected to the analog bus of the dummy node as the sum of a plurality of weight data, and stores each divided weight data independently in the weight memory. A neurocomputer characterized by having node control means (19B). 2) When the weight data of the dummy node cannot be expressed as a fixed point, it is expressed as a sum of weight data that can be expressed as a fixed point, and each divided weight data is stored independently in a weight memory. A neurocomputer having the multi-dummy node according to item 1.