JPH02226382A - Error absorbing system by weight correction in neuro-computer - Google Patents

Abstract

PURPOSE:To absorb off-set or gain error by calculating correct weight for dummy node from a detection output voltage, which is outputted from a second analog bus, by using gain. CONSTITUTION:A dummy node means 6 is connected to the analog bus of a neural network 18 and generates a fixed voltage in the designated analog bus at the time of a test mode. An weight correcting means 19 in a digital control means determines the intermediate weight of the weight to each neuro- processor to be multiplied with the fixed voltage, which is generated from the dummy node means 6 in a second state at the time of the test mode, from the off-set voltage of each neuro-processor and calculates the correct weight for dummy node from the detected output voltage, which is outputted from the second analog bus, by using the gain. An weight memory 14 stores the weight corrected by the weight correcting means. Thus, weight data for dummy node are adjusted and the off-set or gain error can be absorbed.

Description

[Detailed Description of the Invention] [Summary] Regarding the error absorption method using weight correction, in a time-sharing type neurocomputer that can realize data exchange between basic units configuring a hierarchical network with a small number of wires, weights for dummy nodes are used. The purpose is to adjust the data to absorb offset and gain errors. Analog signals are input from the first analog bus in a time-division manner, a product-sum operation is performed, and the analog signals are output to the second analog bus. A neural network consisting of a set of analog neuroprocessors and a test mode connected to the analog bus of the neural network,
dummy node means for generating a fixed voltage on the designated analog bus; and dummy node means for generating a fixed voltage on the specified analog bus;
error measuring means for forcibly inputting volts and detecting an offset voltage generated in the analog neuroprocessor from the second analog bus; and in a second state in a test mode from the offset voltage of each neuroprocessor. , determine an intermediate weight for each neuroprocessor to be multiplied by the fixed voltage generated from the dummy node means, and use the gain from the detected output voltage output from the second analog bus to generate the correct dummy. Weight correction means in the digital control means for calculating node weights, a weight memory for storing the weights corrected by the weight correction means, and a control pattern for controlling the operation of the neural network are sequentially read out under the control of a sequencer. control pattern memory.

[Industrial Field of Application] The present invention relates to a neurocomputer realized by connecting analog neuron chips through an analog time-division transmission line, and more particularly to an error absorption method using weight correction.

[Conventional technology]

With conventional sequential processing computers (Neumann type computers), it is difficult to adjust the computer's data processing function according to changes in the usage method or environment, so we have developed a new parallel and distributed data processing method using a hierarchical network as an adaptive data processing method. Treatment methods have been proposed. especially,
A processing method called backpropagation method (D,
E. Rumelhart.

G, E, Hinton, and R, J, l+Iil
liams, “Learning Internal
Representations by Error
Propagation +PARALLEL D
ISTRIBUTED PROCESSING,
Vol.1. pp.

318-364. The MIT Press, 1
986) is attracting attention due to its high practicality.

In the back propagation method, a hierarchical network is constructed from a type of node called a basic unit and internal connections with weights. FIG. 3F shows the basic configuration of the basic unit 1. This basic unit 1 performs processing similar to a continuous neuron model. In other words, this is a multi-car output system, which includes a multiplication processing unit 2 that multiplies a plurality of inputs (yh) by the respective internal connection weights (Wih l), and an accumulation processing unit that adds all the multiplication results. 3 and a threshold processing section 4 that performs nonlinear threshold processing on this added value and outputs one final output X8.

FIG. 36 is a conceptual diagram of the structure of a hierarchical neural network. A large number of elementary units of the configuration (1h, 1-i, 1
-jl are connected hierarchically as shown in FIG. 36, so that an output signal pattern corresponding to the input signal pattern is output.

During learning, the weights of the connections between each layer (Wu,
) is determined. Such learning is performed for multiple input patterns and multiplexed. In addition, during association, even if the human cover pattern is incomplete information that is slightly different from the complete information input during learning, associative processing becomes possible by obtaining a cover pattern close to the teacher pattern during learning.

In order to make a neurocomputer with such a configuration a reality, it is necessary to realize data exchange between the basic units 1 that constitute a hierarchical network using as few wiring lines as possible. This problem cannot be solved because in order to realize complex data processing, it is necessary to make the hierarchical network structure more layered and increase the number of basic units. This is one of the issues that must be addressed.

[Problem to be solved by the invention]

However, in the data transfer method explained earlier, the third
As is clear from the hierarchical network configuration shown in Figure 6, the number of wires between the two eyebrows is extremely large.
When a hierarchical network is made into a chip, there are problems in that it becomes impossible to make it smaller and it becomes impossible to improve reliability. For example, if the number of basic units in two adjacent layers is the same and we assume a perfect connection in which all basic units 1 are connected to each other, the number of wires will increase in proportion to the square of the number of basic units. become.

In this way, the number of wires increases rapidly.

An object of the present invention is to absorb offset and gain errors by adjusting weight data for dummy nodes in a time-sharing neurocomputer that can realize data exchange between basic units forming a hierarchical network with a small number of wires. shall be.

[Means to solve the problem]

FIG. 1A is a system block diagram of the neurocomputer of the present invention.

The neural network 18 consists of a set of analog neuroprocessors that time-divisionally input analog signals from a first analog bus, perform sum-of-products operation, and output the analog signals to a second analog bus, and includes dummy node means. 6 is connected to the analog bus of the neural network 18 and generates a fixed voltage on the specified analog bus in the test mode, and the error measuring means 20 is connected to the analog bus of the neural network 18 in the test mode. Forcibly inputting 0 volts to the bus via the dummy node means 6, detecting the offset voltage generated in the analog neuroprocessor from the second analog bus, and the weight correction means 19 in the digital control means, In the second state in the test mode, from the offset voltage of each neuroprocessor, an intermediate weight of the weight to be applied to each neuroprocessor to be multiplied by the fixed voltage generated from the dummy node means 6 is determined, and the second The correct weight for the dummy node is calculated using the gain from the detected output voltage output from the analog bus, the weight memory 14 stores the weight corrected by the weight correction means, and the neural network 18 calculates the correct weight for the dummy node by using the gain gain from the detected output voltage output from the analog bus. An analog signal consisting of a control pattern memory 12 in which a control pattern for controlling a control pattern is sequentially read out under the control of a sequencer 13 is input in a time-sharing manner from a first analog bus, and a product-sum operation is performed to convert the analog signal into a second analog signal. The dummy node means 6 is connected to the analog bus of the neural network 18 and generates a fixed voltage on the specified analog bus in the test mode, and the error measuring means 20 is the first state in the first state in the test mode.
0 volt is forcibly input to the analog bus of the second analog bus through the dummy node means 6, an offset voltage generated within the analog neuroprocessor is detected from the second analog bus, and a weight correction means 19 within the digital control means is detected. teeth,
In the second state in the test mode, from the offset voltage of each neuroprocessor, an intermediate weight of the weight to be applied to each neuroprocessor to be multiplied by the fixed voltage generated from the dummy node means 6 is determined, and the second The correct weight for the dummy node is calculated using the gain from the detected output voltage output from the analog bus, and the weight is stored in the weight memory 14.
stores the weights corrected by the weight correction means, and control patterns for controlling the operation of the neural network are sequentially read out from the control pattern memory 12 under the control of the sequencer 13.

[For production]

An analog neurochip is created by inputting an analog input signal into the analog neurochip in a time-division manner, multiplying this signal with weight data, and outputting the sum-of-products signal obtained by adding these product signals through a nonlinear function circuit. Configure. A hierarchical or feedback neural network 18 using multiple analog neurochips
A control signal output from the control pattern memory 12 to which an address to be accessed is given by the sequencer 13 is added to the neural network 18. Further, the neural network 18 is supplied with weight data obtained through learning or the like from the weight memory 14.

The neural network 18, control pattern memory 12, sequencer 13, and weight memory 14 are controlled and managed by digital signals from the digital control means 15. Moreover, the MPU in the digital control means 15
In particular, a learning algorithm is executed, output signals are checked, etc. In this way, an analog neurocomputer system characterized by using time-division analog input signals and time-division analog output signals is realized.

〔Example〕

FIG. 1B is a schematic diagram of a dual in-line package of an analog neuroprocessor (ANP) 11 comprised of the neurochip of the present invention.

This is called MB4442 and executes neuron model processing. The internal dark value processing section is a model in which it is replaced with a sigmoid function. Analog neurochip is A
It is called a NP and is a device that outputs analog data manually. FIG. 1C is an internal configuration diagram of the ANP of the present invention.

As shown in FIG. 1C, ANP II is connected between analog bus B1 and analog bus B2. ANPII is an analog multiplier 22 that multiplies the input analog signal and weight;
It consists of an analog addition section 23 that calculates the sum of products, a sample/hold section 24 that holds the sum, and a nonlinear function section 25 that outputs the value of the sigmoid function. Each terminal of ANPII in FIG. 1B will be explained. The inside of ANPll is composed of an analog circuit section and a digital circuit section. The 1-6 volt terminal is the power supply terminal supplied to the operational amplifier of the analog circuit section. D in andri. ut is a terminal for analog input signals and output signals. AGND is the ground terminal of the analog circuit section. The R1+ and Rt1 terminals are the terminals of the external resistor R of the integrating circuit in the analog circuit section, and the Ct1+ and Ct1 terminals are the terminals of the external capacitor C of the integrating circuit. DGND is a ground terminal of the digital circuit section. +5 volts is the power supply terminal for the digital circuit section. R3T is a reset signal terminal for resetting the charge of the capacitor of the integrating circuit. C3I and C8O are input/output terminals for daisy chain control signals, OC is a terminal for offset cancel control signals, and S/H terminal is
Control signal terminal for sample/bold, 5YNC is a synchronization signal terminal for processing of each layer, DCL K is a basic clock signal terminal for processing analog input signals, WCL
K is a clock terminal for taking in digital weight data, and WD is a terminal for manually inputting digital weight data in a bit serial manner.

FIG. 2 shows the analog neuroprocessor (AN) of the present invention.
It is a principle block diagram of P).

Analog human signals sent in a time-sharing manner from separate ANPs (not shown) are transferred from analog bus B1 to ANPll.
The analog multiplier 22 multiplies the input signal with the digital weight data WD, which is input bit serially via the shift register 27 and is then serial-parallel converted, to obtain the analog input signal and the digital weight. Obtain a product signal indicating the product with the data. The next analog adder 23 is a Miller integration circuit consisting of an external resistor R and a capacitor C, and is connected to a plurality of ANPs in the previous stage connected to the analog bus B1 (the place where an ANP exists is called a node). The sum of the product signals obtained from the analog input signal sent in a time-division manner and the dark value analog input signal sent from the dummy node is calculated. next,
After the product signal is held in the sample/hold unit 24 for a desired time, the sampled/bolded output is further converted via the nonlinear function unit 25. The output control unit 26 outputs an analog output signal after a predetermined time delay under the control of the sequence generator 28. Output ut to analog bus B2. Note that the sequence generator 28 also generates control signals that are supplied internally. The phase control section 29 mainly controls the phase of the control signal so that each switch connecting the analog circuit section and the digital circuit section within the ANP is reliably turned on or off. When the second switch is turned off when the second switch is on, the phase of the control signal is controlled so that the two switches are not turned on at the same time.

Note that the sequence generator 28 receives a reset signal R.
3T,,DCLK,WCLK,5YNC.

Inputs S/H, QC, , C3I from the mask control block described later, outputs C8O, and outputs ANP.
Generates internal control signals.

Neural networks need to perform high-speed calculations through simultaneous processing. Although time-division data is used in the present invention, in a steady state, each ANP performs simultaneous processing in a pipeline manner. In an ideal neural network, each neuron would need to be interconnected with each other, but if the system were to be implemented as is, the number of wires would be large. Therefore, in the present invention, since time-division data is handled, the processing time for the sum of products in each ANP increases, but by arranging the chips in parallel in the vertical direction, that is, in the direction of the same layer, the neurochips in the layer can be AN to configure
Simultaneous processing of P improves the processing time. Also,
Each layer can be pipelined, which also reduces processing time. For example, when an input is input to each of three neurochips connected to an analog bus, it is input to all three at the same time, and in parallel, each ANP generates a product of the weight and the analog voltage. and holds it as a charge in the integrator capacitor.

Then, in the next time period, for the analog input of the same analog bus, each ANP will form a product with a weight and add it to the product determined in the previous time period in the integrator capacitor. After the sum is generated for the product of the weights on the analog input signals from all previous ANPs, the sum is sampled/held. Thereafter, it is output via a sigmoid function, which is output when the C3I control signal is input. Then, when the output is completed, C3I falls, and after a certain time delay, C3o rises, giving the right to use the output bus to the ANP consisting of adjacent neurochips in the same layer.

FIG. 3 is a block diagram of a first embodiment of the basic unit which is a neurochip. The multiplication section 32, addition section 33, and threshold processing section 34 in the figure are continuous neuron model execution sections, but in this embodiment, an output holding section 35 is present. Specifically, the plurality of human power connected to the basic unit 31 is
-, and the weight set corresponding to each connection is Wi, then the multiplier 32 calculates Y i −W i and the adder 33 calculates X−ΣYi −Wi−θ. However, θ is a dark value. If the final output of the dark value section 34 is Y, then Y= 1/ (1+ exp (-X))...
・(11 will be calculated.

“−θ” is applied to the value “+1” input from the dummy node.
”, and the adder 33 outputs the result of “Xθ”. Therefore, in the threshold section 34, only the conversion using the S-shaped curve is performed.

The multiplication section 32 is composed of a multiplication type D/A converter 32a, and receives an input of an analog signal (input via an input channel section 37) from the basic unit 31 in the previous layer or from a dummy node circuit described later. , the input is multiplied by weight information 1 of the digital signal to be multiplied (input via the weight holding unit 38 described later), and the obtained multiplication result is output as an analog signal. The adding section 33 is composed of an analog adder 33a formed of an integrator and a holding circuit 33b that holds the addition result of the analog adder 33a. Multiplying D/A
The converter 32a inputs an analog input signal to the reference voltage terminal of the D/A converter, and inputs each bit of the weight to each digital input terminal as a digital input signal.As a result, the analog input signal and the weight are generate the product of The analog adder 33a is a multiplication type D/
A new added value is obtained by adding the output of the A converter 32a and the added value previously obtained and held in the holding circuit 33b.The holding circuit 33b adds the added value obtained by the analog adder 33a. At the same time, the held value is fed back to the analog adder 33a as the previous addition value. These addition processes are executed in synchronization with an addition control signal output from the control circuit 39. The dark value section 34 is composed of a nonlinear function generation circuit 34a that is an analog function generation circuit, and outputs a nonlinear signal such as a sigmoid function in response to an input.

When the accumulation of the multiplication results including the addition of the threshold value (-θ) is completed, the addition value X held in the holding circuit 33b
The analog output value Y is obtained by adding a threshold value (-θ) to (calculating the sigmoid function of Equation 11). This is to hold the output value Y of the analog signal of the nonlinear function generating circuit 34a, which is the output of the nonlinear function generating circuit 34a.

Further, 36 is an output switch section, which processes so as to output the final output held by the output holding section 35 onto the analog bus B2 by turning ON for a certain period of time in response to an output control signal from the control circuit 39. Reference numeral 37 is an input input unit, which receives an input control signal from the control circuit 39 and accepts input by turning ON when an analog output from the final output is sent from the basic unit 31 in the previous layer. . Reference numeral 38 denotes a weight holding unit, which is composed of a parallel out shift register, etc., and receives the bit serial weight signal sent from the weight memory when the gate of the buffer 38a is opened (the weight input control signal from the control circuit 39 is turned on). At times, this weight signal is held as a bit-parallel weight required by the multiplier 32. The hint parallel weights are applied to the multiplier in parallel when the multiplication control signal is applied. Reference numeral 39 denotes a control circuit of the digital circuit section, which generates an internal synchronization signal from an external synchronization signal, and controls internal analog processing functions.

With this configuration, the input and output of the basic unit 31 having the signal processing configuration shown in FIG. 3 can be realized using analog signals.

Note that the multiplication type D/A converter 32a may be configured to receive and receive the weight information of the digital signal in parallel, or may be configured to receive and receive the weight information in serial and then perform parallel conversion. Alternatively, if the weight information is constituted by an analog signal, an analog multiplier can be used instead of the multiplication type D/A converter 32a.

Figure 4 shows the neurochip (ANP) of the present invention of III.
) is a specific circuit diagram of an embodiment.

In this unit, an input section 42, a multiplication section 43, an addition section 44
, a sample/bold section 45, a nonlinear function section 46, and an output section 47. Here, it is assumed that there is no output holding circuit, and the sample/hold section 45 also has an output holding function.

The input section 42 includes an offset canceling section 51 and a 1x buffer 49. The 1x buffer 49 is a voltage follower and is configured by feeding back the output of an operational amplifier to one terminal and inputting the input voltage to a child terminal. The data input is an analog time-shared pulse signal.

OC is an offset control signal, and when this is 1, the analog switch 66 is turned on, and the 1x buffer 4 is turned on.
9 is forcibly set to 0 voltage. On the other hand, when the offset control signal OC is 0, the analog switch 66 is turned off, the other analog switch 65 is turned on, and the data input is input to the 1x buffer 49. That is, when the offset control signal OC is 1, O volts are forcibly input to the neuron unit so that the neuron unit performs the operation of canceling the offset voltage generated at the operational amplifier output of the circuit up to the multiplier output. ing.

In the figure, switching of the analog switches 65 and 66 is controlled by the inverted phase and positive phase of the OC signal, but a phase control circuit prevents them from being turned on simultaneously. Hereinafter, this will be referred to as the OC being "phase-controlled."

The positive/negative switching circuit 52 is constructed by cascading two multipliers. In a multiplier, the input resistance (10Ω to 10Ω) and the feedbank resistor (10 to Ω) form an inverted version of 10/10, or 1x the voltage, which is then
The sign of the analog voltage is determined depending on whether only one stage or two stages are passed.

The control signal is the code bin of digital weight data) (S
IGN), and this S'lGN bit is connected to the gate of the MOS switch 70. This 5IGN control signal is also phase controlled. When the sign bit 7 is 1, the input voltage from the input section 42 is inverted by the first-stage multiplier, and since the switch 67 is also on, it passes through the subsequent multiplier, resulting in a positive phase. becomes. Further, when the sign bit is O, the switch 69 is turned on via the inversion circuit 68. At this time, the switches 67 and 70 are off, so the input terminal from the input section 42 is connected to the switch 69.
The signal is inputted to one terminal of the operational amplifier 71 at the subsequent stage via the .

Therefore, a multiplier is formed by the resistor 72 at the front stage and the feedback resistor 73 of the operational amplifier at the rear stage, and the signal is multiplied by 1 and inverted. That is, the input to the human power unit 42 is formed as a positive or negative voltage depending on the sign bit, and this becomes a voltage according to excitatory and inhibitory synaptic connections. The output from the positive/negative switching circuit 52 is sent to the multiplier 43
The voltage is input to point 74 of the R-2R resistor network of the D/A converter 53 located in the D/A converter 53, that is, to the reference voltage terminal.

First, the R-2R type D/A converter will be explained. M.S.
The internal switch is turned on or off depending on the digital weight from B to ISB. If the digital value is 1, current flows through the right switch 75 to the virtual ground point 78 of the operational amplifier 76. operational amplifier 7
The virtual ground point 78 of 6 is controlled to have the same voltage as the 10 terminal, and since this is the ground, it is virtual 0 volt.

In the D/A converter 53, R is IOKΩ, 2R is 2
0 to Ω. Regardless of the state of the switch, current flows through the 2R resistor, and it is determined whether the weight current flowing through the 2R flows toward the virtual ground point 78 according to the value of the digital value. Let i be the current flowing through 2R of 1 fragrant stone.

The current of 2R corresponding to the second from the right, that is, LSB, is 1
Since it is the value obtained by dividing the voltage applied to 2R of the incense stone by 2R, 2
RXi÷2R is i? 6υ-・'A current 21 flows in the R in the lateral direction of the first scented stone. 2R is the third 2R from the right
A voltage of X i + RX2 i is applied, and this is divided by 2R, so a current of 21 flows. In the same manner, the current becomes 4i and 8i as it goes to the left, increasing as a power of 2. It is from MSB to LSB that determines whether or not the weighted current, which is a power of 2, flows toward the operational amplifier. Therefore, a current corresponding to the digital weight flows into the virtual ground 78 in the form of a power of two, and since the input impedance of the operational amplifier 76 is infinite, this current flows into the feedback resistor 78 of the operational amplifier 36. Therefore, the output voltage V of the D/A converter. If the input voltage is E, ut is Vout" × (Do +2XDI +22XDz 2"10...+2" MDll-1). Here, Do is the LSB, and Dll-1 is the MSB.
Suppose that That is, the output of the multiplier 43 is equivalently a value obtained by multiplying the input voltage E by weight.

The weighting coefficients are controlled by digital values manually entered from MSB to LSB. On the other hand, the adder 44 performs a cumulative addition operation for the volume of each voltage of the time-division multiplexed analog signal and the digital weight data by using a Miller integrator in a time-division manner. And sample/
A hold circuit 45 samples/holds the addition result.

Next, the adding section 44 will be explained. The adder 44 is an integrator including a resistor R and a feedback capacitor C. A time division addition control section 55 is provided at the input section of the addition section 44, and when the phase-controlled sample/hold signal S/H signal is 1, the output voltage of the multiplication section 43 is inputted to the virtual ground point 79 of the operational amplifier. S
When the /H signal is 0, the switch 81 is turned on by the inverting circuit 80 and the output of the multiplier 43 is connected to the ground via the resistor R, so that it is not added to the feedback capacitor C of the adder 44. Now, when the S/H signal is 1, the output voltage of the multiplier 43 is applied to the operational amplifier 102 via the resistor R.
A current obtained by dividing the input voltage by the resistor R is input to the feedback capacitor C via virtual ground. After this, the S/H signal becomes O again and the multiplier 43 and the adder 44 are separated, so the multiplier 43 can multiply the next input signal by the weight data. An offset canceling function is added to the feedback circuit 82 of the integrating circuit including the capacitor C using four switches. Assuming that the offset control signal OC is now 1, switches 83 and 84 are on, and switches 85 and 86 are off. When the offset control signal OC is O, the data input section 42 and the data input terminal DATA-INPU
An input voltage is applied to T, and the corresponding output of multiplier 43 is input to capacitor C via resistor R. At this time, the switches 85 and 86 are on, and the polarity of the capacitor C is either - on the side connected to one terminal of the operational amplifier, or positive on the side connected to the output of the operational amplifier 102.

Next, when the offset control signal OC is 1, the data input is forced to 0. In this case, the positive/negative switching circuit 42 and the D/A converter 53 of the multiplication section 43
If there is no offset, the output of the D/A converter 44 will be 0 volts. However, op amp 49
．． 103.71.102, an offset voltage is generated, and the offset voltage is applied to the capacitor C of the adding section 44.
is stored in In this case, unlike the case where the previous offset control signal OC was O, the switches 83 and 84 are turned on, and the polarity of the capacitor C is reversed. Therefore, by setting the offset control signal ○C to 1, the polarity of the capacitor C changes, and as a result, the offset voltage that occurs when an input signal is input manually is canceled. In this way, the present invention is configured to have an offset canceling function isometrically by using the polarity reversal of the capacitor C. Note that the switch 87 is controlled by a reset signal, and when the reset signal is applied at the start of processing, the voltage of the capacitor Ch is set to zero, and the output of the adder section is forcibly reset to zero. It is assumed that this OC signal is also phase controlled.

The output of the adder 44 becomes the input of the sample/hold circuit 45. In the sample/hold unit 45, when the phase-controlled sample/hold control signal S/Hout is 1, the output of the adder 44 is stored in the capacitor Ch via the switch 88.

When the S/Hout control signal is 1, the control signal of the switch 90 is set to 0 by the inverting circuit 94, one terminal of the capacitor Ch is not grounded, and the switch 91 is turned on, so that the final terminal of the unit is The output signal is input to the capacitor Ch via the switch 91. That is, the final output signal at that time is fed back from the output terminal of the operational amplifier 96 to the capacitor C6.
given below. Therefore, the voltage obtained by subtracting the value of the final output signal from the output of the adder 44 is held in the capacitor C. On the other hand, when the S/HouL control signal is 0, the switches 89 and 90 are turned on, and the lower side of the capacitor Ch becomes ground, and as a result, the final output value is subtracted from the voltage stored in the capacitor C, that is, the output of the adder 44. The operational amplifier 9 whose voltage value is 1 times the voltage value through the switch 89
The operational amplifier 93 functions as a buffer, and the output of the operational amplifier 93 becomes the input to the sigmoid function. Further, when the S/Hout control signal is 1, the switch 88 is turned on, and when the capacitor C6 stores a voltage equal to the difference between the output value of the adder and the final output value, the switch 92 is turned on. Therefore, O volts are forcibly input to the operational amplifier 93. At this time, the sigmoid function 46 and the operational amplifier 96. An offset voltage ΔV is inputted via the analog switch 100 to the lower side of C, , via the switch 91. Therefore S
/Hout control signal is O, that is, when switch 89 is on and switch 92 is off, C6
The voltage stored in , that is, (output of the adder - offset voltage Δ■) becomes the final output via the operational amplifier 93 and the sigmoid function 46, but when the S/Hout signal becomes 1, the offset voltage generated at this time Since is also Δ■, as a result, the offset voltage is canceled.

The nonlinear function section that generates the sigmoid function has a nonlinear circuit selection control section, and the phase-controlled Se151g signal is
When the switch 95 is turned on, the output of the sigmoid function is input to the next stage. However, when the Se151g signal is 0, the control signal of the switch 98 becomes 1 via the inversion circuit 97, turning it on, and the output of the sigmoid function is cut. That is, when the Se151g signal is 0, the output voltage of the sample/hold section is directly input to the operational amplifier 96 without going through the sigmoid function. operational amplifier 96
is essentially a 1x operational amplifier that feeds the output directly back to one terminal, acting as a buffer. In other words, it becomes a buffer that sets the output impedance to O.

A time division analog output section 64 and an output control section 63 are connected to the output section 47 . When C3I is 1, switch 99 is on and switch 101 is also on, so the final output value of operational amplifier 96 is.

It is output to DATA -0tlTPUT and fed back to one terminal thereof, so that the operational amplifier 96 functions as a 1x operational amplifier. At the same time, the final output value is fed back to the sample/hold section 45. on the other hand,
When C3I is 0, switch 1 is connected via inverter 104.
00 is turned on and switch 101.99 is turned off. In other words, the output of the operational amplifier 96 is DATA -011
It will not be output to the TPUT line. However, since a one-time buffer is formed by turning on the switch 100, the voltage follower operation of the operational amplifier 96 is performed without being destroyed. The output section 47 is a circuit that determines whether or not to transmit the output pulse voltage based on the output control input signal C3I. This C3I is outputted as C3O via the delay circuit 105, and the time timing of the output analog signal to the adjacent neurodep within the layer is determined.

Therefore, in the present invention, since the analog signal from the output section 47 is transmitted in a time-division manner, it does not compete with analog signals from other neurochips on the bus.

In FIG. 5, offset cancel OC is 0CO1OC1 and sign 5IGN is PN in FIG.

PN, sample/bold SH to 5HII, 5HIO,
Sample/Hold S/Hout 5H21, 5H20
, sigmoid selection signal Se151g -S IGM,S
IGM, daisy chain signal C3I cs, -c
Phase control is realized using two signals at s. In other words, by configuring one control signal as two signals, one in positive phase and the other in reverse phase, and shifting the phases, other switches controlled by the positive and negative phases of these control signals will not turn on at the same time. This is an example in which a signal is generated as shown in FIG. In addition, D
/A capacitor C connected to the output terminal of the A converter 53
1. The resistor Rf is for adjusting the feedback signal of the operational amplifier 76 to the calculation speed of the D/A converter, and the digital input of the D/A converter is applied to the DT terminal.

In FIG. 5, the same parts as in FIG. 4 are given the same numbers, and the explanation will be omitted.

FIG. 6 is a timing diagram for an integrator based on the weighted error method of the present invention. data clock dc
LK and the weighted clock WCLK are basic operating clocks, and the high-speed weighted clock WCLK is output during a half cycle of the high state of the data clock DCLK. weight clock W
The CLK signal is a synchronous clock for capturing weighted serial data. The data clock DCLK signal is a basic clock for processing analog input signals. The synchronization signal 5YNC is a synchronization signal for synchronizing each analog neuron processor ANP in one layer in each layer. The change in the output voltage of the integrator is shown by the waveform shown by the triangle at the bottom.

The integration waveform is controlled by pulses of the sample/hold control signal SH, and while this pulse is high, an integration operation is performed. That is, when charging of the capacitor C of the integrator is started, and while the pulse of the sample/hold control signal SH is high, charge is gradually accumulated in this capacitor and the voltage increases, but the voltage of the sample/hold control signal SH is high. When the pulse becomes low and is cut off, the charging operation is stopped.

Therefore, only the charge amount within this integration time range is meaningful. The voltage is proportionally distributed depending on the pulse width of the sample/hold control signal, that is, the input voltage is multiplied by the integral gain. That is, when the pulse width of the input sample/hold control signal S/H is P, the voltage charged to the capacitor C is 3, and when the pulse width of the sample/hold control signal S/H is W, the charging voltage is V. , l.

The sample/hold control signal SH falls, the polarity of the integrator capacitor changes due to switching control, and the integrated output to which the offset has been added is inverted. and,
When the sample/hold control signal SH rises again while the offset control signal OC is in a high state, the offset voltage ■5 (vb') is added to the capacitor, and when the SH multiplier signal falls, the offset is canceled as a result. The integrated output value Va-V.

(V,'-V,') is sampled/held with its polarity restored.

Next, a hierarchical neural network will be explained. 7th
Diagram A is a conceptual diagram of a hierarchical network. In the hierarchical type, input data input from the input node 110 of the input layer on the left side is processed in only one direction, sequentially toward the right side. Each neuron 112 in the middle layer is a dummy node 11
The outputs of the previous layer including 1 are received by complete connections within each layer. If there are, for example, four input two-doors 110 in the input layer, one dummy node 111 is added to them, and when viewed from each neuron 112 in the intermediate layer, the input layer appears to be five neurons. Here, the dummy node 111 controls the threshold, and is a value f (X −θ). This is equivalent to changing the weight corresponding to the dummy node 111 within the neuron, but the constant value θ is generated using a max value node circuit, which will be described later. In this way, by preparing weights for dummy throats, it is possible to express darkness values using weights. It appears that there are four neurons in the intermediate layer starting from neuron 112 in the output layer. The input data applied to the input layer is subjected to a sum-of-products operation using weight data in the intermediate layer neuron 112 and the output layer neuron 12, respectively, and output data is generated as a result.

When the hierarchical structure shown in FIG. 7A is realized using the ANP of the present invention, as shown in FIG. Each independent analog bus B1. B2. It is necessary to provide B3.

The structure is such that all vertical ANPs can be executed in parallel. A sample and hold circuit SH is attached to the output of the output layer.

FIG. 8 is a block diagram of a neurocomputer of the present invention that implements a hierarchical neural network. Analog neuron processor ANP 1~ from neurochip
5 are arranged in parallel on each layer, and analog buses (Bl, B2, B3) are provided independently between each layer. In the same figure, ANP
An intermediate layer is formed using i2.3, and an output layer is formed using ANP4.5.

Also, there is no ANP in the input stage, and the input side has a daisy circuit 17 for inputting analog input signals with good timing.
1.172 exists. The circuit indicated by S/H is a sample/
These are hold circuits 173 and 174. Since each of ANP1 to ANP5 requires logic signals for control, many control signal lines are sent from the mask control block (MCB) 181 to each layer. data clock D
CLK is the daisy circuit 171 on the input side of all ANPs.
and 172, and serves as the basic clock for analog processing. Weight clock WCLK is also given to all ANPs and input side daisy circuits 171 and 172, and is a high speed clock for weight data.

Each ANP4.5 from weight memory blocks 185 and 186
Weight data is input to ANPI and ANPI 2.3 in synchronization with the weight clock WCLK.

Also, the synchronization signal 5YNCI is the layer synchronization clock given to the intermediate layer ANP, and the synchronization signal 5YNC2 is the output layer ANP.
This is the layer synchronization clock given to the NP. SHI and O
Cl is a sample/hold control signal and an offset control signal for the ANP in the intermediate layer, and SH2 and OC2 are a sample/bold control signal and offset control signal for the ANP in the output layer.

Daisy circuits 171 and 172, which are blocks on the left, are
This is an input side circuit corresponding to the human power layer. In order to realize an input node, that is, a neuron in the input layer, the analog input signal given from the analog input port 0°1 must be input into the circuit at the same timing as the ANP outputs the analog signal in time division. It won't happen. In other words, from the perspective of the output layer, the ANP of the output layer is 4.5, which is the ANP of the previous intermediate layer.
The basic operation is to receive analog signals from 1 and 2.3 in a time-division manner via analog bus B2. This same relationship must exist in the intermediate layer and the input layer.

The relationship between the input layer and the intermediate layer must be such that when viewed from the intermediate layer ANP, it appears that the input layer ANP is in front of it. This means that the middle layer ANP has the same function as the timing for outputting an analog signal to the analog bus B2, and the analog human input signal from the analog input port 0.1 must also be output to the analog bus B1 according to a fixed rule. There is a restriction that it cannot be done. i.e. analog input port 0.1
The input signals from the input terminals are time-divisionally routed onto the analog bus B1. The analog signal from the analog human-powered boat O gets on the analog bus B1 at an appropriate timing, but at the next timing after being output there, the next analog input signal from the analog input boat 1 gets on the same analog bus B1. In order to achieve this synchronization, the daisy circuit 171 manually outputs an input control signal C8I that is output at a certain timing, and after a certain period of time, an output control signal C8O is output from the circuit. This C3I is output from C3O1 of the mask control circuit 181. Daisy circuits 171 and 172 are a type of delay circuit.

When each daisy circuit 171 inputs the input control signal C3I from the mask control 181, it transmits the C8O signal to the next vertically adjacent daisy circuit 172 so as to output the analog output signal of the analog input port 1. will be handed over to.

This operation is called daisy control.

When C3O1 of the mask control circuit 181 rises, the switch 175 is turned on, and the analog input signal of the analog input port O held in the sample/hold circuit 173 is transferred to the analog bus B1. Since C3OI is C3I of the daisy circuit 171, C8O rises after a certain period of time after C3OI falls. This is C3I of daisy circuit 172 as well as switch 176.
is controlled and turned on, so that the analog input signal of analog input port 1 held in sample/hold circuit 174 is placed on bus B1. This system, which has a hierarchical structure, requires this daisy control. In other words,
The analog input signal is connected to the analog bus B1 from the analog input port O through the sample/hold circuit 173.
Then, the analog input signal is output from the analog input port 1 to the same analog bus B1 via the sample/hold circuit 174. Looking at each neuron in the intermediate layer, the analog input signal from analog human-powered boat 0 and the next analog input signal from analog input boat 1 are inputted sequentially in a time-sharing manner.

Each daisy circuit 171, 172 outputs an output control signal C8O by delaying the input control signal C3I by a specific time to prevent bus contention on the analog bus B1.

Also in the middle layer, the mask control block 181
AN that receives the output control signal C3O2 from as C3I
When P1 outputs an analog signal, C8O is passed to ANP2 as C8I, and then ANP2 outputs it. AN
ANP3, which receives C8O of P2 as C3I, will output an analog signal next. In short, here AN
PI and 2°3 are output in this order, and the daisy operation of the intermediate layer is completed.

In parallel with this, the mask control block 181, which manages all operations, supplies C3O3 to ANP4 of the output layer.
When given, ANP 4 outputs, and after the output is completed, ANP 4
gives C8O to ANP5, ANP5 outputs.

The output from ANP4.5 of the output layer is the C8O3 signal and ANP from the mask control block 181, respectively.
The sample/hold circuits 177 and 178 sample/hold the daisy-chain output control signal C80 from the output signal C80. This output voltage is output as an analog output signal from the analog output port 0.1, and is also selected by the analog multiplexer 179 and then
/D converter 180 performs A/D conversion, and MPU 182
, a memory 183, and a communication interface 184. And MPU
At step 182, for example, it is compared with a teacher signal stored in the MPU given during learning to check whether it is a desired output signal, and based on this result, weight data in a weight memory, which will be described later, is changed. Max value node circuit 187 receives dummy node control signals DC3L and DC32 from mask control block 181 to output enable 1 and 2, and output terminals are connected to analog buses Bl and B.
Connected to 2.

FIG. 9A is a timing diagram of the hierarchical neurocomputer according to the embodiment shown in FIG. 8.

Control signal lines are extracted and written for each layer. ,
First, the data clock DCL, which is the basic operating clock.
K and the weight clock WCLK enter all the ANPs in the same layer and the daisy circuit 171172 on the input side at the same time.

The weight clock WCLK is a serial synchronization pulse for serially sending the digital data of the weight, and is a synchronization clock for reading the weight from the weight memory block. The timing at which input data is taken in is determined by each control signal. First, in the timing chart of FIG. 9A, C801 is the daisy-chain control signal C801 output from the mask control block 1131, that is, the daisy-chain control signal C3I to the daisy circuit 171. Daisy circuit 17
1, C3I outputs the first analog input signal from analog input port 0 to sample/hold circuit 5H17.
3 to the analog bus B1. That is, the analog signal is output to the analog bus B1 at point (■) in the timing chart. At this moment, analog bus B1
Voltage is applied to the top, ANPI, ANP2. ANP3 performs a sum-of-products operation on this analog signal in parallel.

The C8O passes through the daisy circuit 171, and after a predetermined time after the C8O falls, the next C3I rises as shown in (■), and the C8I enters the daisy circuit 172. The next C8 is a control signal that enters the second daisy circuit 172 in the input layer.

Then, while C3I is high, the sample/hold circuit S H174 samples the analog input signal from analog input port 1.
ANPI, ANP2. The signal is input to ANP3, and sum-of-products calculation is performed here. Mask control block 181
Kamono II) C31 is a control signal to the dummy node. In addition to the input, each layer has a signal from a dummy node, so the number of Neelon nodes + 1) nodes is required, and the input layer requires 2 people, but from the ANP of each middle layer, it requires 3 people. appear. To explain this in terms of time, two C3Is and one DC31 constitute one control signal. The input cycle is the first C3
It starts with I and ends with the input to the dummy of DC31. The dummy node is a Manx value node circuit 187, which outputs a fixed threshold voltage on the analog bus while DC3I is input. In other words, as shown by ■, while this voltage is being output after DC3I rises, each ANP in the intermediate layer performs a sum-of-products operation in the same way as a normal input, and the fixed voltage is used as the previous two analog input signals. will be added to the result of the product-sum operation. In other words, after multiplication, addition is performed. 5YN
CI goes high at the falling edge of DCLK before csoi rises, and the next D after C31 rises.
It becomes low at the falling edge of CLK. This is the signal that synchronizes the input layer. While WCLK is being input manually, the analog input and weight data are multiplied.

Two peaks M1 and M2 are output to the sample/hold signal SH1 that enters the ANP of the intermediate layer, and the product is multiplied slightly before the first peak M1, and the sum is generated at the peak and held. Then, at the next peak M2, the offset voltage Vb
(See Figure 6) and sample/hold.

Such processing is sequentially repeated for all input analog signals, and the calculation of the sum of products is completed. In this case, each ANP in the middle layer, including the dummy, executes the sum-of-products operation three times. This completes the processing of each ANP in the middle layer, and ends the addition of products for the three human forces.

Also, in the timing chart, when DCLK is high immediately after DC31 falls, analog 2 covers 0.1. The result of the product-sum calculation for the three signals from the dummy node is each ANPI, a 2°3 capacitor (Fig. 4,
It is held at 06) in the sample/hole 1 section 45. This kind of operation is basically repeated, but AN
When to output the Pl output signal is determined by the rising edge of the C802 signal output from the mask control block 181.

Offset control control signal O shown below SHI
CI performs offset cancellation inside ANP. That is, each ANP is an analog circuit that internally includes an operational amplifier, and since the circuit itself has an offset, the control signal for canceling this offset is the OC signal. As shown in OCl, one pulse is output every time one product-sum operation is executed, and offset cancellation is executed internally. In the timing chart, as shown by ■, when C3O2 rises, the signal held in ANP1 is output from ANPI to analog bus B2, and while C3O2 is high, ANP4 in the output layer performs a sum-of-products operation. C3 indicated by ■
The rise of O2 is the timing for outputting the product-sum result of the previous input results.

Next, the timing between the intermediate layer and the output layer will be explained using FIG.

In the figure, outputs of daisy chain control signals from the intermediate layer ■, ■, ■, ■ and outputs from the output layer ■,
The analog signal that is applied to the analog bus in synchronization with ■ is processed by the analog signal that is input on the analog bus in synchronization with the output of the daisy chain control signal from the input layer described above ■, ■, ■. The results from before the cycle will appear. The execution of pipeline processing will be explained later, but
At the rise of C802, which is indicated by ■ in the timing chart, the ANPI output is output.

When looking at the S H2 signal (timing chart) shown in (2) at the rise of C3O2, two pulses are output. In the block diagram of FIG. SH2H2O28, it is input to the first ANP4 of the output layer. That is, S.H.
One sum operation is performed in ANP 4 in the two peak pulses of 2H2O. As shown in the figure, there are three intermediate layer neurons of ANPI and 2°3 in the intermediate layer, but one dummy node is added by the max value node circuit 187, so it is assumed that there are four neurons in total. . Therefore, the two pulses of SH2H2O2 are ■
The analog signal in the middle layer is output as ANP4 by the four sets of peak pulses of this S H2 signal.
The sum of products is calculated.

Naturally, this operation is performed at the same time as the intermediate layer ANP is performing the product-sum operation on the input signal, and this is pipeline processing. C3O2
The signal below is the ANPI C8O signal in the middle layer,
This is the C8I for ANP2 in the same middle layer. This is the part indicated by ■. Below that is C8O of ANP2, and below that is C3I of ANP3, which is ■.

Below it is C8○ of ANP3, and below it is C3I of the dummy node, which is the signal output from DC32, that is, the master control block. C3
Looking at I, the middle class A is in the order of ■, ■, ■, ■.
NPI, ANP2. ANP3 and the max value node circuit 187 of the dummy node are manually operated. During this time, the S H2 signal outputs four pulse signals with two peaks.

That is, the output layer neuron of ANP4 adds four products of the human input analog signal and the weight. In the part (2), when the C3I is manually inputting to the ANPI, an analog signal is output from the ANPI to the analog bus between the intermediate layer and the output layer, and this signal is input to the ANP4. And at this time, the corresponding weight data is input to ANP4, and the product is performed with it, summed at the peak of SH2H2O21, and sampled/held at the second peak. When this calculation is completed, the signal C8O rises from ANPI, and this becomes C3I of ANP2. This is the state (2), and at this time the weight data and the data on the analog bus are multiplied and the sum is calculated. C8I to ANP3 goes high after a predetermined period of time after {circle around (2)} falls, and a sum-of-products operation is performed in ANP4 as shown by {circle around (2)}. Such a product-sum operation is calculated in ANPd, and the fixed voltage output from the max value node circuit 187 at point (3) is calculated in ANPd.
4, and this is added to the sum of products that has been stored inside.

The above operations are performed in parallel on the ANP5 of the output layer. There is simultaneous processing here. The result of the product-sum operation calculated by ANP 4 is connected to the analog bus B3 to the output layer.
The timing at which this signal is output is the rising edge of C3O3 output from the mask control block 181. The control signal for output by the MASOX BALINE node circuit 187 to the analog bus B2 is DC32, which corresponds to (2). The operation up to DC32 is the operation up to outputting the calculation result in the intermediate layer. The signals below this in the timing chart operate in a similar manner, and are signal pulses that define the operation of the output layer connected in cascade to the intermediate layer. When C3O3 rises, the result of the sum-of-products operation, which is incremented by 1 at ANP4, is output.

In the output layer, two ANP4 and ANP5 are output. Note that, for example, the rise of C3O2 in ■ is a signal that enters ANP 1, and this rise is delayed from DCLK. This is because when performing a product operation between an analog input signal and digital weight data, it is serial when reading digital data using WCLK, and the digital data reading time to internally convert it to parallel and the analog input signal is D/A. The rise of C3O2 is delayed by taking into account the time it takes to reach the converter, that is, the multiplication processing unit (
This is because it is. In other words, what is missing at the beginning includes the time required to call the data, that is, read the serial data. The data setting ends after a while after the rise of DCLK, that is, after 16 cycles of WCLK. The start time of analog multiplication is 8 on WCLK after C3O2 rises.
After the cycle.

FIG. 10 is a timing chart showing the timing of reading digital weight data. In the figure, a master clock MCLK, a synchronization signal 5YNC, a weight clock WCLK, a data clock DCLK, and actual weight data WDATA are shown. Weight data WDATA
is read bit serially from the weight memory, and 16 bits are entered serially. S is the sign bit, B
14 to BO are numerical bits. In the figure, B8. of weight data WDATA. B7. The portion B6 is shown enlarged in the lower part of the diagram in correspondence with the weighted clock WCLK. The weight clock WCLK has a period of 250
The duty ratio is 50% in n5ec. W.C.L.
The address is given to the weight memory after the propagation delay time of the address update counter inside the sequencer from the fall of K. i.e. bit n of weight memory (RAM)
The address is the address of the weight memory where bit 7 of the weight data WDATA is stored. After this address is determined, bit 7 is read out after time tAA. The change from bit 7 to bit 6 is determined by the change to the next cycle of the weight clock, with bit 6 being read out in the next cycle. The 16 bits of weight data are input to ANP, and the product with the analog voltage input to ANP is calculated by the internal D/A converter, so the analog voltage input starts long after the rise from the data clock DCLK. be done. That is, since there is a time from when the analog input voltage is input until it reaches the D/A converter, that time and the time when the digital weight data is sent internally are controlled, and the arrival time of the weight data and the analog input voltage are controlled. You need to input the analog voltage so that the arrival times exactly match.

For example, the analog input terminal rises from around B7 of the weight data, the weight data BO is input, and the time is set so that the multiplication with the analog value starts when all the weight data is internally determined. It is necessary to take control of the

The addition is performed during the next period when DCLK becomes low.

ANP operation time is 5YNC signal and WCLK.

and data DCLK. Since the analog input voltage has a considerable time error in the time it takes for the voltage to arrive from the input terminal of the ANP to the D/A converter that performs the product with the digital weight data, the rise of C3I is delayed from the rise of DCLK to allow for a margin. It will start from there.

FIG. 11A is a block diagram of the mask control block 181. The mask control block 181 is a part that integrates all control signals.

The main components are an external bus interface circuit 200,
They are a control pattern memory 201, a microprogram sequencer 202, a microcode memory 203, and an address generation section 204. External bus interface circuit 20
0 is an interface for connecting to an MPU, etc., and is connected to an address line 205, a data line 206, and a control signal line 207. Upper address comparison circuit 208 of external bus ink face circuit 200, DFF2 which is a register
09 respectively decodes the upper address given from the MPU etc., and when the upper address is a predetermined address, the lower address and data are respectively sent to the D-FF20.
The latch signal from the timing circuit 214 is set at 9, 211 as a trigger. The address and data are input internally via an internal address bus and an internal data bus via bus drivers 210 and 212, respectively.

The address is used when referring to the microcode memory 203 and writing the microcode from the MPU side via the data bus. Furthermore, the microcode address of the lower address is also passed to the microprogram sequencer 202 via the bus driver 210, so that the control pattern memory 201 can be referenced with a specific address from the MPU side.

Data from the MPU or main memory is launched to the D-FF 211 via the data line 206, and then transferred to a separate I in the microcode memory via the bus driver 212.
10 RAM 213 or control pattern memory 201
Separate 1-I11-11ORA, 216 within. When a data strobe signal from the MPU or memory is applied to the timing circuit 214 via the control signal line 207, an acknowledge signal is sent back, thereby controlling the transmission and reception of addresses and data. The timing circuit 214 is a D-FF 211. It controls the launch timing to the DFF 209 and the writing timing to the microcode memory 203 and the control pattern memory 201 via the WR multiplication signal.

The "1" and "0" patterns of complex control signals given to the neurochip as shown in the timing chart of FIG. It is generated by reading from the control pattern memory 201 under the control of the control pattern memory 202 . For example, reset signal Re5et% data clock DCLK, weight clock WCLK.

C3OI, tso2. C3O3, 5YNCI, 5YNC
2. Control signals such as SHI, SH2, OCl, and OC2 are separated) The control information that accompanies the pattern, that is, the sequence control flag, is read from the I10RAM215 and is
The data is read from the separate I10RAM 216 of . For example, the control pattern memory 201 is 10001.1000.
If the pattern 1 is stored, 1,0"
Since it is a bit pattern, if the address of the control pattern memory 201 is controlled to repeat this "10" bit pattern, the repetition of this pattern will be read out from the control pattern memory 201. In other words, the control signal Since the patterns are very complex, these patterns must be prepared in advance with this Separate I10.
Store it in RAM215 and its separate I10R
The structure is such that by specifying the address of the AM 215 under the control of the microprogram sequencer 202, the bit pattern is sequentially output.

Therefore, since some of the same patterns are repeated, how the repetition is realized depends on address control. This pattern for one period will be called an original pattern. In order to repeat the original pattern, it is necessary to feed back specific information from the control pattern memory 201 to the microprogram sequencer 202. That is, the second separate I10RA
By inputting the sequencer control flag in M216 to the microprogram sequencer 202 as a conditional input, the microprogram sequencer 202 is controlled to return to the top address containing the original pattern in the first separate I10RAM 215.

This causes repetition of the original pattern. That is, the microprogram sequencer 20
2 is the general-purpose boat output line 202 until the condition is met.
-1 to sequentially generate an address signal to the separate I10RAM 215. Normally, this address is incremented, but when the condition that the original pattern reaches the end is met, it returns to the first address where the original pattern is stored. As a result, a specific pattern is repeatedly separated into I/ORAM.
A control pattern is output from 215.

FIG. 11B Lt shows the interrelationship of information in memories 201 and 203 that control mask control block 181. In the figure, control pattern memory 1 corresponds to a first separate I10RAM 215, and control pattern memory 2 corresponds to a second separate I10RAM 216. In the microcode memory 203, the sequencer 2
02 control codes are stored, mainly Jump commands and R
Instructions are stored in epea. Looking at the increasing direction of addresses, there is a Repeat command at a specific address, and the number of repetitions of pattern 1 in the control pattern memory according to this repeat command is stored in the corresponding address of the control pattern memory 2, for example "10". Then, 10
This will result in several iterations. In this way, when the address increases and comes to the Jump instruction in the microcode memory, the second J in the microcode memory 203
Jump to 500H with ump and output Pattern2. When Pattern 2 is repeated 5 times, r, 100 is again set at the third Jump in the microcode memory 203.
It will jump to HJ and output Pattern 1. In this way, the original pattern is read out from the control pattern memory 1 repeatedly.

WCLK is created in synchronization with the read clock of the address that refers to this control pattern memory 201, and information is read out from the weight memories 185 and 186 in synchronization with WCLK. Addresses to weight memories 185 and 186 are accessed by address signals output from address 1 and address 2 of address generator 1 204. Address 1 and address 2 are separated corresponding to the intermediate layer and output layer, respectively. The weight data to be given to the ANP in the intermediate layer is read from the weight memory 185 specified by address 1, and the weight data to be given to the ANP in the output layer is the content read from the weight memory 186 specified by address 2. It is. Each address has weight memory 1
Since the contents of 85 and 186 are stored one bit at a time in the direction of increasing address for each bit of weight data, it is necessary for the microprogram sequencer 202 to provide a count control signal to the address counters 217 and 218. The address counter 217218 increments this address and provides it as an address signal to the weight memory 185, 186 one after another via the bus driver 219, 220. A plurality of weight data are then read out from the weight memories 185 and 186.

The WCLK from the first separate I10RAM 215 and the counter control signal from the microprogram sequence 202 are connected to the AND circuit 221.2 in the address generation unit 204.
It has been added to 22. When the counter control signal is high, the address counter is updated by WCLK and the WCLK
The 1st to 16th bits of LK are handled by the address counter 217,
Increment 218. And the rest of WCLK
For bits 17 to 26, the counter control signal is set low to inhibit WCLK and stop incrementing the address counters 217 and 218. Then, in synchronization with 5YNCI and 5YNC2, counter reset signals are sent from the microprogram sequence 202 to the AND circuits 221 and 222, respectively, to reset the address counters 217 and 218. This returns the addresses of the weight memories 185 and 186 to the starting address. In addition, mask control block 1
The mode signal output from 81 is for one normal use of the weight memory, that is, a mode in which the weight memory is disconnected from the MPU data bus and weight data is given to the ANP, and a mode in which the weight memory is connected to the MPU data bus and the weight memory is referenced from the MPU. This is to form a mode that

The mode signal is formed by setting the lower bit of the data from the MPU to the flip-flop 224 using the AND signal generated by the AND circuit 223 as a trigger from WR from 1 bit of the lower address and the write signal from the timing circuit 214. be done. When this mode signal is O, the weight memory is normally used.

Write signal WR and one pin of the internal address bus are input to the clock terminal of the flip-flop 224 via the AND circuit 223, and the LSB of the internal data bus is input to the data terminal of the flip-flop 224. The upper address is compared with the master control by the comparison circuit 208, and the reference controller 18
Determine whether 1 is selected, and if it is selected,
The lower address and data are taken into the DFFs 209 and 211.

Such interface operations are performed similarly for other devices connected to the MPU, but since the weight memory normally supplies weight data to the ANP, the MP
Connecting directly to U's data bus will result in bus contention. To prevent this, when the LSB of the internal data bus is taken into the flip-flop 224, the mode is set to 1, and the weight memory is not chip-selected as described later, so that no data is generated from the weight memory on the data bus. Do it like this. The internal address bus specifies an address in either the microcode memory 203 or the control pattern memory 201 at a predetermined timing, and writes desired data from the internal data bus to the accessed address. This allows you to change the program stored in the microprogram sequencer 202, microcode memory 203, or Separate 1-111-110RA, or
5. Change the control pattern stored in 5.

FIG. 12A is a data storage configuration diagram of this weight data memory 230. In the figure, 8 bits in the column direction are information of 8-bit data entered at the same address, and each bit is applied to ANPI, ANP2, . . ., ANP8 from the bottom. The addresses are different in the row direction, and as shown in the figure, the row (row) address increases in the left direction.The weight data is 16 bits including the sign bit, so it is sorted from the smaller address to the larger address. The MSB is a sign bit and the other 15 bits are numerical bits. Microprogram sequencer 20
When the address is incremented from 2 in synchronization with WCLK, one word of weight data, that is, 16 bits, is read out in order from MSB to LSB.

These weight data are passed to eight ANPs simultaneously. Since the structure is such that data is stored in the direction in which the addresses increase, an address counter for this weight data is required.

That is, when one word of weight data from MSB to LSH is counted, control is required so that it is counted as one piece of weight data. This control is also performed by the microprogram sequencer 202.

FIG. 12B shows a specific circuit of the weight memory blocks 185 and 186. Memory 230 is a RAM called MB8464A-70. The output is 8 bits corresponding to ANP1 to ANP8. Basically, either the bus signal line seen from the MPU bus or addresses 1 and 2 seen from the mask control block 181 are used. Addresses 1 and 2 are addresses 1 and 2 in FIG. 11A described above. These addresses 1 and 2 are input in a form that is incremented in synchronization with WCLK. 8-bit data is read simultaneously,
Each bit is given to ANPI to ANP8 simultaneously.

When the mode signal is O, the weight memory 230 is chip-selected via the AND gate 233, and at this time, the address 1.2 from the microprogram sequencer 202 is
is available at multiplexer 234. Weight data is then sent from the weight memory 230 to ANPs 1-8. On the other hand, since the output of the inversion circuit 231 is high, the tri-state path transceiver 232 is disabled and the output of the weight memory 230 is not output to the MPU.

When outputting to MPU, set the mode signal to 1 and
Chip selects the memory 230 via the address decoder 235 according to appropriate address information from the PU,
An address is given to the memory 230 from the MPU. When the mode signal is 1, control of reading from the MPU bus or writing from the bus to the memory 230, that is, the direction of the read/write is determined by the read signal Read Signal on the pig line coming from the MPU via the AND gate 236.

Next, the learning algorithm will be explained.

FIG. 12C is a flowchart of a learning algorithm called backpropagation used in the present invention. Learning proceeds as follows. Complete information to be learned is input to the neural network of the present invention, that is, a hierarchical network constituted by a set of ANPs, from the MPU via an input control circuit (not shown). Then, the input signal is passed through the input end circuit, the intermediate layer, and the output layer to the network output via the A/D converter, and then to the MP.
given to U. A learning algorithm exists in the main memory on the MPU side. The MPU takes in the teacher signal from the main memory and checks the error between the network output and the teacher signal. If the error is large, the MPU will change the weight data, which is the strength of the network connection, so that the network outputs the correct output. This weight data is added to the ANP of each layer via the weight memory 230.

If the weight data is updated by a learning algorithm, it will follow the pack propagation learning algorithm of Figure 12C. When the learning algorithm starts, the MPU causes the L-th neuron ANPL in the output layer to calculate the error between the teacher signal YL and the current output YL and substitute it into ZL. Output YL is neuron A N P
Since it is the output of L, for example, if a sigmoid function is used as a nonlinear element, it is the output value of this nonlinear function. Therefore, neuron A N P
In L, it is necessary to propagate the error ZL to the input side of the nonlinear function. When performing error propagation, the energy function, i.e. the energy of the square of the error signal multiplied by 1/2, i.e. EL = 1/2 (yt, -YL) 2, is the deviation of the nonlinear function human power X, i.e. 29X.

can be transformed as follows.

=(Yt, Yt, ) · r(xL). Here, if the nonlinear function f (xL) is a sigmoid function, then (1). Therefore, the deviation of δ, that is, the energy with respect to the nonlinear function input XL, becomes VL×ZL, that is, Ul− shown in S2. It is necessary to further back-propagate the error amount δ with respect to the input of this energy nonlinear function to the intermediate layer.

Let A be the th neuron in the intermediate layer.

The output of Al1 is assumed to be Yk. Output layer neuron ANP
The nonlinear function input XL of L is expressed as the sum of products obtained by multiplying each of the outputs (Y, . . . Yk, 4A, l) of neurons in all hidden layers by a weight WLK.

Therefore, the deviation of Xt with respect to the weight WLK is as follows. On the other hand, the variation of the weight WLK with respect to the energy EL is given by the following equation.

When we transform the differential molecule ′(XL) of this sigmoid function, we get f′(XL) −Yt, (1−YL
). This is shown at 82 in the flowchart. That is, Ttx in S3 expresses EL θWLk and is a partial division of energy with respect to weight. Therefore, this TLK may be set as the change in weight ΔW, but in order to accelerate convergence, the second term of the first equation shown at 84 is added to form the following recurrence equation to modify the weight.

ΔWLk−αTLk+β·ΔW1WLll=W, , +ΔW, where α and β are constants. We are now focusing on a specific neuron ANPL in the output layer, but if we assume that this ANPL is connected to all neurons in the middle layer, it is necessary to repeat K from 1 to K mnx for each ANPL. be. This is the repetition shown in R1 of the flowchart, and is repeated as many times as there are neurons in the intermediate layer. When this repetition is completed, the back propagation for the specific neuron ANPL in the output layer is completed.

Therefore, this can be applied to all output layer neurons (ANPI
, ANP2, ・--, ANPL mai+)
Therefore, as shown in R2 of the flowchart, L is repeated from 1 to LllaX. That is, the number of neurons in the final output layer L ma
It will be repeated x times.

Next, learning will begin from the middle layer to the human-powered layer. Although the algorithm is almost the same, the error signal cannot be expressed by the difference between the teacher signal and the output voltage, and is expressed by the formula S5. That is, Z, is the second neuron in the hidden layer, A,
This term corresponds to the output error signal of This is clear from the following equation.

-ΣWL k -UL. Therefore, the error signal bone Z of the intermediate layer is calculated by repeating the process from 1 to L ma times for the index L of Zk in S5, that is, as many times as the number of outputs (R3). After that, the algorithm between the intermediate layer and the output layer is the same. That is, first, the differential value V of the sigmoid function is obtained, and using it, Uk, that is, the change in energy with respect to the input of the nonlinear function is obtained at 86. That U in S7.

The output of the human power layer and the product Tk1 with YJ are obtained using . Taking this as the main part of the weight change, a second term is added to speed up the convergence as shown in S8, and ΔW5. is calculated, and ΔWk J is added to the previous value WkJ to obtain a new Wkj. This is a weight update. Input number J to update this weight
Repeat B□ (R4). That is, from j=1 to Jff
By repeating up to lax, the weights between the input layer and the hidden layer will be updated. Note that Z in S5 corresponds to the error signal of the output of the intermediate layer, and this is expressed as the partial division of the energy of the output layer with respect to the function input value, L, which is backwardly propagated, and WLK is This is determined only after the weights of the intermediate layer and the output layer are determined. In other words, the calculation for updating the weights is performed by the neuron ANP in the output layer.
Starting from L and moving to the middle layer neuron ANPK, the weight change ΔW of the middle layer neuron ANPK cannot be calculated unless the preceding stage ΔW is determined.

Therefore, this learning is called backpropagation because 81 arithmetic is possible only when going back to the last input layer.

Learning by backpropagation inputs the training data as complete information, and changes the strength of all connections backwards to reduce the forward motion that outputs the result and the resulting error. Therefore, this forward movement is also necessary. In this forward motion, the analog knee neural net portion of the present invention is effectively utilized. Also, the algorithm for backpropagating the output values is executed on the MPU.

Note that in the case of nonlinearity that is not a sigmoid function, the differential value of the nonlinearity is different. For example, if tanh(X), the learning algorithm is as shown in FIG. 12D, and the nonlinear differentiation result is VL-1-IYLI in the output layer (S2'), and ■ in the intermediate layer.

-1-IYk l (36').

Others are given the same reference numerals as in FIG. 12C, and their explanation will be omitted.

FIG. 13 is a configuration diagram of the daisy circuits 173 and 174 on the input side. In the figure, 240, 241, and 242 are D-type flip-flops. D at the rising edge of the DCLK signal
The data input to the terminal is set, and the output Q is set to 1. The first flip-flop 240 sets the C3I signal at the falling edge of DCLK. Then, at the next rising edge, the output signal is set in the second flip-flop 241.

Its output is input to the D terminal of the third flip-flop 242. The clock signal that sets its input is the output of 4-bit counter 243. counter 24
3 is triggered on the falling edge of WCLK. It is cleared at the falling edge of DCLK. Therefore, DCLK
At the falling edge of , the counter 243 becomes all O, and WC
After the falling edge of LK is input 8 times, the Q of the upper bit
Since the D times signal becomes high, this serves as a trigger and the flip-flop 242 outputs a high signal to C3O. When the output of the flip-flop 241 becomes 0, C3O is cleared. With this operation, a daisy operation is performed in which C3I falls and C8O is output after a predetermined time period corresponding to eight WCLK pulses has passed.

FIG. 14 is a specific circuit diagram of the max value node circuit 187 forming a dummy node neuron. In the same figure, a resistor 250 and a Zener diode 251.2
52, resistor 253, and voltage followers 254 and 255 are circuits that form a constant voltage. When current flows from +12 volts to -12 volts through resistor 250.253 and Zener diode 251.252, voltage follower 254.
For 255 human power, +7 volts and 17 volts are formed respectively. These voltages are the voltage follower 254.25
It is outputted via the output resistor 256 of 5. The analog switch 25 is configured to draw out these two constant voltages in a time-division manner.
7 to 264. When the T-mode signal is 0, the constant voltage is applied to the next voltage follower 265 via the analog switch 257. When the T mode is 1, that is, in the test mode, the output is suppressed to analog ground by the analog switch 258, so the output is 0.
Port is input to voltage follower 265. In test mode, the offset on the bus will be notified to the MPU. Voltage follower 265 is enabled by switch control of the output. When the output enable is 1, the analog switch 260 is turned on and functions as a voltage follower, and its output is provided, but at this time, it is not output to the dummy node output. Conversely, when the output enable is O, it is output to the dummy node output. analog switch 26
Switch control of 0 and its output is controlled by output enable l or 2, which is 0 enable. That is, when output enable 1 or 2 is O, a constant voltage is output to the dummy node output. Note that the upper dummy node output is for the input layer dummy node, and the second dummy node output is for the intermediate layer dummy node. Since the output voltage of this dummy node is fixed to an appropriate value, it can be used as a threshold voltage. In addition, Zener diode 2
Reference numerals 51 and 252 output a constant voltage in a reverse bias state, and the fixed voltage can be varied in the range from +7 volts to 17 volts. Output enable 1.
2 has its enable state determined by the dummy node control signal DC3 from the mask control block 181 in order to avoid collision on the analog bus with output voltages from other ANPs connected to the analog bus.

Figure 15 shows a nonlinear number generation circuit, Figure 16,
7 and 18 show the hardware on the digital logic side inside the ANP.

FIG. 15 shows a transistor network realizing a sigmoid function. The sigmoid function here refers to a continuous, monotonically non-decreasing function, and does not specifically exclude linear functions. In the same figure, 343.356,378,390
, 298, and 314 and their paired transistors form a differential amplifier, and each transistor group connected to the collector side is a current mirror circuit. The collector current flowing through the collector of the left transistor of the differential ANP is the output current. A current mirror changes the direction of the current and outputs it. Current enters a resistor 336 connected to the output vO. A resistor 336 converts it into a voltage or current.

Since it has no drive capability, the output is received by a high impedance operational amplifier buffer. transistor 337,
The circuit at the input end of 339 is a bias circuit. A piecewise linear method is used to realize the sigmoid function. The slope of each section of the sigmoid function is determined by the ratio of the emitter resistor 344 connected to the emitter and the output resistor 336. At this time, emitter resistors such as the transistor 343 are also included. Gain of each differential ANP: Different. Breakpoints for each piecewise linear transition utilize saturation characteristics. Their saturation characteristics are all different. (2) At the 0 output point, the saturation characteristics of each ANP are changed so that the sum of the currents output from each operational amplifier becomes a sigmoid function. Transistor 345 and resistor R1 are current sources. Transistor 346 and resistor R2, transistor 3
53, resistor R3, etc. are all current sources that supply the same current. That is, the resistances are determined so that the current values are the same. All have the same current source. transistor 3
Since the collectors of 45 and 346 are connected, the sum current flows at the intersection of resistors 344 and 347. The collector currents of transistors 343 and 348 become the same at the time of balance.

The transistor 351 is for improving the characteristics of the current mirror. Transistor 350 is diode connected. Changing the direction of current means that there are two ways to draw in current and to send out current. As shown in the figure, a current flows from the collector of the current mirror transistor 351 toward the output.

There are many lower transistor arrangements, but transistors whose emitters and collectors are connected to the same point are identical transistors. For example, transistors 358 and 360 are the same transistor, which is the same as transistor 345. Also, 359 and 361 are the same transistor, and correspond to 346. The transistors of 368 and 369 are the same, which corresponds to 353. The same applies hereafter. Therefore, the circuit includes a total of six operational amplifiers each having a constant current power supply driven by the same current and operating in such a way that the current direction changes depending on the positive or negative output voltage. Further, transistors 337 and 338 are level shifters, and 330 and 327 are also level shifters. The level shift circuit is used to ensure that the operating ranges of the positive and negative sigmoid functions are approximately the same. The transistor 352 is for correction so that the collector current of the transistor 351 and the collector current of the transistor 353 become equal. Transistors 367, 385, 287.
The same applies to 307.

FIG. 16 shows a sequence generator 28 (FIG. 2) for forming pulse signals to be supplied to the neurochip.
This is a concrete circuit. 401 and 402 and 404 and 405
is an inverter, and each inverter is a clock inverter. Separate clocks are created for the rise and fall of the launch signals of flip-flops F and F. The flip-flop shown in the figure latches with a rising clock, and the inverter and F form the rising latch F and F. For example, in the case of DCLK, the signal that passes through one inverter 401 becomes the falling launch clock signal. The clock that passes through the inverter 402 becomes the rising latch clock DCLK. Similarly, the output of inverter 404 is WCLK for falling, and the output of inverter 405 is clock WCL for rising.
It is K. In F, F410, 5YNC signal is DC
The falling edge of LK is latched. F, F410
and 415, the 5YNC signal is delayed by one DCLK cycle to create the 5NC2 signal, and the 5YNC and the signal delayed by one clock create a 1τ pulse. The integrating capacitor in ANP is discharged with a pulse of 1τ (one cycle of DCLK) after 5YNC rises. That is, the signal CR3T is the reset signal for that capacitor.

The other DSH2 is DCL from the falling edge of 5YNC.
This is a pulse with a length of 1τ of K, which is AN
This becomes the sample/hold signal for the sample/hold capacitor in P. In F and F of 411, the clock is WCLK and the data is DCLK, so WCLK
latches the DCLK signal. After that, the 5YNC signal is high at the Nant gate 414, and the DCL
When K is high, the first WCLK that comes is triggered and becomes the clock for F and F443. Nantes Gate 41
4 and inpark 440 result in an AND.

In F and F443, the first signal WCLK that comes while the 5YNC signal is high takes in digital weight data, that is, the sign bit of WD.

This signal is the MSB, or sign bit, of the serially incoming weighted digital data. That is, F, F
F and F443 launch the code bit at the timing of 411 and Antogame-1-(414, 440). A 4-bit binary counter 416 counts the number of WCLK pulses. Since 16-bit digital weight data is input, it is counted 16 times. When the count is finished, the output becomes high and enters the inverter 423. This signal is a signal indicating that 16 counts have been completed. This signal is used to control the input of weight data serially input to the ANP into the shift register 27 (FIG. 2). The least significant bit of counter 416 is also input to inverter 422 . The output of this inverter 422 generates the C8O signal. C8O is a daisy chain control signal, and in order to prevent conflicts between the signals output from the two previous ANPs on the analog bus B1, the daisy chain is set so that the next C8 is output after the previous C8 falls. It is necessary to form a delay circuit to perform the operation. The delay time of this delay is formed by counting WCLK and using the counter value. When the counter 416 finishes counting, a signal indicating that the count is finished is latched to the flipflop 433 via the inverter 423, which is beat by WCLK. That is, the 17th (WCL)
I'm hitting K. The latched signal is returned to the counter 416 through inverters 437 and 438, and disable control is performed so that the counter 416 is no longer incremented. Inverter 43
When the output of 8 becomes low, the counter 416 stops counting. The outputs of pages F and F433 enter a flip-flop 442. This becomes the gate signal of the output of shift register 408. That is, the input digital weight data of 16111 is transferred to the shift register 408.
1 of the numerical bits, excluding the sign bit.
Once the 5-bit data is arranged in parallel, it is output. No output is output while shifting,
The gate signal for outputting when all the signals are input is WR. The contents of shift register 408 are provided to the ANP multiplier. The signals output from F and F433 are branched and used as shift register enable signals. F
, F442 latches the outputs of F and F433 at the rising edge. The shift is completed with 16 falling latches of WCLK, and then rising latches may be used to open the gate, but this is done at falling. F, F4
Reference numeral 12 generates a pulse signal for selecting a sigmoid function. When a reset signal is input using F and F412, whether or not to use the sigmoid is selected depending on whether the WD, that is, the weight digital input signal is O or 1.

This method may not be used in this system. In reality, the sigmoid selection signal is directly generated externally. The circuit below is a daisy-chain circuit. The output of the counter 416 is used to create a delay using F and F434, and this delay is used to trigger the final F and F445. As a result, in addition to being shifted by 1τ of DCLK, the head is lowered instead of being shifted as it is. That is, C3
The I signal itself may not last for one cycle of DCLK, so in order to change C8I to C8O, for C3I,
The first part, for example, is removed by 2 microns to delay the front of the waveform, and the rear part remains as it is to create the signal. Gates 425 and 427 are C3I buffer gates. positive buffer and inverter buffer.

FIG. 17 shows the phase control circuit 29 (FIG. 2) that forms the sample/hold S/H signal and the OC signal. S/H
The signal is separated into one that enters the inverter 515 and one that enters the gate 524. The same applies to the OC signal. S/H
The signal is divided into a gate 524 and an inverter 515, and after passing through the inverter 515 and entering the gate 525, there are eight stages of inverters. Two types of signals are generated with respect to the S/H signal: one with the same phase and one with the opposite phase. This prevents two outputs from becoming 1 at the same time by connecting inverters to several stages of bone cascades and cross-crossing them. That is, two sample/hold S/H signals, S/HO
and S/H1, and try to prevent both of them from becoming J. That is, the inverter chain is a delay circuit that prevents both S/H signals from being turned on at the same time. The delay time is determined by the length of the inverter chain; one side is turned on, then delayed several steps, and then the other side is turned on.

The same applies to S/HDO and S/HDI.

The circuit for the 00 signal is basically the same, but since the CR3T signal is input to gates 528 and 529, when CR3T is 1, both outputs are forced to 1. Both OCO and OCl are prevented from becoming 1 at the same time, but in the case of ○C, they are made to become 1 at the same time only when CR3T is 1. This realizes a reset function of discharging the charge in the integrator capacitor through the tti control of the analog switch.

Figure 18 shows a 15-bit shift register 27 (Figure 2)
It is. Gates 602, 603 and 6014, and F
, F627 corresponds to 1 bit, and will be explained using this. The output of the previous time is input to the gate 603, which is F.

This is the output of F628. Since it is an input from the previous bit, it becomes a data signal for shifting. The other signal entering gate 603 is a 5HFT, an inverter of the shift signal. This is a shift control signal, and when it is valid, it instructs a shift. Also gate 6
02 contains the output of F and F627 itself.

This means that you are feeding back your own output.

The other input of gate 602 similarly receives an inversion of the 5HFT signal, but its phase is different from that of gate 603. This will keep the current output as is when shift is disabled. Since the clock signal always comes in regardless of the shift, even if the clock comes in, if the shift is not valid, the shift will not be performed.

Shift signal 5) Shifts the previous bit only when IFT is valid and inputs it through gate 603 to perform a shift operation. It is included in the AND of WR multiplication signal gates 632, 633, etc. This serves as a selection signal for outputting or not outputting each bit, and serves as a control signal for passing the data stored in the shift register to the multiplier. Also, in order to take out the fan-out, for example, the inverter 620
Therefore, the receiver has 5 F, F reset signals out of 15, and the gate 626 has 10 F, F reset signals. The fan-out shift register 608 has a shift enable 5HFT and an output enable WR function.

Next, a case will be described in which the neurocomputer according to the present invention is configured with a feedback network.

FIG. 19A is a conceptual diagram of a feedback network.

In the case of a feedback network, there is basically an input, but the structure also has a feedback path in which the signal it outputs also returns. This feedback method may be used as a type in which one layer in a hierarchical neural network is time-division multiplexed, or as a so-called Hopfield type neural network.

In the former case, since the input/output signals of the ANP are time-divided, the output data of that same ANP is sequentially output at every certain sequence cycle at the output point of each ANP.
It operates sequentially as an input layer, a middle layer, and an output layer of a hierarchical neural network for each sequence cycle.

In the latter case, the output voltage is fed back until the output of the ANP reaches a specific value, that is, until it becomes stable. When the returned result is produced, the state is repeated until the result matches the previous data, that is, the data produced before, and convergence occurs when a stable solution is reached.

According to the embodiment of the present invention, as shown in FIG. 19B, the feedback path is realized by the common analog bus CB, so that a U-shaped feedback section exists. Then, the calculated result of the first ANP is output, and the output from each ANP is fed back through the feedback path. This return operation is repeated.

FIG. 20 shows an embodiment in which the neurocomputer of the present invention is realized by a feedback network that operates as a hierarchical network. A multiply-accumulate operation is performed in ANP1, 2.3 on the time-sharing analog human input signal from the analog input port 1.2, and ANPI, 2.3 operates as an intermediate layer, and the time is transferred from ANP1, 2.3 to the analog bus B2. The output signal is divided and outputted, and this output signal is returned to the analog bus B1 via the analog common bus CB, which is a feedback path, and the sum-of-products operation is performed on this feedback signal again by ANPI, 2.3. By operating 3 as an output layer, a hierarchical network is realized by the - layer ANPI, 23. The max value node circuit 187 receives the DO3 output of the mask control block and generates a dummy signal on the analog bus B2. And from the mask control block D CL K and WC
LK are respectively input to the daisy circuit 171, and C8I
Specifies the rising and falling timing of the signal.

FIG. 21A is a timing chart of the feedback hierarchical network shown in FIG. 20.

WCLK is generated only while DCLK is rising, and after DCLK has risen, the analog signal has stabilized, and weight data has entered serially, but before it is aligned in parallel, C3O1 from the mask control block 181 is generated. is input to the daisy circuit 171 and rises as shown in ■. At this time, the analog signal held in the sample/hold S/H from the analog input port 1 appears on the analog bus B1 via the analog switch 175, and a sum-of-products operation is performed at ANPI, 2.3. When the next DCLK is input, C3I to the daisy circuit 172 rises as shown in ■, and the signal of the sample/hold circuit S/H holding the input signal from the analog input port is transferred to the analog switch via the analog switch. It appears on bus B1, and the second sum-of-products operation is performed at ANPI, 2.3. Furthermore, after DCLK is input at the next timing, the dummy signal DO3 is generated from the mask control block as shown in (2), and the third summation of the fixed voltage is performed in ANP12.3. Next 5
While the YNC signal is rising, ANPl, 2.3
A sum-of-products operation is performed for the output layer. WCLK that counts the address counter only while the address count prohibition signal of the address 1 signal to the weight memory is rising.
is enabled, and its counting is suppressed at other times. Next, when C3O2 is given to ANPI from the mask control block, ANPI outputs the previous product-sum result to analog bus B2, returns to analog bus B1 through analog common bus CB, and returns as shown by ■. A sum-of-products operation is performed in ANPi, 2.3.

After C3O2 is subjected to a predetermined delay in the internal daisy-chain circuit of ANPI, the input signal C3I is applied to ANP2 as shown in ■, and at this time, the output signal of ANP is transferred to the analog bus B2 and the common bus C'B. and is applied again to ANPI via analog buses A1 and B1, where a sum-of-products operation is performed. Similarly, C8o from ANP2 becomes the C3I signal of ANP3 after being delayed for a predetermined time.
When the 8I signal rises as shown in ■, the output signal of ANP3 is transferred to the analog bus B2. The signal is fed back to the ANPI 2.3 via the common bus CB and the analog bus B1, where the product-sum operation is performed. Similarly, as shown in (2), when the signal DC3 from the dummy node rises, the sum calculation is again performed with respect to the fixed voltage by ANP1, 23. Then, at the next rising edge of the C802 signal, ANPI becomes 2.
Outputs are generated as shown in (1) and (2) through the S/H.

Note that no output is made from the analog input port 2.

Here, ■, ■, ■ are ANPI, 2.3 operates as the middle layer, ■, ■, ■ are ANPI, 2.3 is the output layer,
It works. Therefore, according to this embodiment, ANPl,2
．． A hierarchical network can be configured with only one layer of 3.

FIG. 22 shows an embodiment in which the analog neurocomputer according to the present invention is configured with a Hopfield type feedback network, and FIG. 23 is a timing chart thereof. The memory of the mask control block 181
The outputs of the address terminal and the mode terminal are added to the weight memory block 185, and the data output of this weight memory block 185, BIO, is ANP 1, Bll is ANP
2, B12 is connected to ANP3. The output signal from the C3O1 terminal of the mask control block 181 is
It is applied to the daisy circuit 171 and the switch 175, and at the rising edge of this signal, the output of the sample/hold circuit 173 from the analog input port 1 is placed on the analog bus B1. Then, after being delayed for a predetermined time in the daisy circuit 171, an output of C8○ is generated, which is transmitted to the daisy circuit 172.
The signal of the sample/hold circuit 174, which is added as C8I to the analog input port 2 and connected to the analog input port 2, is placed on the analog bus B1 via the switch 176.

Similarly, the sample/hold circuit 17 to which the output signal C8O of the daisy circuit 172' is connected to the analog human powered boat 3
4' output switch 176' is opened to put the signal on the analog bus B1. In ANPl, as shown in Fig. 23, one sum of products calculation is performed in one cycle of the DCLK signal,
Digital weight data that drives a weight clock when the DCLK signal is high and enters in synchronization with the weight clock;
Multiplication with the analog input signal is performed, and when the signal is low in the latter half of DCLK, the sample/hold signal SH becomes high, and the sum operation is performed in the capacitor of the integrator.

That is, during period (3) when C3O1, that is, C3I of the daisy circuit 1, is high, ANP1, 23 performs a product-sum operation on the analog signal on the bus BI. Furthermore, when the OC signal from the mask control block 181 becomes high, ANPl2.3 performs offset cancellation,
One product-sum operation cycle is completed by sampling/holding.

Next is the second Daisy episode! ? Since the human input signal C3I of &172 becomes high ■, ANPI 2.3 performs a sum-of-products operation on the input signal from the next analog input port. Then, after the product-sum calculation cycle ends, the daisy circuit 17
2', the C8I signal enters the sample/hold circuit [11
An output signal is generated from 74', and the third sum-of-products calculation cycle begins, as indicated by ■.

Next, the C802 signal ■ is generated from the mask control block 181, and the signal that was formed during the previous multiply-accumulate cycle is fed back from ANPI via the analog bus CB.
．． ANP3 performs product-sum calculations at the same time. Next, after a predetermined time delay, the C8O output signal of ANP1 is applied to ANP2 at point 2, where ANP2 outputs the signal stored in the previous product-sum cycle in a daisy chain manner. This signal is fed back via analog bus CB to ANPI, A
NP2. Drive the sum of products operation with APN3. Similarly, after a predetermined time delay, C8○ of ANP2 becomes AN with ■.
P3, where the output from ANP3 is fed back via analog bus CB, and ANPI, ANP2 . A
In PN3, a product-sum operation is performed with ■. In the feedback network, as shown in FIGS. 23A and 23B, the outputs of the three ANPs go through six product-sum calculation cycles and are sent to sample/hold circuits 177.1, respectively.
Analog output ports 0, ■, 2 via 78.178'
is output to. In addition, the sample/hold circuit 177
．． The output signals of 178 and 178' are selectively outputted by the analog multiplexer 179 and sent to the A/D converter 18.
0 to a digital control circuit including an MPU 182, a memory 182, and a communication interface 184. The MPU 182 checks whether the neuron output state at the current time and the neuron output state at the previous time have become the same. If they are the same, it is determined that they have converged. In this way, it is carried out via one common analog bus CB. If a stable solution is reached by repeating the feedback operation, this is used as the final output.

Figure 24 shows a feedback network and a hierarchical network.

This is an optimal embodiment of a combination of networks.

A daisy circuit is provided as an input layer, and an AN
PI, 2.3 is provided. ANP4.5 is provided in the output layer. The output of the intermediate layer ANP1, 2.3 is fed back to the analog bus B1 via the analog bus B2 and the common analog bus CD. In addition, analog bus B1,
A max value node circuit 187 that functions as a dummy node is connected to B2. The output of ANP 4.5, which constitutes the output layer, is sent to the sample/hold circuit 17.
7.1.78 to analog output ports 0 and 1, respectively. B3 is an output layer analog bus.

The operation of the neural network shown in FIG. 24 will be explained using FIG. 25.

First, DCLK, WCL, and K are connected from the master control block to the daisy circuit 171 and ANP1, 2, 3, 4.
．． 5 respectively. When C3Ol is input as C3I to the first daisy circuit 171 from the mask control block 181 as shown in (■), a signal from the analog input port O is generated on the analog bus B1 via the sample/hold circuit 173 and the switch 175. AN
Pi, 2. In 3, the product-sum operation is SHI and CS
This is done under the control of 1.

Next, after a predetermined period of time has passed after C3O1 falls, the second
When the C8I signal input to the daisy circuit 172 rises as shown in (■), the signal from the analog input port 1 or the sample/hold circuit 174 and the switch 176
ANPI, 2 of the intermediate layer by analog bus B2 via
．． 3, a product-sum operation is performed as shown in SHI. Similarly, when the C3I signal to the third daisy circuit rises after a predetermined time has elapsed after the C8O signal falls, the product-sum operation is performed in the intermediate layer ANPI, 2.3. Then, the output of the intermediate layer ANP 1°2.3 is outputted to the analog bus B2 when C3O2 rises as shown by ■ and is applied to the ANPI, and the output is fed back to the analog bus B1 via the common analog bus CB. Therefore, the middle layer ANPI, ANP2. In ANP3, the product-sum operation is performed again under the control of SHI and OCI, and since the output of ANPI is generated on the analog bus B2, ANP4. In ANP5 as well, the sum of products calculation is performed under the control of SH2 and OC2. That is, in this embodiment, the middle layers ANPI, ANP
2. A product-sum calculation is performed simultaneously at ANP3 and outputs FJANP4 and ANP5.

Next, after a predetermined period of time has passed after C3O2 falls, the C3I signal is input to ANP2 in the intermediate layer as shown in ■, and the output signal of ANP2 is fed back to analog bus B1 via ANP2 and common bus CB. Therefore, ANPI, 2.
In ANP4.3, the product-sum calculation is performed again.
5, the product-sum calculation is also performed at the same timing.

Further, when the C3I signal is input to the ANP3 as shown by ■, the ANP3 generates an output signal on the address bus B1, so that the sum-of-products operation is simultaneously executed by ANP1, 2.3 and ANP4.5 of the output layer.

Next, when the dummy signal DSCI is given to the max value node circuit 187 at ■, the analog bus B is
A constant voltage is output to the AN, and this voltage is fed back via the common bus CB and the analog bus B1.
A sum-of-products operation is performed in PI, 2.3. At the same time, a product-sum operation is also performed in the output layer ANP4.5.

5YNCI is high during the period in which the sum of products is calculated in the intermediate layer and the period in which the sum of products is calculated in the intermediate layer and the output layer,
5YNC2 is high while the sum-of-products operation is performed in the intermediate layer and the output layer. When C803 is output, ANP4 produces an output at point 2. After the C3O3 signal falls, ANP5 also produces an output at point 2 after a predetermined time.

Note that while the address 1 and enable signal are low, WCLK is inhibited.

According to the present invention, when considering a former layer consisting of n neurochips and a subsequent layer consisting of a plurality of m neurochips, conventionally the number of wires is nm, but in the embodiment of the present invention, the number of wires is nm. According to the method, it is possible to reduce the number of wires to a single analog bus, which greatly reduces the number of wires.In addition, when inputting analog signals to a layer consisting of n neurochips, it is possible to use analog Since inputs can be made simultaneously via the bus, n neurochips in one layer can perform parallel operations. Furthermore, since pipeline processing is performed for each layer, the calculation speed can be increased.

Furthermore, since the neurochip is constructed of analog circuits, the scale of the circuit can be small, and therefore the power consumption can be small, so that a neurocomputer can be constructed with a large number of neurochips. Furthermore, the number of neurochips can be easily increased by changing the control pattern stored in the control pattern memory in the mask control block.

FIG. 26A and FIG. 26B are conceptual diagrams of the mechanism that generates errors that AN and P actually have.

Figure 26A shows the case where the input voltage is a known specific value, and even if the output voltage of the neural network is the theoretical value shown by the dotted line, there is a variation in the integral gain in the ANP corresponding to neurons from G1 to 05. Therefore, an output value that deviates from the theoretical value as shown by the dotted line may be generated. FIG. 26B shows the case where the input voltage is 0 volts, and in this case as well, the output voltage is output as an offset voltage. An important question is how to measure the error voltage at such an output.

FIGS. 27 and 28 show error measurement circuits in hierarchical and feedback type neural networks that measure this pulse-like error voltage, respectively. The output of max value node circuit 187 is connected to the analog bus. Also, the data given from the MPU is set in the port register, and each bit of the control signal En, Tnm de, and layer mode is given to the max value node circuit 187, but at the same time, the enable signal given to the max value node is output from the operational amplifier. It is also used to control switches. For example, when the enable signal is 1, the intermediate layer voltage enters the MPU via the A/D converter 707. This makes it possible to measure errors. The T mode is a test mode, and by using this T mode, the voltage on the intermediate layer and output layer analog buses for the aforementioned O input can be measured by the MPU via the A/D converter. For example, in T mode-1, a hierarchical neural network should respond to an offset voltage with respect to 0 human power. A/D at 0 man power
Monitor what voltage is sensed through the converter. Analog bus voltages connected to the intermediate layer and output layer are sequentially transferred to the MPU for monitoring on a layer-by-layer basis. Conversely, if the T mode is set to low, a high level voltage will always be output. DC31 and DC32 are input to an OR circuit, and are kept low while En output from the port register is high. Set T mode to 1 by port register, set layer to 1, and set En
By setting it high, it is possible to monitor the output by setting only the input layer to low. Memorize the result. Next, by setting the layer low, you can sense the offset on the analog bus in the middle layer. The port register makes this selection. In the case of the feedback type shown in Figure 28, the bus is basically 1
Since it is a book, it is possible to measure it only once. In either case, offset voltage and gain error cannot be measured unless the product-sum operation is performed. As will be described later, the weight for the dummy node is changed and the output voltage at that time is measured. In this case, each C3I and C8O is controlled and the output state of each layer is sensed in T mode. This sensed data is sent to the MPU via an A/D converter.

The error measurement circuit shown in FIG. 27 will be explained in more detail using the operation timing charts shown in FIGS. 29A and 30A. In FIG. 29, in one cycle of DLCK,
One cycle of product sum is performed. At least 16 weighted clocks WCLK are input when DCLK is high in the first half of one period of DCLK. This weight clock WCLK
is a clock for storing serial data in the shift register. The 5YNC signal specifies the operation timing of one ANP, and rises half a cycle before DCLK, and rises at half the cycle of DCLK. SH is the sample/hold signal and its WC
After LK is input, that is, after the input analog data and digital weight data are multiplied, the product signal is first charged into the capacitor. Then, when the first high signal of the sample/hold signal falls, the offset cancel signal ○C rises and reverses the polarity of the capacitor. At that time, the input signal to the capacitor is cut to 0, and sampled again.
When the capacitor is charged at the next high signal of the hold signal, a voltage equal to the offset voltage at the time of the input signal O is charged to the capacitor. As a result, the capacitor receives a signal that is reduced by the offset,
In other words, signals whose offsets have been canceled are accumulated. However, if an offset remains even after this, the error absorption method of the present invention is further utilized.

In FIG. 27, when a command signal to enter the test mode is input from MPtJ to the port register 1, the En output rises to high and the test mode 1 or 0 becomes enabled. When the layer rises from low to high, test the middle layer.

When the T mode is 1, an offset error signal is detected. Since the layer is high and the En signal is high, the output enable is high through gates 702 and 703.

The circuit is then enabled to bring the fixed voltage to the intermediate scrap input of the max value node circuit to O volts or high. Once the output enable goes high, it remains high until the end of the middle layer test. That is, offset errors are detected for the intermediate layer. Here, as described above, DCLK becomes a high signal and one cycle of product-sum is performed. After that, C3I (
C3O2) goes high and an output is output from ANPI onto the analog bus (interlayer).

Next, after a certain period of time, the C3I signal is also sent to ANP2.
Since the signals are added in a chain manner, an analog output signal is similarly outputted from ANP2 and is placed on the output bus of the intermediate layer. Further, after a predetermined period of time has elapsed, the C3I signal of ANP 3 rises to high, so that the analog output signal from ANP 3 is output onto the analog bus of the output of the intermediate layer. Each analog signal output to the intermediate layer is applied to an A/D converter 707 via an operational amplifier 704 and analog switches 705 and 706. and A/D converter 707
The output Qf of is sent to the MPU and temporarily held in the main memory. Next, as shown in the timing chart of FIG. 29B, the T mode is set to 1, the output voltage of the dummy node is set to high, and the MPU senses the output voltage Qg of each ANP.

The MPU calculates the adder gain Ag=(
Find Qg.

Furthermore, when the output of the layer of port register 1 is O, a high signal is applied to output enable 2 via inverter 708 and gates 709 and 710, so that the dummy node output terminal of the intermediate layer, that is, the second An output signal is generated from the output terminal and the AN of the output layer
Detect P4.5 offset. Then, the output is the operational amplifier 711. analog switch 71
2713 to the A/D converter 707. Note that the analog switches 714 and 715 protect the voltage with voltage followers when not outputting to the A/D converter 707. Note that the outputs of ANP4 and ANP5 are temporarily held by sample/hold 715 and 716,
Output as an analog output signal. The above is T
Since the mode is 1, ANPI, 2.3.4.
Since an input signal of 0 is added to each of 5, ANP
Offset voltages of operational amplifiers 1 to 5 are output from the A/D converter 707 to the ANP.

As shown in FIG. 29B, when the T mode is activated, high-level analog signals are output from the dummy node output for input layer and the dummy node output for intermediate layer. Since the A/D converter 707 outputs
Output voltage Qf (described later) of the operational amplifiers configuring PI to 5 is output.

FIG. 28 shows an error measurement circuit in a feedback neural network, and FIGS. 3OA and 30B show its operation timing. Only output signals indicating dummy node outputs to the input layer are output from the max value node circuit 187 and input to ANP12.3, respectively. AN
The output analog bus of PI, 2.3 is fed back to the input analog bus via the common bus CB. Port register 1, gates 702, 703, operational amplifier 704, analog switch 705, 706, A/D converter 707
, gates 708, 709, 710 and analog switch 714 are similar to those shown in FIG. Since one layer also serves as the output layer, there is no output layer like in a hierarchical type, so no dummy node output to the output layer occurs. A second output enable terminal that provides a dummy node output to the output layer is grounded.

And the output signal from ANPI, 2.3 is sampled/
It is output via hold circuits 718, 719, and 720.

The error measurement circuit shown in FIG. 28 will be explained in more detail using the operation timing charts shown in FIGS. 30A and 30B. In FIG. 30, in one period of DLCK,
One cycle of product sum is performed. At least 16 weighted clocks WCLK are input when DCLK is high in the first half of one period of DCLK. This weight clock W CL
K is a clock for storing 16 pieces of serial data in the shift register. 5YNC signal is one ANP
This specifies the operation timing of the signal, which rises half a cycle before DCLK and falls at half a cycle of DCLK. SH is the sample/hold signal and its W
After CLK is input, that is, after the input analog data and digital weight data are multiplied, the product signal is first charged into the capacitor. Then, when the first high signal of the sample/hold signal falls, the offset cancel signal OC rises.
Reverse the polarity of the capacitor. At that time, the input signal to the capacitor is cut to 0, and when the capacitor is charged again at the next high signal of the sample/hold signal, the capacitor has a voltage equal to the offset voltage when the input signal is O. Equal voltages are charged. As a result, the capacitor stores a signal reduced by the amount of the offset, that is, a signal with the offset canceled. However, if an offset remains even after this, the error absorption method of the present invention is further utilized.

In FIG. 28, when a command signal to enter the test mode is input from the MPU to the port register 1, the En output rises to high and the test mode 1 or 0 becomes enabled. When the layer rises high from the two mouths, test the first layer.

When the T mode is 1, an offset error signal is detected. Since the layer is high and the En signal is high, the first output enable goes high through gate 702. The circuit is then enabled in order to bring the fixed voltage to the scrap input of the max value node circuit into a 0 volt or high state. That is, the offset error is detected for one layer. Here, as described above, DCLK becomes a high signal and one cycle of product-sum is performed. In the feedback type, the output enable signal falls immediately, so the bus on the input side is disabled. However, while the output enable is high, all ANP offset voltages are already sampled and held internally. After that, csI (C
3o2) goes high, and the output from ANPI is output onto the output analog bus.

Next, after a certain period of time, the C3I signal is also sent to ANP2.
Since the signals are added in a chain manner, an analog output signal corresponding to the held offset is output from ANP 2 as well, and is placed on the output bus of this layer. Further, after a predetermined period of time has elapsed, the C3I signal of ANP3 rises to high level, so that the analog output signal from ANP3 is outputted onto the layer output analog bus. Each offset analog signal Qr output to the layer is sent to the A/D converter 707 via an operational amplifier 704 and analog switches 705 and 706.
added to. And the output Q of A/D converter 707
f is sent to the MPU and temporarily held in the main memory. Next, as shown in the timing chart of FIG. 30B, the T mode is set to 1, the output voltage of the dummy node is set to high, and the MPU senses the output voltage Qg of each ANP.

The MPU calculates the adder gain Ag=(
Calculate Qg. The above is a case where the T mode is 1, so input signals of 0 are applied to ANPI and 2.3, so the offset voltage of the operational amplifiers ANPI to 5 is output from the A/D converter 707 to the MPU side. be done.

As shown in FIG. 30B, when the T mode is activated, a high-level analog signal is output from the dummy node output fo input layer, so the A/D converter 707 outputs ANPI or The output voltage Qg (described later) of the operational amplifier configuring the circuit 3 is output.

Next, a calculation error generation model in an analog neuron processor and a weight correction type method using dummy nodes will be explained.

FIG. 31 is a schematic diagram of the algorithm for the primary correction and secondary correction processing of the present invention. The first correction process is a process of setting measurement conditions for estimating the adder gain and measuring an offset voltage. That is, when the flowchart starts, the fixed voltage of the dummy node is set to O, and 1-Qf of the weight data for error measurement is reset as the weight for the dummy node. The second correction process is the result of multiplying the offset voltage Qf for a dummy node of 0 volts by 1-Qf as an intermediate weight for a dummy node of 1 volt, and then the offset voltage is added to the result, which is a mixed error output Qg. Correct adder gain A using the two pieces of information
The purpose is to calculate g=(Qg=(Q-Qr)/(l-Ql).

The weight correction method assigns a weight to each neurochip with respect to a fixed voltage generated from a dummy node (this weight is hereinafter referred to as
This is a method of absorbing error voltages by adjusting the dummy node weights.

Errors include offset errors and gain errors.

In the present invention, dummy nodes logically exist.

That is, since a method is adopted in which a fixed voltage of, for example, 1 volt is generated from the max value node circuit to the analog bus, errors are absorbed by adjusting the weight for the dummy node. How much error occurs when 0 volt input is applied to the ANP and what path the error occurs will be explained below as an example.

FIG. 32 is a conceptual diagram of a weight correction type method using an arithmetic error model and dummy nodes in an analog neuron processor. For example, if the weight is 0°3 and the input is 0, theoretically the output will be O. Adder gain is 0.98
Then, even if you multiply the input 0 by this 0.98, it will be 0,
There's no point. On the other hand, if the input is 1 volt, I
0.3 is the original product value. If the adder gain is 0.98, the value actually multiplied is 0.98
It becomes X O,3. Similarly, for 1 volt from other nodes, 1 x 0.4 and (-1,0) x (-0,6)
If we calculate the sum of all the values obtained by multiplying these volumes by 0.98, we get 1.274 volts, as shown in the figure. In addition to this, there is an offset of the op amp, and 0
．． 01 volts is applied, resulting in 1.264
Becomes a bolt. The actual value is 1 x 0.3 + I
Since 0°4+(-1,0)X(-0,6), the voltage is 1.3 volts. However, what should have been 1.3 volts turned out to be 1.2 volts due to gain error and offset error.
It will be 64 volts. This is the calculation error generation model.

The figure below is a conceptual diagram of the weight correction type method, where the offset voltage -0, which is generated in the calculation error generation model, is
In order to correct the error of 01 volts, in this figure, the voltage of the dummy node is first set to 090 volts. At this time, even if the adder gain is 0.98, the 1-lead output is 0.0 volts, but an offset voltage is added, and the offset voltage Qf becomes -0.01 volts. Then, as a correction, the weight for the dummy node is changed from 1 to 1.01 in order to offset Qf. This is the first correction, and generally has a value of 1, O-Qf. However, the adder gain should be 1.0 but is 0.98, so 1.0IX0.98-〇,
9898 volts is #Hayabusa result. Furthermore, if the offset of the operational amplifier is similarly set to -0.01 volts, this is added and the output becomes 0.9798 volts, which is taken as the mixed error output value Qg. Calculate the additive gain Ag = (Qg The second correction follows the following equation as shown in the figure.

%Formula%(1 In other words, in the current example, Ag=(Qg is 0.98, so finding Ag=(Qg is the first step in the second correction.
This is the next step. This is because the first correction amount (iQf) is multiplied by the fixed voltage of 1 volt, and the adder gain Ag=(
The product multiplied by Qg plus the offset voltage Qf becomes Qg, so Qg-(1-Qy) X I XA
This is because g=(Qg +Qf, and furthermore, the weight Wll
However, 0-W8×Ag=(Qg+Q.

Since it is necessary to satisfy , the second step of the secondary correction is to find . The reason for this is that when the output signals ~123 from all ANPs in the previous layer are all 0, the output signals of each ANP in the subsequent layer should also be 0 even if there is an offset in the ANP. In other words, since the output of the dummy node is always 1, the weight for the input signal from the dummy node to each ANP in the subsequent layer is determined so that the output signal of the subsequent layer becomes 0. This is because it would be a good thing.

This W3=0.010204 is applied to a 1 volt dummy node as a second-order corrected weight, and the adder gain A
g = (multiplying Qg = 0.98, 0.0099999
Therefore, it becomes approximately 0.01 volt. Since the offset voltage -0601 is added to this, the result is O volts.

In this way, the offset is also related to the gain. It would be better to correct both offset gains simultaneously and independently, but since this is not possible, a method is used in which both of them are corrected in a mixed form. In this way, when a fixed voltage of 1 volt is generated from the dummy node, the offset voltage that is output when the ANP input is 0 volts is canceled, and the output can be corrected to become 0 volts as a result. It becomes possible. Such correction processing is performed by setting the correction enable flag "F,n" to 1.

When the T mode is set to 1 in the error measurement circuit in the hierarchical neural network shown in FIG. 27, the fixed voltage generated in the max value node circuit is forced to 0 volts. This forces O volts to be input to the analog bus ■ or ■. For example, if the input layer analog bus ■ is set to 0 volts, each intermediate layer ANPI, 2
．． The output of No. 3 is output as an offset voltage. This is input to the MPU side via the A/D converter 707.

The MPU side stores this offset voltage as Qf. M
The PU uses this Qt to perform the first correction. When Qf is given, it is necessary to use this to calculate the weight for the dummy node and correct the weight. In the second half of the first correction, the fixed voltage of the dummy node is set to 1 volt. For example, if the first ANP 1 in the intermediate layer is targeted, the weight data for this dummy node voltage is controlled to be 1.01. If the adder gain of that ANPI in the middle layer is 0.
98, the output of the adder will be 0.9898,
Furthermore, an offset voltage of -0.01 volts is added and Qg
0.9798 volts is output. This is given again to the MPU as Qg information via the A/D converter inside the error measurement circuit connected to the output analog bus of the intermediate layer. The MPU then calculates Ag=(Qg and 1/Ag=(Qg to W). As a result, Wa is the weight data for the dummy node that generates 1 volt as the secondary correction amount, that is, 0.010204 At this time, since the adder gain in ANP is 0.98, the adder output becomes 0.01 volts, and the offset is added to become O, volts.If the MPU confirms this, the dummy Since the weight for the node is found to be the correct value of 0.010204, the MPU stores W8 in the weight memory.

The above operation will be performed for all ANPs.

FIGS. 33A to 33D are weight data correction algorithms for a hierarchical network.

Set the mass and mask control block to error measurement mode. The test mode, that is, T mode, is set to 1, the layer is set to 1 (intermediate layer), and En is set to 1. Then, the fixed voltage of the dummy node in the input layer becomes 0 volts, so the offset voltage of each ANP in the intermediate layer is measured.

In order to perform the first correction, the MPU performs the first correction and the second correction by starting an interrupt process. First, the interrupt number counter variable that counts the number of ANPs being processed in the intermediate layer is set to O, and interrupt processing is started.

Then, when the ANP outputs the output voltage with respect to the 0 volt input voltage as Qf, the MPU reads the data Qf into an internal register through the A/D conversion of the measurement circuit, and then transfers it from the register to the main memory. . Then, the interrupt counter variable is incremented, and it is checked whether the interrupt counter number matches the number of objects to be measured, that is, the number of ANPs. If they do not match, the count number is sensed and the wait state is entered. If the count variable is equal to the number of objects to be measured, the interrupt processing routine is ended and the first correction processing for the ANP in the intermediate layer is started. That is, the voltage of the dummy node is set to 1 volt for each ANP, and the first correction value is 1.0.
First, all ANs are stored in the main memory using Qr as a weight.
Store it as a weight for P. That is, the T mode is set to O.

Then, in T mode -0, a fixed voltage of 1 volt is generated from the dummy node, so the effects of offset voltage and gain error are simultaneously measured, and the second
Perform preprocessing for the next correction. The interrupt number counter variable is set to 0 again and interrupt processing is started. That is, Qg is measured through A/D conversion of the measurement circuit. The voltage of the dummy node is 1.0, and the first correction makes the weight to ANP, for example, 01, so the output is 0.9898 for the adder gain of 0.98, and the offset voltage is -0. ,01 volt is added and Qg is 0,
9798 volts is calculated. The MPU reads this Qg via the A/D converter, moves it from the register to memory, and increments a count variable to do this for all ANPs in the middle layer. A check is made to see if this incremented number has reached the number to be measured, and if they do not match, further measurement is performed. If the counter number is equal to the number of objects to be measured, the interrupt processing routine is ended and the second correction processing begins. That is, Ag
=(Wa, which is the inverse of Qg multiplied by -Qf, is obtained for each ANP. This Wll is given as weight data to a dummy node that generates 1 volt.

That is, a routine is entered to modify the weight for the dummy node, and for example, the weight data for ANP is set to 0.010204.
I decide. These weights are stored in weight memory.

The above operation will also be performed for all ANPs in the output layer. Therefore, moving to (2), set the T mode to 1, set the layer to O, set the output layer mode, set En=1, and enter the same process. That is, the interrupt counter is set to 0, the interrupt handling routine is entered, and the measurement circuit calculates the offset voltage, Qf, with respect to the 0 volt dummy node.
is calculated for the ANP in the output layer, and these data are stored in memory. This process is performed on the number of measurement targets (output layer ANP
number), exits the interrupt processing routine and returns to the first
Next correction processing will be performed. That is, intermediate weight data for the fixed voltage of 1 volt of the dummy node is set.

Move on to ■. Set T mode to O, layer to 0, and En to 1. Then, preprocessing for the second correction begins, and Q
Calculate f. That is, from the dummy node to the fixed voltage 1
Since volts are generated, the effects of offset voltage and gain error will be measured simultaneously. Set the interrupt count variable to 0, start interrupt processing, and set Qg to A.
/D converter to read and transfer to memory. Then increment the interrupt count variable and increment the ANP in the output layer.
After repeating the number of times, the CPU exits from the interrupt processing routine, calculates Wa for the second correction, and stores it in the weight memory as a dummy node weight.

In the case of the hierarchical type, Wa obtained by the second correction needs to be added to the weight originally given to the dummy node. Moreover, all weights need to be multiplied by 1/Ag=(Qg.

The weight data modification algorithm for the feedback network is shown in Figures 34A and 34B. The same applies to this case. A feedback network has one layer.

Set the mask control block to error measurement mode and press T.
Set the mode to 1, layer to 1, and En to 1, and first measure the offset voltage by the 0 volt voltage output of the dummy node. Set the interrupt counter variable to 0, enter the interrupt processing routine, and connect the A/D on the output side of the measurement circuit.
Read Qf via transformation and move it to memory. This process is repeated by the number of ANPs, the interrupt processing routine is ended, and the first correction process is started.

Since there is only one layer in the feedback network, the first and second corrections only need to be performed once. Next, in order to perform the second correction, leave the T mode at 0 and the layer at O,
Set En to 1. Then, a fixed voltage of 1 volt at the dummy node is input, and the effects of offset voltage and gain error are simultaneously measured. Set the interrupt count counter variable to 0.
and start interrupt processing. Qg is read for each ANP via A/D conversion of the measurement circuit. That is, when weight data determined by the first correction is given to 1 volt from the dummy node, ANP multiplies the internal adder gain of 0.98 and outputs Qg as a voltage with an offset voltage added. . This process is performed on all ANPs for one layer, and Qf and Qg corresponding to each ANP are
When the information is written to memory, the interrupt handling routine ends.

Then, the MPU performs secondary correction processing. That is, M
The PU calculates Ag=(W from the reciprocal of Qg) for each ANP and stores this in the weight memory as a weight correction amount.

〔Effect of the invention〕

According to the present invention, by setting the weight for the dummy node so that the output signal is 0 when the input signal to the ANP is O, errors in the offset voltage of the neurocomputer and errors in the gain of the adder are suppressed. can do.

[Brief explanation of the drawing]

FIG. 1A is a principle block diagram of the neurocomputer of the present invention, and FIG. 1B is a block diagram of the principle of the neurocomputer of the present invention.
1C is an internal configuration diagram of the ANP of the present invention, and FIG. 2 is a schematic diagram of a package composed of a chip of
FIG. 3 is a block diagram of an embodiment of the basic unit of the present invention; FIG. 4 is a specific circuit diagram of an embodiment of the basic unit of the present invention; 6 is a diagram illustrating the operation timing of the integrator used in the basic unit of the present invention. FIG. 7A is a diagram illustrating the operation timing of the integrator used in the basic unit of the present invention. A conceptual diagram of the network. Figure 7B is a conceptual diagram of a hierarchical neural network according to the present invention. Figure 8 is a specific circuit diagram of an embodiment in which the neurocomputer of the present invention is realized by a hierarchical network. Figure 9A 9B is a timing diagram of the signal processing shown in FIG. 8, FIG. 10 is a diagram showing the timing of reading digital weight data, FIG. 111A is a specific circuit diagram of the mask control block, and FIG. 11B is a diagram showing the timing of reading digital weight data. , FIG. 12A is a diagram showing a method of filling data into the weight data memory, FIG. 12B is a specific configuration diagram of the weight data memory, FIG. 12C is a diagram showing the structure of the control pattern memory and the microcode memory, Fig. 12D is a flowchart of the learning algorithm, Fig. 13 is a specific circuit diagram of the daisy circuit, Fig. 14 is a specific circuit diagram of the max value node circuit, and Fig. 15 is a specific circuit diagram of the sigmoid function generation circuit. , Fig. 16 is a specific circuit diagram of a sequence generator, Fig. 17 is a specific circuit diagram of a phase control circuit, Fig. 18 is a specific circuit diagram of a shift register, and Fig. 19A is a specific circuit diagram of a feedback network. A conceptual diagram to be explained. FIG. 19B is an explanatory diagram when a feedback network is configured using the neurocomputer of the present invention. FIG. 20 is an explanatory diagram when a first feedback network is configured using the neurocomputer according to the present invention. 21A and 21B are timing diagrams showing the signal processing of the embodiment shown in FIG. 20, and FIG. 23A and 23B are timing diagrams showing the signal processing of the embodiment shown in FIG. 22, and FIG. 24 is a hierarchical FIGS. 25A and 25B are block diagrams of other embodiments combining the feedback type and the feedback type, and FIGS. 26A and 26 are timing diagrams showing signal processing of the embodiment shown in FIG.
Figure B is a conceptual diagram of the mechanism that generates errors in an actual ANP, and Figures 27 and 28 are error measurement circuits in hierarchical and feedback neural networks that measure this pulse-like error voltage, respectively. Figure 29 showing
Figure A is a control sequence for measuring offset voltage in the middle layer of a hierarchical network, Figure 29B is a control sequence for measuring gain error in the middle layer of a hierarchical network, and Figure 30A is a control sequence for measuring offset voltage in a feedback network. Control sequence, FIG. 30B is a control sequence for measuring gain error in the feedback network, FIG. 31 is an explanatory diagram of the first and second correction processing used for error measurement of the present invention, and FIG. 32 is: 33A to 33D are flowcharts showing a weight data correction method in the case of a hierarchical network; Figure 34B is a flowchart showing the weight data correction method in the case of feedback type network, Figure 35 is a diagram showing the principle configuration of the basic unit of the neuron model, and Figure 36 is a conceptual diagram of the configuration of the hierarchical neural network. be. 6. Dummy node means, 12. 13. 14. 15. 16. 17. 18. Work, 19. 20. - Control pattern memory, - Sequencer, - Weight memory, - Digital control means, - D/A converter , ・A/D converter, ・Neural network composed of ANP, ・Weight correction means, ・Error measurement means. Figure 1B Figure 1C Weight data correction algorithm (for hierarchical network) Offset voltage is measured by 0V voltage input. Weight data correction algorithm (in case of feedback network) Measure offset voltage by inputting 0v voltage.

Claims (1)

[Claims] 1) A neural network consisting of a set of analog neuroprocessors that time-divisionally input analog signals from a first analog bus, perform product-sum operations, and output the analog signals to a second analog bus. (18); dummy node means (6) connected to the analog bus of the neural network (18) and generating a fixed voltage on the specified analog bus in the test mode; and a first state in the test mode. error measuring means for forcibly inputting 0 volts to the first analog bus through the dummy node means (6) and detecting an offset voltage generated within the analog neuroprocessor from the second analog bus; 20), and an intermediate weight to each neuroprocessor to be multiplied by a fixed voltage generated from the dummy node means (6) in the second state in the test mode from the offset voltage of each neuroprocessor. Weight correction means (19) within the digital control means that determines the weight and calculates the correct weight using the gain from the detected output voltage output from the second analog bus, and the weight corrected by the weight correction means. Weight correction in a neurocomputer characterized by comprising a weight memory (14) for storing weights, and a control pattern memory (12) from which control patterns for controlling the operation of the neural network are sequentially read out under control of a sequencer (13). Error absorption method 2) The weight correction means sets the fixed voltage of the dummy node to 0.
The first correction means calculates the offset voltage Q_f by calculating the weight data (1-Q_f) for error measurement; The mix error output formed from the voltage Q_f is set as Q_g, and the adder gain A_
Second step to find g=(Q_g-Q_f)/(1-Q_f)
3) The dummy node comprises a max value node circuit, and the max value node circuit comprises means for generating a fixed voltage. , comprising means for outputting the fixed voltage from a dummy node output at a predetermined timing.4) An error absorption method using weight correction in a neurocomputer according to claim 1, wherein the error detection means is configured to operate in a test mode according to a designation from an MPU. an enable signal for enabling the max value node circuit; and a test mode signal for selecting a specified output voltage of the max value node circuit as a 0 volt voltage or a non-0 volt voltage;
port register means (701) for generating a layer signal for specifying each layer; and a port register means (701) for generating a test mode signal from the max value node circuit to an input analog bus on the input side of the layer corresponding to the state of the layer signal when the enable signal is in the enable state. control means (702, 708, 7) for generating an enable signal for enabling generation of a fixed voltage corresponding to a state;
09, 703, 710) Error absorption method by weight correction in a neurocomputer according to claim 1. buffer means (711) for receiving the voltage; switching means for effectively outputting the detected output voltage when the enable signal output from the digital circuit means is in an enable state; and switching means connected to the switching means for outputting the detected output voltage. 2. An error absorption system using weight correction in a neurocomputer as claimed in claim 1, further comprising an A/D conversion means (707) for outputting a digital quantity to a digital control means including an MPU.