JPH04111185A - Signal processing network - Google Patents

Abstract

PURPOSE:To improve processing speed and reliability by constituting a neuron with a digital circuit and parallely executing the function of the neuron on a hardware or software including self-learning function. CONSTITUTION:A neuron imitation circuit 45 is composed of blocks B1, B2, and B3. The block B1 transmits calculation output to each neuron ahead by one with the use of a coupling coefficient variable circuit. The block 2, composed of a block 2-1 of one-ahead layer and a block B2-2 of the corresponding neuron, is corresponded to an error signal generation circuit. Blocks B3-1 to B3-N are corresponded to the coupling coefficient variable circuit and an A/D converter. A coupling coefficient variable circuit 20 in the block B2-1, B3, and B1 constitutes a self-learning circuit. The part of the block B2-2 and the remaining parts of the block B1 are corresponded to the neuron imitation element.

Description

[Detailed description of the invention]

[0001]

[Industrial application field]

The present invention relates to a signal processing circuit network such as a neural computer that imitates a neural network. [0002]

[Conventional technology]

This "neuron-mimetic element" imitates the functions of neurons (neurons), which are the basic units of information processing in living organisms.
The network was used to realize parallel processing of information.
This is a so-called neural network. Character recognition, associative memory, motor control, and many other things that are easily accomplished in living organisms are difficult to achieve with conventional Neumann computers. Many attempts are being made to solve these problems by imitating the nervous system of living organisms, especially functions unique to living organisms, such as parallel processing and self-learning. Many of these attempts have been carried out using computer simulations, and in order to achieve their original functionality, parallel processing is required, and for this purpose it is necessary to implement neural networks in hardware. Although some attempts have already been made to turn it into hardware, the self-learning function that characterizes neural networks has not been achieved, which has been a major bottleneck. Furthermore, most of them are implemented using analog circuits, which poses problems in terms of operation, as will be described later. Unexamined Japanese Patent Publication No. 1-111185 (6)

[0003]The conventional methods will be considered in order below. First, a conventional neural network model will be explained. FIG. 34 is a diagram showing a certain neuron unit 1, which connects with other neuron units, receives signals, processes them, and outputs them. In the case of FIG. 33, the network is hierarchical, and a signal is received from a unit in the previous (left) layer and output to a unit in the next (right) layer. [0004] This will be explained in more detail. First, in neuron unit 1 in FIG. 34, what is called the coupling coefficient represents the degree of coupling between other neuron units and the own unit.
The coupling coefficient between the first unit and the first unit is generally expressed as To. There are excitatory connections in which the larger the signal from the other neuron, the higher the own output, and conversely, the larger the signal from the other neuron, the smaller the own output. Now, suppose your unit is the first unit,
This is the input to the first unit. As mentioned above, each unit is an input to its own unit which is connected to a number of units. This is called the internal potential and is U. It is expressed as [0005]. [0006] Next, this input is subjected to nonlinear processing and output. The function at this time is called a neuron response function, and a sigmoid function as shown in equation (2) and FIG. 35 is used as the nonlinear function. [0007]

[0008] When a network is created as shown in FIG. 33, each coupling coefficient is 1. The final output can be obtained by calculating equations (1)j and (2) one after another. [0009] On the other hand, an example of such a network realized by an electric circuit is shown in FIG. 36. This is shown in Japanese Patent Application Laid-Open No. 62-295188, and basically consists of a plurality of amplifiers 2 having an S-shaped transfer function and the output of each amplifier 2 being connected to the input of an amplifier in another layer at a single point. A resistive feedback network 3 is provided, which is connected as shown in dashed lines. A CR time constant circuit 4 consisting of a grounded capacitor and a grounded resistor is individually connected to the input side of each amplifier 2. And input current I
I~, , is applied to the input of each amplifier 2, and the output is obtained from the set of 1'2' output voltages of these amplifiers 2. [0010] Here, the strength of input and output signals is expressed by voltage, and the strength of the connection between neurons is expressed by the resistance 5 (lattice point in the resistive feedback network 3) that connects the input and output lines between each cell. The nerve cell response function is represented by the transfer function of each amplifier 2. Further, as mentioned above, the connections between nerve cells have excitatory and inhibitory characteristics, which are mathematically expressed by the positive and negative signs of the connection coefficient. However, it is difficult to realize positive and negative values using constants on the circuit, so here we divide the output of amplifier 2 into two,
By inverting one output, two positive and negative signals are generated, and this is achieved by appropriately selecting one of these signals. Furthermore, an amplifier is used as an equivalent to the sigmoid function shown in FIG. [0011] However, these circuits have the following problems. Self-learning is not possible as the user has no choice but to use the set values. ■ Represents signal strength with analog values such as potential and current,
When internal calculations are also performed analogously, the values change due to temperature characteristics, drift immediately after power-on, etc. 'J+l:il +4-111185 (8)■ Since it is a network, it is difficult to make the characteristics of each power gourd the same, which requires a large number of elements. ■ When the accuracy or stability of one element becomes a problem, when it is made into a network, new problems may arise, and the behavior of the entire network cannot be predicted. [0012] On the other hand, as a learning law used in numerical calculations, there is the following one called backpropagation. [0013] First, each coupling coefficient is given randomly. If input is given in this state, the output result will not necessarily be desirable. For example, in the case of character recognition, handwritten "1"
'', it is desirable that the output result be ``This character is 1.'' However, if the force plate coupling coefficients are random, this is not necessarily a desirable result. Therefore, this network has the correct answer (teacher signal)
, and then change each coupling coefficient so that the correct answer is obtained when the same input is given. At this time, the algorithm used to find the amount by which the coupling coefficient is changed is called backpropagation. [0014] For example, in a hierarchical network as shown in FIG. 33, the j-th neuron output of the final layer is bound by y,
Let d be the teacher signal for that neuron, J

J and E expressed as are minimum, ΔT,,=c3E/,EAT,,...
・・・・・・・・・・・・・・・・・・・・・(4) IJ
Using IJ, the coupling coefficient T1. change. More specifically, when determining the coupling coefficient between j0 and the previous layer, first, the output layer and its δ, = (d--y,)Xf' (u,)...
・・・・・・・・・・・・・・・・・・・・・(5) J
Using JJ J, δ
(error signal), and when calculating the coupling coefficient between layers further before that, δ, = Σδ, T,, Xf' (u,)...
・・・・・・・・・・・・・・・・・・・・・(6)J
Determine δ (error signal) using Jlcoj, and 8T1. =η(δ,y,)+α△T9. (Last learning time) IJJJ Co-j T,,=T,Before previous learning time)+△T1. ...(7)
IJ IJ
Find IJ and change To. Here, η is a learning constant, and α is a stabilization constant. Each cannot be determined logically, so they are determined empirically. In general, the smaller these numbers are, the slower the convergence is, and the larger these numbers are, the more likely they are to oscillate. In terms of order, it is about 1. Further, f' is the first-order differential function of the sigmoid function f. [0015] Learning is performed in this manner, and then input is given again to calculate the output and learning is performed. By repeating this operation many times, you will eventually reach a coupling coefficient T0 that yields the desired result for the given input. is determined. j [0016] Now, if it is attempted to implement such a learning method in hardware by some method, the learning requires a large amount of four arithmetic operations, which is difficult to implement. The learning method itself is also unsuitable for hardware implementation. [0017] On the other hand, an example of a neural network realized using a digital circuit will be described with reference to FIGS. 37 to 39. Figure 37
shows the circuit configuration of a single neuron, and each synaptic circuit 6
is connected to the cell body circuit 8 via the dendrite circuit 7. FIG. 38 shows an example of the configuration of the synapse circuit 6, in which a rate multiplier 10 is inputted with a value obtained by multiplying the input pulse f by a multiplication factor a (a multiplication factor of 1 or 2 for the feedback signal) via a coefficient circuit 9. A synapse weight register 11 storing a weighting value W is connected to the rate multiplier 10. FIG. 39 shows an example of the configuration of the cell body circuit 8, in which a control circuit 12, an active down counter 13, a rate multiplier 14, and a gate 15 are connected in order, and an up/down memory 16 is provided. There is. [0018] :Jll;rl'1l-111185(10) This is
The input and output of a neuron unit is represented by a pulse train, and the pulse density represents the amount of signal. The coupling coefficient is expressed as a binary number and stored in the memory 16-H. By inputting the input signal to the clock of rate multiplier 14 and inputting the coupling coefficient to the rate value, the pulse density of the high power signal is reduced in accordance with the rate value. This corresponds to the T,,V, parts of the backpropagation model equation. Next J 1 Mark ○R for each group. An output is obtained by inputting these two outputs to the up side and down side of the counter 13 and running them. Since this output is a binary number, the rate multiplier 14 is used again to convert it into a pulse density. By making these units into a network, a neural network can be realized. Learning is realized by inputting the final output to an external computer, performing numerical calculations inside the computer, and writing the results into the coupling coefficient memory 16. Therefore, there is no self-learning function at all. In addition, the circuit configuration uses a counter to convert the pulse density signal into a numerical value.
After that, it was converted back to pulse density, making it more complicated. [0019]

[Problem to be solved by the invention]

As described above, the conventional technology has the disadvantage that self-learning cannot be performed on the hardware. [00201] Furthermore, analog circuits do not operate reliably, and learning methods based on numerical calculations require complicated calculations, making them unsuitable for implementation in hardware. On the other hand, digital systems that operate reliably have complex circuit configurations. [0021] Means for Solving the Problems In the invention according to claim 1, in a hierarchical signal processing circuit network consisting of a plurality of aggregates each having logical operation means,
The final output signal outputted from the logic operation means and the teacher signal corresponding to this logic operation means are compared, and a signal that exists only in this teacher signal is determined as a positive error signal, and a signal that exists only in the final output signal is determined as a positive error signal. Comparison output means for generating an error signal in this logic operation means as a negative error signal, and logic operation means in a certain aggregate that provides the output signal to the operation means forming another aggregate, A coupling coefficient signal consisting of at least one of an excitatory coupling coefficient signal and an inhibitory coupling coefficient signal representing a coupling state with the constituent computing means, and a positive error in the computing means constituting the other aggregate. An excitatory coupling teacher signal and the positive error signal among the coupling coefficient signals, and an inhibitory coupling coefficient signal among the coupling coefficient signals. and the negative error signal to generate a positive error signal in the logical operation means in the certain aggregate, and among the coupling coefficient signals, the suppressive coupling coefficient signal and the other aggregate A logic operation is performed based on the positive error signal, an excitatory coupling coefficient signal among the coupling coefficient signals, and the negative error signal to generate a negative error signal in the logic operation means in the certain aggregate. All the input signals input to the logic operation means constituting the other aggregate, the positive error signal and the negative error signal in this logic operation means, and the output to this logic operation means. A coupling coefficient control means is provided for controlling the coupling coefficient signal based on a coupling coefficient signal representing a coupling state with the arithmetic means constituting the certain aggregate that provides the signal. [0022] In the invention according to claim 2, the first aggregate, the final aggregate, and the output signal from the first aggregate, each having a logical operation means, are received and the output signal is supplied to the final aggregate. It consists of an intermediate aggregate, and signals are exchanged between the logic operation means in one aggregate and the logic operation means in another aggregate in the aggregate, so that an input signal is given to the first aggregate. By comparing the final output signal outputted from the final assembly with a specific teacher signal when The invention according to claim 1 is applied to a hierarchical signal processing circuit network configured to converge the final output signal from the logic operation means in the final aggregate obtained for the input signal to the teacher signal. 0
[023] In the invention according to claim 3, in the invention according to claim 1 or 2, the input signal, the output signal, the final output signal,
All signals of coupling coefficient signal, teacher signal and error signal are
The signal was expressed by a pulse train and a pulse density.

[0024] In the invention as claimed in claim 4, a self-learning circuit comprising a variable coupling coefficient circuit and a coupling coefficient generation circuit that generates a variable coupling coefficient value of the variable coupling coefficient circuit based on an error signal with respect to a teacher signal is imitated by neuron. Multiple neuron mimicking circuits attached to the device were connected in a network. Alternatively, in the invention according to claim 9, the neuron mimicking element that calculates the input represented by the pulse density and the coupling coefficient represented by the pulse density to obtain the output determined by the pulse density signal has its own A plurality of neuron imitation circuits equipped with a control means for controlling the coupling coefficient based on the error signal between the output and the teacher signal were connected in a network. [0025] In the invention set forth in claim 5, the self-learning circuit and the neuron imitation element are formed by a digital logic circuit. [0026] In the invention described in claim 6, processing is performed for each coupling coefficient divided into two groups, an excitatory coupling group and an inhibitory coupling group, and negating the result of one group and combining the result of the other group. A logic circuit that performs logical product is provided. In the invention described in claim 7, processing is performed for each coupling coefficient divided into two groups, an excitatory coupling group and an inhibitory coupling group, and the negation of the result of one group and the logical sum of the result of the other group are performed. A logic circuit is provided to take the following steps. In the invention as set forth in claim 8, a majority circuit is provided that determines the output of the neuron mimicking element based on the ratio of the output result by the excitatory coupling coefficient group and the output result by the inhibitory coupling coefficient group. [0027] In the invention described in claim 9, in the invention described in claim 10, a memory storing a coupling coefficient for each input is individually provided, and it is determined whether the input is an excitatory connection or an inhibitory connection. They were divided into two groups in advance. Or claim 11
In the described invention, a memory storing coupling coefficients for each input is individually provided, and information representing two types of coupling is stored, and the inputs are grouped into two groups, excitatory coupling and inhibitory coupling. memory for use. In these cases, in the invention according to claim 12, the control means is a self-learning circuit that generates a variable coupling coefficient value by learning based on an error signal between its own output and a teacher signal, and variably controls the coupling coefficient in the memory. And so. [0028] In the invention according to claim 13, a memory storing an excitatory coupling coefficient and a memory storing an inhibitory coupling coefficient are separately provided for each input, and the excitatory coupling and the inhibitory coupling coefficient are separately provided for each input. The content of the read memory and the input content are logically operated on each group to select either the excitatory or inhibitory output, and the self output and the teacher signal are divided into two groups. A self-learning circuit is provided that generates variable coupling coefficient values through learning based on the error signal of , and variably controls the coupling coefficients in each memory. [0029] In the invention described in claim 14, in the invention described in claim 9, a learning constant setting means for arbitrarily setting and varying the learning constant used in the variable coupling coefficient circuit from the outside is provided. [0030]

[Effect]

The functions of the neuronal network, including the self-learning function, can be performed in parallel on hardware, and the self-learning function is demonstrated, significantly improving processing speed compared to calculations using serial processing in conventional computer simulations. At this time, the digital circuit configuration ensures reliable operation. In particular, since all signal processing is done digitally using pulse density expression, there are no disadvantages like in analog systems, such as being affected by the temperature characteristics of the amplifier. [0031] In particular, if configured as in the invention according to claim 1 or 2,
It is also possible to realize the functions of neurons with software, and the configuration of the network can be changed simply by changing the software, making it possible to construct a highly flexible and versatile network. [0032] At this time, there are two types of coupling, excitatory and inhibitory, depending on the sign of the force plate coupling coefficient [0033] and is rewritable and versatile. [0034] It becomes easy to use. [0035]

【Example】

A first embodiment of the present invention will be described based on FIGS. 1 to 12. This embodiment is provided with a self-learning function, and in order to enable self-learning, the coupling coefficient is made variable by appropriately switching the coupling coefficient between 2 and 2. The switching circuit 22 also switches and controls the switching circuit 24 using a commercially available sign bit that can be switched by inputting a binary number from an external controller, and selects whether or not to pass the inverting amplifier 25. Switching between excitatory output and inhibitory output occurs. This makes the coupling coefficient variable according to the external signal S,
An output is obtained by multiplying the input signal by an arbitrary coupling coefficient. [0036] FIG. 3 shows that (1
)(2). Here, the coupling coefficient variable circuit 20 has the function of multiplying and matching the input from the previous layer by each coupling coefficient, and the input is input to the adding circuit 26 and added. The adder circuit 26 can be easily realized by using a commercially available operational amplifier 27, for example, as shown in FIG. Here, the operational amplifier 27 is used for addition, but since it has an inverting amplifier configuration, the amplifier 28 is further used to invert the signal again and output it. Addition circuit 2
A nonlinear amplifier 29 having an input/output relationship as shown in equation (2) is connected to the output side of the amplifier 6. Its input signal is (1
) corresponds to the internal potential of the equation. [0037] Next, a method of creating the external signal S for determining the coupling coefficient will be described. This corresponds to equations (5) to (7), and requires a circuit to realize this. First, f' in equations (5) and (6) is a first-order fja component function of the sigmoid function f shown in FIG. 35, and has characteristics as shown in FIG. For example, as shown in FIG.
.about.35 are connected in multiple stages and constituted by a circuit having nonlinear chevron-shaped characteristics. The input/output characteristics of this f' signal generating circuit 30 are as shown in FIG. Although this circuit 30 does not necessarily realize the characteristics shown in FIG. 5 exactly, it can be said that it approximately achieves the characteristics. Furthermore, if an amplifier (not shown) having nonlinear input/output characteristics is provided in advance before inputting to the f' signal generation circuit 30, the input/output characteristics will be as shown in FIG. This will bring us closer to the characteristics. [0038] FIG. 9 shows an example of the error signal generation circuit 36 corresponding to equation (5). The circuit 30 in the figure is shown in FIG. 6, and performs functional processing as shown in FIG. 7 or 8 on the internal potential (input to the amplifier 29 in FIG. 3). On the other hand, a subtraction circuit 37 is provided which takes the difference between the neuron unit output of the output layer and the teacher signal. This subtracting circuit 37 may use a circuit as shown in FIG. 4, and one input may be inverted in advance by an amplifier. The outputs of these circuits 30 and 37 are input to the drop calculation circuit 38 and multiplied, yielding a result similar to equation (5). [0039] On the other hand, an example of the error signal generation circuit 39 corresponding to equation (6) is shown in FIG. That is, the coupling coefficient variable circuit 20, the addition circuit 26, and the f' signal generation circuit 30 have the circuit configurations as described above.
and the product of the outputs of these circuits 26.30 and 6
A percentage calculation circuit 40 is provided. Such a configuration is equivalent to equation (6). Therefore, a circuit 36 as shown in FIG. 9 or a circuit 3 as shown in FIG. 10 in another layer is added to this in advance.
By inputting the error signal created by 9 and the internal voltage, an output similar to that of equation (6) can be finally obtained. Furthermore, a coupling coefficient generation circuit 41 corresponding to equation (7)
can be realized by a multiplication circuit. FIG. 11 shows this, and first, a multiplication circuit 42 is provided. This may be any of a variety of commercially available products, and takes the product of the output of a neuron in a certain layer, the error signal created by the circuit described above, and a constant η. The output of this multiplier circuit 42 is the output of the adder circuit 4
3 and uses the delay circuit 44 to generate a new T from T and ΔT. Therefore, the output from the adder circuit 43 corresponds to equation (7). [0040] FIG. 1 shows an example of a neuron imitation circuit 45 that is configured by combining these circuits. That is, this circuit 4 of FIG.
5 corresponds to the portion surrounded by a broken line in FIG. 33 on the network. Block B1 corresponds to the circuit shown in FIG. 3, and its calculation output is sent to each neuron immediately ahead. Further, block B2 is compatible with the error signal generation circuit 39 shown in FIG. That is, block B2 of the next layer
-1 and block B2-2 of the neuron shown in this figure.
The configuration is exactly the same as that in FIG. 10, and similarly, the block B2-1 of the neuron and the previous block B2-2
The configuration is exactly the same as that shown in FIG. Since the entire network has a multilayer structure as shown in FIG. 33, it is equivalent to divide the block of the error signal generation circuit 39 into two by cutting it in the middle. Also, block B5-1. B5-2. ~
．． B3-N corresponds to the coupling coefficient generation circuit 41 and A/D converter 23 shown in FIG. 11 (note that the delay circuit 44 is not shown in FIG. 1). This block B5-1.
B5-2. ~B3-N newly found coupling coefficient T
is used to change each coupling coefficient in the coupling coefficient variable circuit 20 shown in FIG. The same coupling coefficient is applied to blocks Bl and B2
-1 is used in two locations, so the two are linked and variable as shown in FIG. That is, in FIG. 1, block B2-1. B3
The circuit and the coupling coefficient variable circuit 20 portion in block B1 correspond to a self-learning circuit, and the remaining portion of block B1 and the portion of block B2-2 correspond to a neuron imitation element. [0041] The blocks configured as shown in FIG. 1 are connected in a net shape as shown in FIG. 33 to form a network, and further, an error signal generation circuit 36 as shown in FIG. 9 is installed in the output part of the final output layer.
By attaching , a neural network can be realized. [0042] The above circuit configuration will be explained using a specific example. First, the adder circuits and the like of each block described above are all constructed using commercially available general-purpose operational amplifiers, and a neuron imitation circuit 45 as shown in FIG. 1 with 256 inputs and 256 outputs and a coupling coefficient generation circuit 41 as shown in FIG. Many were made. Next, the input and output lines of each of these circuits 45 and 41 were connected to form a network. The network configuration was a three-layer structure, with the first layer consisting of 256 neuron imitation circuit units, the second layer consisting of four units, and the third layer consisting of five neuron imitation circuit units. Furthermore, the output of the third layer was connected to an error signal generation circuit 36 as shown in FIG. When some input is given to each unit in the first layer of such a network, the output result will not necessarily be the desired one at first, but as it has a self-learning circuit, the output result will eventually become the desired one. , that is, it becomes a teacher signal. [0043] An example in which this network is applied to a self-learning character recognition device will be described. First, handwritten characters as shown in FIG. 12 were read with a scanner, divided into 16×16 meshes, and each mesh was input to each neuron in the first layer of the network. A mesh with a text part was used as an input IV, and a mesh without a text part was used as an OV input. The output was connected to a voltmeter and the output results were displayed directly. The recognition result will be the position of the one with the sharpest output among the five units, so when you input a number from ``1'' to ``5'', the output of the number corresponding to that number will be the sharpest. I made them learn like this. The recognition rate was 100% for characters that had been sufficiently learned. [0044] Next, a second embodiment of the present invention will be described with reference to FIGS. 13 to 17.
This is explained by: This embodiment is designed to be digitalized, and basically all input/output signals, intermediate signals, coupling coefficients, teacher signals, etc. related to the neuron unit are rO
It is expressed as a pulse train expressed as a binary value of J Ill. ■ The amount of signals inside the network is expressed by pulse density (the number of "1"s within a certain period of time). 1N Kaihei 1-111185 (18) ■ Calculations within a neuron unit are expressed by logical operations between pulse trains. ■ Place the coupling coefficient pulse train in memory. ■Learning is achieved by rewriting this pulse train. ■ Regarding learning, the error is calculated based on the given teacher signal pulse train, and the coupling coefficient pulse train is changed based on this. At this time, the calculation of the error and the calculation of the change in the coupling coefficient are all performed by logical operations on the pulse train of "0" and "1". This is how it was done. [0045] Hereinafter, an explanation will be given based on an example that embodies this idea. First, the configuration of the signal calculation section will be explained with reference to FIG. FIG. 13 shows a portion corresponding to one neuron imitation circuit 50, and the network configuration is hierarchical as in the case of FIG. 33. All inputs and outputs are ``1'' and 2 to ``O''.
Valued and synchronized values are used. The intensity of the signal of input y is expressed by the pulse density, and is expressed by the number of states of "1" within a certain period of time, for example, as in the pulse train shown below. That is,

[Mathematical 2] Input signal n = 4/6 ・・・・・・・・・(
8) The synchronization signal is L. The example is an expression representing 4/6, and the signal has four "1"s and two "0"s in six synchronization pulses. At this time,"
1'' and rOJ are preferably arranged randomly as described later. [0046] On the other hand, the coupling coefficient Tij is similarly expressed in terms of pulse density and prepared in advance as a column on the memory. Pulse of “0” and “1yo” [0047]

[Formula 3] Coupling coefficient LL unit 3/6 Synchronizing signal LLL12'' - (9) An example of this is an expression that expresses ``l0IOIOJ = 3/6. In this case, ``1 ” and 10” are preferably arranged randomly. The specific method for determining this will be described later. [0048] Then, this pulse train is sequentially read out from the memory in accordance with the synchronization clock, and as shown in FIG.
T.). This is taken as an input to neuron j. To explain in the case above, 1 1J When the input signal is input as "101101", a pulse train is called from the memory in synchronization with this, and the AN
By taking D,

[Equation 4] Input signal LLLL=4/6 Coupling coefficient m=3/6 y, n T,, LL-1=2/6 -----
------- "101000" as shown in (10) is obtained, which means that the input y is the coupling coefficient T3. According to 1,
This shows that the pulse density is 2/6 after IJ conversion. [0049] The pulse density of the output of the AND gate 51 is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the coupling coefficient of the analog system. The longer the signal string is, and the more random the arrangement of "1" and rOJ, the closer the function will be to a product. Note that the pulse train of the coupling coefficient is shorter than the high-power pulse train, and when there is no more data to be read, it is sufficient to return to the beginning of the data and repeat the reading. [00501] Since one neuron unit has multiple inputs, there are many ANDJs between input signals and coupling coefficients as described above, and then the OR circuit 52, which is a logic circuit, calculates the OR of these. The inputs are synchronized, so for example, the first data is [101000] and the second data is [010000].
．． In the case of J, if we take the OR of both, we get “111N Kaihei 4-
111185 (20) becomes 1000j. This is calculated simultaneously for multiple inputs and output. That is,

[Equation 5] nT, 1 y+ n TlJ 4 . This corresponds to the sum calculation and nonlinear function (sigmoid function) part in analog calculation. [0051] When the pulse density is low, the ORed pulse density approximately matches the sum of the respective pulse densities. As the pulse density increases, the output of the OR circuit 52 gradually becomes saturated, so it does not match the sum of the pulse densities and nonlinearity appears. In the case of OR, the pulse density is neither greater than 1 nor less than 0, and is a monotonically increasing function, approximately similar to a sigmoid function. [0052] By the way, coupling has excitatory and inhibitory properties, and in the case of numerical calculations, it is expressed by the sign of the coupling coefficient, and in the case of analog circuits, as mentioned above, when To is negative (inhibitory coupling J ) uses an amplifier to invert the output and convert it to T1. J is connected to other neurons with a resistance value corresponding to J. In this regard, in this embodiment of the digital system, first, T1. Divide each connection into two groups, excitatory connections and inhibitory connections, according to the sign of J, and then [A of the pulse train of input signal and coupling coefficient]
The OR between NDJs is calculated for each group. Then, when only the output of the excitatory connection group is "1", it outputs "1", and only the output of the inhibitory connection group is "1".
1, outputs rOJ. When both are "1" or "0", either "1" or "0" may be output, or "1" may be output with a probability of about 1/2. In this example, the output is “1” only when the output of the excitatory connection group is “1” and the output of the inhibitory connection group is “0”.
”. To achieve this functionality,
It is sufficient to AND the (N0T of the output of the inhibitory connection group) and (the output of the excitatory connection group). That is,

[Equation 6] Excitatory connection group output m Inhibitory connection group output -U output shi-] - (12). [0053] Expressed as a logical formula,

[Formula 7] a=U (yln Tlt) (T='A stimulatory)
----------...(13) b=U (yln T
, +) (T=inhibitory) −・−・−・・(1
4) yJ=a n b
It is shown by (15). The network of neuron units is of a hierarchical type similar to pack propagation (ie, FIG. 33). If the entire network is synchronized, each layer can be calculated using the functions described above. [0054] On the other hand, each connection is divided into two groups j, excitatory connections and inhibitory connections, depending on the sign of To, and then [A of the pulse train of input signal and coupling coefficient]
Calculate the OR between NDJs for each group, and then
If you want to output an output other than when the output of the excitatory connection group is "0" and the output of the inhibitory connection group is "1", (N0T of the output of the inhibitory connection group) and (excitatory connection group output)

[Math. 8] excitatory connections group output inhibitory binding group output Mouth-IL becomes. [0055] Expressed as a logical formula,

[Formula 9] a=U(ylnTl,) (T=excitability) ・-・
・−・−(13′)b=tJ (yr n T, j>
(T=inhibitory) , , , -(14')y,=
:aUb・・・・・・・・・
...(Is'). [0056] Next, processing during learning will be described. a. Error signal in the final layer The error signal in each neuron is calculated in the final layer, and the coupling coefficient related to that neuron is changed based on it. For this purpose, a method for calculating the error signal will be described. In this embodiment, the "error signal" is defined as follows. Errors can be expressed numerically, generally both + and - can be expressed, but in the case of pulse density, it is not possible to express both positive and negative at the same time, so a signal representing the ten commandments and a signal representing the - component are used. The error signal is expressed using two types. That is, the error signal of the fourth neuron is

[Formula 10] Output result teacher signal error signal 1 δr D) = (y; low 1y ``upper n dj −L■− ・・・・・・・・・(16) ・・・・・・・・・(17) EXOR tun) AND d, ...... (1B)
It is indicated by. In other words, the difference between the ten-component teacher signal pulse of the error signal and the output pulse is (1,0) or (0
, 1), the pulses exist on the teacher signal side, and the pulses exist on the output side similarly to the other component. In other words,
The error signal 10 pulses are added to the output pulse, and the error signal -
Removing the pulse will result in a teacher pulse. Based on such an error signal pulse, the coupling coefficient is changed as described later. [0057] b. Error Signal in Intermediate Layer Furthermore, the error signal is back-propagated, and not only the coupling coefficient between the final layer and the previous layer but also the coupling coefficient of the previous layer changes. Therefore, it is necessary to calculate the error signal at each neuron in the hidden layer. In the same way as propagating a signal from a neuron in the middle layer to each neuron in the next layer, we collect the error signals in each neuron in the next layer and calculate the own error. Signal. This means that calculation formulas (7) to (
This can be done in the same way as 10). That is, first, the connections are divided into two groups depending on whether they are excitatory or inhibitory, and are expressed as a flat part AND and a Σ part ○R. However, the difference from equations (7) to (10) within a neuron unit is that y is one signal, whereas δ has two signals representing positive and negative, and both signals need to be considered. Therefore, it is necessary to divide into four cases depending on the sign of T and the sign of δ. [0058] First, the case of excitatory connections will be explained. In this case, let A be the coupling coefficient between the error signal in the @th neuron in the next layer and the self (jth). That is,

[Formula 11] δ, (+) T nok ”−L−. [0059] Also, by ANDing the next error signal - and the coupling coefficient, and then ORing them, similarly,
Let the error signal of this layer be -. That is,

[Formula 12] δbar) n jk [-111. [0060] Next, the case of inhibitory binding will be explained. In this case, the error signal in the neuron in the next layer ahead is ANDed with the coupling coefficient between that neuron and itself, and then the O
Take R. Let this be the error signal element for this layer. That is,

[Formula 13] δ, (-) J l-Tori-L δb(-)nTu= J-1-Tori-1 δ1 (+) 口''±-LL ・・・・・・
...(22). [0061] Also, by ANDing the next error signal 1 and the coupling coefficient, and then ORing them, similarly,
Let the error signal of this layer be -. That is,

[Formula 14] δA(”) n Tzzj −口− δb(+) n−rJ, −111[− δ, (−) −One dow−・・・・・・・・・・
(23). [0062] Since there are excitatory connections from one neuron to another neurons and inhibitory connections, the error signal δ, (+) obtained by equation (20) and (2
2) OR with the error signal δ, (+) obtained using the equation, and use it as the error signal δj(+) of the own neuron. Similarly, the error signal δj(-) obtained by equation (21) and (23
) is ORed with the error signal δ·(−) obtained by the equation, and this is used as the error signal δ,(−) of the own neuron. j
J[0063] To summarize the above,

[Formula 1.5] 1G excitatory iE inhibitory. [0064] Furthermore, a function corresponding to a learning rate (learning constant) may be provided. When the rate in numerical calculation is less than 1, the learning ability further increases. This can be achieved by thinning out the pulse train in pulse train calculations. This was thought of as a counter, as shown in the following examples 1) and 2). For example, when η=0.5, every other pulse train of the original signal is thinned out. Even if the pulses of the original signal are not equally spaced, the original pulse train 'It l;l Hei 4-111185 (27)

00651 [Equation 16] Example 1) In the case of [upper"-["-η20.5 (pulses are thinned out every other pulse, m-"-one-one-cho-one-one"-force =: 0.33) Case (Leave every two pulses) LL-ten"-1"-"-force = 0.67 (Example 2 where pulses are thinned out every two) LL-111"-""-Original Signal [-L-1] - When katana = 0,5 (thinning out every other pulse) When m-η = 0.33 (leaving every second pulse) LL-11111 "-η: o, in the case of 67 (thin out every two pulses) [0066] In this way, by thinning out the error signal, a learning rate function is provided. Such thinning of the error signal is as follows. This can be easily realized by performing logical operations on the output of a commercially available counter or by using a flip-flop.In particular, when a counter is used, the value of the learning constant η can be arbitrarily and easily set. (28) It is also possible to control the characteristics of the network. [0067] By the way, it is not necessary to always multiply the error signal by a learning constant. In addition, the learning constant used when propagating the error signal backwards may be different from the learning constant used in the calculation for calculating the coupling coefficient. This means that the characteristics of each neuron unit can be set individually, making it possible to construct an extremely versatile system. Therefore, it is possible to adjust the performance of the network as appropriate. [0068] Each connection is determined from the C0 error signal. Changing the coefficient The error signal is obtained by the method described above, and the method of changing each coupling coefficient will be explained.The error signal is ANDed with the signal flowing through the line to which the coupling coefficient to be changed belongs (δny ).However, in this embodiment, there are two error signals (10-), so each is calculated separately. [0069]

[Formula 17] δ, (+) low U y・1−[−m− ”; (”)ny r ts T
r; D)・”””・(26)δ, (-)
-Sat [-Satichi [0070] The two signals obtained in this way are ΔT, (+),
Let ΔT, , (-) be. Month IJ [0071] Next, a new T, - is calculated based on this ΔT -, but To in this example is 1]
1 piece
Since it is an IJ absolute value component, the original T1. Cases are differentiated depending on whether it is excitatory or inhibitory. excitatory j i.e.

[Math. 18] Original T i j ΔTiJ(+) ΔT1j(-) L top”-L ”-11- 111-L

[Math. 19] △T I J (+) L-1111'' --
11111ΔT, j(−) −11−L−L. [0072] The network is calculated based on the above learning rule. [0073] Next, an actual circuit configuration based on the above algorithm will be explained. Examples of the circuits are shown in FIGS. 14 to 16. The entire network is the same as that in FIG. 33. FIG. 14 shows a circuit corresponding to the lines (connections) in FIG.
5 shows the circuit of the part corresponding to the circle (neuron unit 1) in FIG. Further, FIG. 16 shows a circuit for calculating an error signal in the final layer from the output of the final layer and the teacher signal. A self-learning digital neural network can be realized by forming the configurations of FIGS. 14 to 16 into a network as shown in FIG. 33. [0074] First, FIG. 14 will be explained. In the figure, 55 is an input signal to the neuron, which corresponds to equation (8). The value of the coupling coefficient in equation (9) is stored in the shift register 56. This shift register 56 has an outlet 56a and an inlet 56b, but it may be of any type having the same function as a normal shift register, for example, it may be a combination of a RAM and an address controller. Input signal 5
5 and the coupling coefficient in the shift register 56 are ANDed by a logic circuit 58 equipped with an AND gate 57 and corresponding to equation (10). The output of this logic circuit 58 must be divided into groups depending on whether the connections are excitatory or inhibitory.It is better to prepare outputs 59 and 60 for each group in advance and to switch which one to output. It becomes highly versatile. Therefore, in this embodiment, the bit indicating whether the connection is excitatory or inhibitory is stored in the grouping memory 61.
The information is stored in the switching gate circuit 6.
Switch by 2. The switching gate circuit 62 has two AN
D game) 62a, 62b and an inverter 62c interposed at one input. [0075] Furthermore, as shown in FIG. 15, a gate circuit 63a having a plurality of OR gate configurations corresponding to equation (11) processes each input,
63b is provided. Furthermore, as shown in the figure (
12) AND that outputs “1” only when the excitatory connection group in the expression is “1” and the inhibitory connection group is “0”
Gate circuit 6 including gate 64a and inverter 64b
4 are provided. [0076] Next, the error signal will be explained. The error signal in the final layer is generated by a logic circuit 65 based on a combination of AND and exclusive OR shown in FIG. 16, which corresponds to equations (16) to (19). That is, the output 66 from the final layer and the teacher signal 6
7 to produce an error signal of 68°69. Equations (20) to (23) for calculating the error signal in the intermediate layer are:
This is performed by a gate circuit 72 having an AND gate configuration shown in FIG. 14, and an output cuff of 3.74 is obtained depending on + and -. In this way, it is necessary to differentiate the cases depending on whether the connection is excitatory or inhibitory.
) information and the error signal + and - signals 75.76,
This is performed by a gate circuit 77 having an AND and OR gate configuration. Further, calculation formula (24) for collecting error signals is performed by a gate circuit 78 having an OR gate configuration shown in FIG. Furthermore, equation (25) corresponding to the learning rate is performed by a frequency dividing circuit 79 shown in the figure. Finally, the part that calculates new coupling coefficients from the error signal, that is, (26) to (2
9) The part corresponding to equation 9 is performed by a gate circuit 80 having an AND, inverter, and OR gate configuration shown in FIG. 14, and the contents of the shift register 56, that is, the value of the coupling coefficient is rewritten. This gate circuit 80 also needs to be differentiated depending on the excitatory and inhibitory nature of the connection, but the gate circuit 77
This is done by [007,7] Here, the grouping method and output determination method shown in FIGS. 14 and 15 are extracted and shown as shown in FIG. 17. That is, this corresponds to the inventions set forth in claims 6 and 11, and the shift registers 56ij as memories storing coupling coefficients are individually provided for each input 55ij instead of being divided into groups at the input stage. Memory 61 for grouping logical results by AND game) 57ij
is divided into two groups via a switching circuit 62 according to the content of
A logical sum is calculated on the side of the OR gate 63b, and if it is an inhibitory combination group, a logical sum is calculated on the OR gate 63b side. After this, gate circuit 6
The output is determined by logical product processing using 4. [0078] A specific example will now be described using the self-learning character recognition device using the network described above. First, handwritten characters were read with a scanner and divided into 16×16 meshes, and meshes with text portions were designated as “1” and meshes without character portions were designated as “10” (same as in the case of FIG. 12). This data (
256 units) were input into the network, and the network was trained so that the position of the unit with the sharpest output among the five units would be the recognition result. Next, the structure of the network consists of 256 neuron units in the first layer, 20 neuron units in the second layer, and 5 neuron units in the third layer. At this time, unconnected input sections are connected to ground. If each coupling coefficient is initially set at random, the output result will not necessarily be the desired one. Therefore, we added the self-learning function of this circuit to 78
Each coupling coefficient is newly obtained using the equation 4-111185 (32), and this is repeated several times to obtain the desired result. In this embodiment, since the input is "0" or "1", the input pulse train is always a simple L level or H level. Further, the output was connected to an LED via a transistor, and the light was turned off when it was at L level and turned on when it was at H level. Since the synchronization clock was set to 1000kHz, the brightness of the LED changes to the human eye depending on the pulse density, so the brightest LED part becomes the answer. A recognition rate of 100% was obtained for characters that had been sufficiently learned. [0079] Note that the method of grouping excitatory connections and inhibitory connections may be configured as shown in FIG. 18, for example. This corresponds to the invention described in claim 10,
At the input stage, the groups are divided in advance into excitatory connection group a and inhibitory connection group b, and each input 55ij
A shift register 81 as a memory of at least 2 bits or more is provided for storing the coupling coefficient Tij. After that, OR Age-63a, 63 for each group
b etc. are similarly processed. [0080] Regarding the gate circuit 64, as shown in FIG.
An OR gate 64c may be used instead of the ND gate 64a to perform the logical sum processing. This corresponds to the invention set forth in claim 7, and is the processing of equations (12') to (15') described above. [0081] Next, a third embodiment of the present invention will be described with reference to FIG. 20. This embodiment corresponds to the fourteenth aspect of the invention, and is provided with learning constant setting means 82 for arbitrarily variably setting the learning constant used in the variable coupling coefficient circuit from the outside. That is, in addition to the basic idea shown in (1) to (2) in the above embodiment, the learning constant (learning rate) used during learning shown in (2) is made variable to obtain a network circuit with performance suitable for the application. It is designed to add the following functions. [0082] That is, this learning constant setting means 82 is provided in place of the frequency dividing circuit 79 shown in FIG. OR gates 84 to 87 that process learning constants
and one AND gate 88. More specifically, the OR connected to the binary outputs A-D of the counter 83
A switch Sa provided on the input side of each of the gates 84 to 87
When ~Sd are all set to H level side, η=1.0, and when all switches Sa~Sd are set to L level side, η=1/1
It becomes 6. Therefore, if the number of switches on the H level side is IN, then η=(2 to the power of N)/16. Therefore, by using a switch (or an external signal in place of the switch), the learning constant can be set arbitrarily. Note that when the pulse density is used as the clock input of the counter 83, an AND gate 89 may be provided as appropriate for the input of the error signal). The learning constant setting means 82 is not limited to such a circuit configuration. In addition, by providing a plurality of such learning constant setting means 82 or appropriately controlling them using an external signal, the value of the learning constant used for calculation of the coupling coefficient and the value of the learning constant used for back propagation of the error signal can be set. It is also possible to make them different. [0083] Furthermore, a fourth embodiment of the present invention will be described with reference to FIGS. 21 to 23. This embodiment corresponds to the invention set forth in claim 8. That is, based on the basic idea shown in Items 1 to 2 in the second embodiment, 1) Prepare two types of coupling coefficients, excitatory and inhibitory, and calculate the calculation result for the input signal for each coupling. Decisions are made by majority vote based on the ratio of results using coefficients, increasing the flexibility of the network. It is designed to add the following functions. [00843] In other words, one neuron unit has two coupling coefficients, excitatory and inhibitory. The processing is done at the same rate as in the case of combinations. Here, processing in proportions refers to the number of cases in which the output result obtained using the excitatory coupling coefficient is "1" for multiple input signals that are calculated synchronously, and the number of cases in which the output result obtained using the excitatory coupling coefficient and the inhibitory coupling coefficient Compare the output result obtained using
1185 (34) This means that the neuron unit outputs "0" if it is true, and "1" otherwise. Alternatively, if the two are equal, "0" may be output. [0085] FIGS. 21 and 22 show examples of circuit configurations for this purpose. First, each input 55 is individually provided with a set of shift registers 90a and 90b as memories. These shift registers 90a, 90b have a data output port and a data population similarly to the shift register 56.One shift register 90a stores the excitatory coupling coefficient, and the other shift register 90b stores the excitatory coupling coefficient.
is a memory of the inhibitory coupling coefficient. Reading means (not shown) from these shift registers 90a, 90b
The contents sequentially read out are inputted together with the input 55 to the corresponding AND games 91a and 91b, and a logical product is taken. There are two types of such logical results, one in which the connection is excitatory and one in which the connection is inhibitory, but here, the results are input to the majority circuit 92 and the output is determined. That is, the shift register 90
In the calculation group using excitatory coupling coefficients based on a, the digital signals thereof are subjected to addition processing by the amplifier 93a, and similarly, in the calculation group using the inhibitory coupling coefficients based on the shift register 90b, their digital signals are subjected to addition processing by the amplifier 93b. The comparator 94 determines the magnitude of both by majority vote. Note that the majority circuit 92 is not limited to the illustrated example;
It may be a general majority voting circuit. [0086] Here, the grouping method shown in FIG. 21 is extracted and shown as shown in FIG. 23. That is, this corresponds to the invention set forth in claim 10, and a set of memories (shift registers) storing coupling coefficients of excitatory connections and inhibitory connections for each input is prepared, and the memory is roughly refined. The process up to calculating the logical product for each group is performed. [0087] In the example shown in FIG. 23, the majority circuit 92 is replaced by the circuit shown in FIG.
7 and FIG. 18, OR gates 63a, 63b and the following are shown for calculating the logical sum for each group. The gate circuit 64 in this case may also be configured as shown in FIG. 1-111185 (35) [0088] By the way, in this embodiment, one set of shift registers 90a is provided for each input 55. 90b, the coupling coefficients are also rewritten for each shift register 90a and 90b by a self-learning function. Therefore, as shown in FIG. 21, new coupling coefficients are calculated using + and - error signals (21) to (2).
A self-learning circuit 95 that processes equation 4) is provided, and is connected to the data input side of each shift register 90a, 90b. [0089] According to this embodiment, the connectivity of the neuron units is not limited to either excitatory or inhibitory, so the network is flexible and has versatility in actual applications. [0090] The frequency dividing circuit 79 in the case of FIG. 22 may also be replaced with the learning constant setting means 82 as shown in FIG. 20. [0091] Furthermore, the output determination method by the majority circuit 92 is not limited to the method having two memories (shift registers 90a, 90b) for each input as shown in FIG. The present invention can be similarly applied to those having the memory 56. That is, instead of the combination of FIGS. 14 and 15, a combination of FIGS. 14 and 22 may be used. [0092] Further, a fifth embodiment of the present invention will be described with reference to FIGS. 24 to 28. The second embodiment described above was conceived from a higher level concept than the neuron imitation element and its network (circuit network) composed of circuits (hereinafter referred to as neuron circuits) shown in FIGS. 13 to 23. In this case, even if it is desired to configure all of these circuits, the signal processing may be performed by software following the procedures described in (8) to (29). This embodiment is one such example. [0093] That is, in this embodiment, the functions of the neurons constituting the network are realized by software. First, in the case of a network as shown in FIG. 33, signal processing is performed by software in any neuron constituting this network. The number of neurons using software may be one or all, or may be determined for each layer forming the network. FIG. 24 shows the configuration of a neuron that does not perform signal processing using a neuron circuit. Here, the input/output device 101 is connected to a device that inputs/outputs signals to/from other neurons or networks using neuron circuits, and sends and receives signals. The memory 102 stores software and data that control the CPU 103, and signals are sent to the CPU 10.
Processed in 3. Although the signal processing procedure is as described above, it is shown in FIGS. 25 and 26 again. Figure 2
5 shows the algorithm in the forward process,
Such signal calculation processing is performed within a digital circuit or within a computer. FIG. 27 shows the context of the neurons during processing shown in FIG. 25. FIG. 26 shows an algorithm in the learning calculation process, and such signal calculation processing is performed within a digital circuit or a computer. FIG. 28 shows the context of the neurons during processing shown in FIG. 26. Software is created according to the procedures shown in FIGS. 25 and 26 and stored in the memory 102. Here, it is also possible to cause one of the neurons in FIG. 24 to function as a plurality of neurons using software. However, it is necessary to process the signal in a time-division manner. [0094] By adopting such a configuration, the network configuration can be changed simply by rewriting the memory 102 without changing the hardware, and a highly flexible and versatile network can be constructed. I can do it. [0095] Further, a sixth embodiment of the present invention will be described with reference to FIG. 29. In this embodiment, part of the functions of one neuron is executed by software. Figure 24
In the configuration shown in FIG. 25, by storing software based on the signal processing procedure shown in FIG. 25 in the memory 102, a neuron using software that can execute a forward process can be realized. In order to realize a neuron with a learning function, the input/output device 101 shown in FIG.
4 or the circuit q+open+4-1111 as shown in Figure 21
85 (37) should be added. In either case, the circuit portion shown in the right half of FIG. 15 and FIG. 16 is required. The circuit shown in FIG. 20 may be provided as appropriate. FIG. 29 shows a circuit for providing such a learning function as a learning circuit 104. [0096] According to this embodiment, as in the case of the fifth embodiment, it is possible to change the network configuration simply by changing the software, and it is possible to construct a highly flexible and versatile network. [0097] Also, from a practical point of view, ordinary electronic equipment has a CPU.
is often installed in advance, so it can be said that there is no need to newly provide the components shown in FIG. 24. Furthermore, if the system does not require a learning function, the amount of hardware can be significantly reduced. [0098] Further, a seventh embodiment of the present invention will be described with reference to FIG. In this embodiment, the learning process function is realized by software. With the configuration shown in FIG. 24, by storing software based on the signal processing procedure shown in FIG. 26 in the memory 102, a neuron using software that can execute a learning process can be realized. In order to realize a neuron with a forward process function, the input/output device 101 has the functions shown in FIGS.
The circuit shown in FIG. 14 or the circuit shown in FIG. 14 and FIG. 22 may be added. The circuit shown in FIG. 19 may be provided as appropriate. FIG. 30 shows a circuit provided with such a forward process function as a forward circuit 105. [0099] According to this embodiment, as in the case of the fifth and sixth embodiments, it is possible to change the network configuration simply by changing the software, and it is possible to construct a highly flexible and versatile network. . In particular, it becomes easier to respond to changes in learning rules. Furthermore, in this case as well, if attention is paid to the fact that a CPU is often pre-installed in ordinary electronic devices, it can be said that there is no need to newly provide the constituent elements as shown in FIG. 24. Furthermore, if the system does not require a learning function, the amount of hardware can be reduced to 1N, 1-111.
It can also be reduced to 185 (38) width. [0100] According to the embodiments using these software, since the signal processing method can be executed using only digital logic operations, the required software may be based on a low-level language, and the software can be executed at high speed. becomes. [0101] By the way, the neuron network structure may have a structure as shown in FIG. 31 or FIG. 32, for example, in addition to the one shown in FIG. FIG. 31 is a conceptual structural diagram corresponding to the invention according to claim 1 or claim 2. In order from the input side, the first aggregate weight 10, the intermediate aggregate weight 1
11. When final aggregate 112 is used (aggregates can be classified in this way even in Figure 33) Neuron unit 1 included in a certain aggregate (○ indicates each logical operation means)
indicates a state in which the neuron unit is not connected to all of the 1,000 neuron units included in other aggregates. In FIG. 33, all neuron units 1 in a certain aggregate transmit and receive signals to and from all neuron units in another aggregate. As shown in FIG. may wish to fully connect the neuron units 1 in each aggregate. [0102] FIG. 32 shows the first assembly 1 of the invention according to claim 1.
10 and the final assembly 112 have two layers of intermediate assembly 11
The network is configured as a four-layer structure using 3,114. Generally, any suitable number of intermediate aggregates may be provided. [0103] In addition, in each of these Figures 311, 32, and 33, the number of neuron units 1 included in each aggregate is four. As explained above, this is arbitrary, and the number of neuron units may be different for each aggregate. [0104]

【Effect of the invention】

Since the present invention is configured as described above, the functions of the neuronal network, including the self-learning function, can be performed in parallel on hardware, and the self-learning function can be exerted.
,- Unexamined Japanese Patent Publication No. 4-. 111185 (39) The processing speed can be significantly improved compared to calculations using serial processing in conventional computer simulations, and at this time, the digital circuit configuration ensures reliable operation. Since signal processing is used, there are no problems associated with analog systems, such as being affected by the temperature characteristics of the amplifier. [0105] In particular, if configured as in the invention according to claim 1 or 2,
It is also possible to realize the neuron functions with software, making it possible to change the network configuration just by changing the software without requiring any changes to the hardware, making it possible to build a highly flexible and versatile network. This is the result. [0106] In addition, there are two types of coupling, excitatory and inhibitory.As in the invention described in claims 6 to 8, after dividing into two groups depending on the positive and negative of the coupling coefficient and processing each group, ,
It may be a process of logical product or logical sum of one negative result and the other result, or a flexible process of determining by majority logic based on the ratio of both, and can therefore be realized by a digital circuit called a logic circuit. [0107] Further, according to the invention described in claim 12 or 13, the coupling coefficient is also prepared on the memory, and unlike the case of using a resistor or the like, it is rewritable and has versatility. [0108] Furthermore, according to the fourteenth aspect of the invention, since the learning coefficient is made variable, it can be made efficient and easy to use in accordance with the actual application environment.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing one neuron imitation circuit in a first embodiment of the present invention.

[Figure 2] FIG. 2 is a circuit diagram showing a coupling coefficient variable circuit.

FIG. 3 is a circuit diagram of a coupling coefficient multiplication and matching circuit using this variable circuit.

[Figure 4] FIG. 2 is a circuit diagram showing an example of the adding circuit.

[Figure 5] It is a graph showing first-order differential characteristics of a sigmoid function.

[Figure 6] FIG. 3 is a circuit diagram of an f' signal generation circuit.

[Figure 7] It is a graph showing the human output characteristics.

[Figure 8] 7 is a graph showing examples of different input/output characteristics.

[Figure 9] FIG. 3 is a circuit diagram of an error signal generation circuit.

[Figure 10] FIG. 3 is a circuit diagram of an error signal generation circuit.

[Figure 111 FIG. 2 is a circuit diagram of a coupling coefficient creation circuit. [Figure 12] It is an explanatory diagram.

FIG. 13 is a logic circuit diagram of a configuration example of a signal calculation section showing a second embodiment of the present invention.

[Figure 14] FIG. 2 is a logic circuit diagram showing a configuration example of each part.

[Figure 151 FIG. 2 is a logic circuit diagram showing a configuration example of each part. [Figure 16] FIG. 2 is a logic circuit diagram showing a configuration example of each part.

[Figure 17] FIG. 2 is a logic circuit diagram showing a configuration example of each part. [Figure 18] FIG. 7 is a logic circuit diagram showing a modified example.

[Figure 19] FIG. 7 is a logic circuit diagram showing a modified example.

[Figure 201 FIG. 3 is a circuit diagram showing a third embodiment of the present invention. [Figure 21] FIG. 3 is a circuit diagram showing a fourth embodiment of the present invention.

[Figure 22] It is a circuit diagram.

[Figure 23] It is a circuit diagram.

[Figure 24] FIG. 3 is a block diagram showing a fifth embodiment of the present invention.

FIG. 25 is a flowchart showing processing in a forward process.

FIG. 26 is a flowchart showing processing in a learning process.

[Figure 27] FIG. 2 is a schematic diagram showing the anteroposterior relationship of neurons.

[Figure 28] FIG. 2 is a schematic diagram showing the anteroposterior relationship of neurons.

[Figure 29] FIG. 3 is a block diagram showing a sixth embodiment of the present invention.

[Figure 30] FIG. 3 is a block diagram showing a seventh embodiment of the present invention.

[Figure 31] FIG. 7 is a conceptual diagram showing a modified example of the network structure.

[Figure 321 It is a conceptual diagram which shows the modification example with which a network structure differs. [Figure 33] It is a conceptual diagram of a neural network showing a conventional example.

[Figure 341 FIG. 2 is a conceptual diagram showing the configuration of one of the units. [Figure 35] It is a graph showing a sigmoid function.

[Figure 36] FIG. 2 is a specific circuit diagram of one unit.

[Figure 37] FIG. 2 is a block diagram showing an example of a digital configuration.

[Figure 38] It is a circuit diagram of a part.

FIG. 39 is another circuit diagram of a part thereof. [Explanation of symbols] ■Logic operation means 20 Coupling coefficient variable circuit 41.80.95 Coupling coefficient generation circuit 45.50
Neuronal mimicry circuit 51.52, 58.62-65, 72.77-8056
．． 81.90a, 90b Measuring device 61 Grouping memory 82 Learning constant setting means 92 Majority circuits 110 to 114 Aggregate digital logic circuit

[Document person]

drawing

[Figure 1]

[Figure 2]

[Figure 3]

[Figure 4] q’ Kaihei 4-111185 (44)

[Figure 5]

[Figure 6]

[Figure 7]

[Figure 8]

[Figure 9] ■

[Figure 10]

[Figure 11]

[Figure 12]

[Figure 13]

[Figure 14]

[Figure 15]

[Figure 16]

[Figure 17]

[Figure 18]

[Figure 19]

[Figure 20]

[Figure 21]

[Figure 22]

[Figure 23]

[Figure 24]

[Figure 25]

[Figure 26]

[Figure 27]

[Figure 28]

[Figure 29]

[Figure 30]

[Figure 31]

[Figure 32] 7N open-4''4-111185 (60)

[Figure 33]

[Figure 34]

[Figure 35] ”

[Figure 37]

[Figure 38]

[Figure 39] Intercostal +4-111185 (63)

Claims (14)

[Claims] Claim 1: In a hierarchical signal processing circuit network consisting of a plurality of aggregates each having a logic operation means, a final output signal output from the logic operation means is compared with a teacher signal corresponding to the logic operation means. a comparison output means for generating an error signal in this logic operation means, in which a signal present only in the teacher signal is a positive error signal and a signal present only in the final output signal is a negative error signal; an excitatory coupling coefficient signal and an inhibitory coupling coefficient signal representing the coupling state of a logical computing means in a certain aggregate that provides an output signal to the computing means forming the body with the computing means forming the other aggregate; and an error signal consisting of a positive error signal and a negative error signal in the arithmetic means constituting the other aggregate. A logic operation is performed based on the combined teacher signal of the above, the positive error signal, and the inhibitory coupling coefficient signal among the coupling coefficient signals and the negative error signal to determine the positive error in the logic operation means in the certain aggregate. of the coupling coefficient signals, an inhibitory coupling coefficient signal and the positive error signal in another aggregate, and an excitatory coupling coefficient signal and the negative error signal of the coupling coefficient signals. error signal generation means for generating a negative error signal in the logic operation means in the certain aggregate by performing a logic operation based on the error signal of the error signal; Based on the input signal, a positive error signal and a negative error signal in this logic operation means, and a coupling coefficient signal representing the coupling state with the operation means constituting the certain aggregate that provides the output signal to this logic operation means. A signal processing circuit network comprising coupling coefficient control means for controlling a coupling coefficient signal of the lever. 2. A first aggregate each having a logical operation means;
It consists of a final aggregate and an intermediate aggregate that receives an output signal from the first aggregate and supplies an output signal to the final aggregate, and includes a logic operation means in one aggregate in the aggregate and a logical operation means in another aggregate. A final output signal outputted from the final assembly when an input signal is given to the first assembly by transmitting and receiving signals to and from the logical operation means is compared with a specific teacher signal. , by controlling all the coupling coefficients between the logic operation means based on this comparison result, the final output signal from the logic operation means in the final aggregate obtained for the given input signal is In a hierarchical signal processing circuit network configured to converge to a teacher signal, the final output signal output from the logic operation means in the final aggregate is compared with the teacher signal corresponding to this logic operation means, and the teacher signal is Comparison output means for generating an error signal in this logical operation means, in which a signal present only in the final output signal is a positive error signal, and a signal present only in the final output signal is a negative error signal, and another aggregate is formed. At least one of an excitatory coupling coefficient signal and an inhibitory coupling coefficient signal representing a coupling state with the computing means constituting the other aggregate in the logical computing means in a certain aggregate that supplies the output signal to the computing means. Using a coupling coefficient signal consisting of a signal and an error signal consisting of a positive error signal and a negative error signal in the calculation means constituting the other aggregate, an excitatory coupling teacher signal is determined from among the coupling coefficient signals. and the positive error signal, and a suppressive coupling coefficient signal among the coupling coefficient signals and the negative error signal to perform a logical operation to obtain a positive error signal in the logic operation means in the certain aggregate. and generate an inhibitory coupling coefficient signal among the coupling coefficient signals and the positive error signal in another aggregate, and an excitatory coupling coefficient signal and the negative error signal among the coupling coefficient signals. error signal generation means for generating a negative error signal in the logical operation means in the certain aggregate by performing a logical operation based on the above, and all input signals input to the logical operation means constituting the other aggregate; This coupling coefficient is calculated based on the positive error signal and negative error signal in the logic operation means, and the coupling coefficient signal representing the coupling state with the operation means constituting the certain aggregate that provides the output signal to the logic operation means. A signal processing circuit network comprising coupling coefficient control means for controlling signals. 3. Claim 1, characterized in that all the signals including the input signal, the output signal, the final output signal, the coupling coefficient signal, the teacher signal, and the error signal are signals expressed by a pulse train and a pulse density. Or the signal processing circuit network described in 2. 4. A plurality of self-learning circuits each comprising a variable coupling coefficient circuit and a coupling coefficient generation circuit that generates a variable coupling coefficient value of the variable coupling coefficient circuit based on an error signal with respect to a teacher signal, attached to a neuron imitation element. A signal processing network characterized by connecting neuron imitation circuits in a network. 5. Claim 4, characterized in that the self-learning circuit and the neuron imitation element are formed by digital logic circuits.
The signal processing circuitry described. 6. A logic circuit that processes coupling coefficients divided into two groups, an excitatory coupling group and an inhibitory coupling group, and calculates the logical product of the negation of the result of one group and the result of the other group. Claim 5 characterized in that it has
The signal processing circuitry described. 7. A logic circuit that processes each coupling coefficient divided into two groups, an excitatory coupling group and an inhibitory coupling group, and calculates the negation of the result of one group and the logical sum of the result of the other group. Claim 5 characterized in that it has
The signal processing circuitry described. 8. The signal according to claim 5, further comprising a majority circuit that determines the output of the neuron mimicking element based on the ratio between the output result from the excitatory coupling coefficient group and the output result from the inhibitory coupling coefficient group. processing circuitry. 9. A neuron mimicking element that calculates an input represented by a pulse density and a coupling coefficient represented by a pulse density to obtain an output determined by a pulse density signal,
1. A signal processing circuit network comprising a plurality of neuron imitation circuits connected in a network, each of which is provided with a control means for controlling the coupling coefficient based on an error signal between its own output and a teacher signal. Claim 10: A claim characterized in that a memory storing coupling coefficients for each input is provided individually, and the inputs are divided into two groups in advance depending on whether they are excitatory coupling or inhibitory coupling. The signal processing circuit network according to item 9. 11. A group that separately provides a memory storing coupling coefficients for each input, stores information representing two types of coupling, and groups the inputs into two groups: excitatory coupling and inhibitory coupling. 10. The signal processing circuit network according to claim 9, further comprising a dividing memory. 12. The control means is a self-learning circuit that generates a variable coupling coefficient value through learning based on an error signal between its own output and a teacher signal and variably controls the coupling coefficient in the memory. The signal processing circuitry according to claim 10 or 11. 13. A memory that stores excitatory coupling coefficients and a memory that stores inhibitory coupling coefficients are separately provided for each input, and two memories, excitatory coupling and inhibitory coupling, are provided for each set of these memories. The read memory contents and input contents are divided into groups, and logical operations are performed on each group to select either excitatory or inhibitory output, and learning is performed based on the error signal between the own output and the teacher signal. 10. The signal processing circuitry according to claim 9, further comprising a self-learning circuit that generates variable coupling coefficient values and variably controls the coupling coefficients in each memory. 14. The signal processing circuit network according to claim 4, further comprising learning constant setting means for arbitrarily setting and varying the learning constant used in the variable coupling coefficient circuit from the outside.