JPH05233582A - Signal processor - Google Patents

Abstract

PURPOSE:To easily make a large-scale network by providing a substrate dividing method which puts restriction even on the number of mountable LSIs with restrictions on the size of the substrate, on which neuron imitation elements constituted of LSIs are mounted, but is efficient. CONSTITUTION:In the signal processor provided with plural neuron imitation elements 20 with learning function connected like a hierarchical net and provided with a control means controlling these neuron imitation elements 20, neuron imitation elements 20 are divisionally mounted on substrates 55a and 55b individually provided on every layers A2 and A3, thereby constituting a neuron computer system for which the number of layers can be arbitrarily increased by only adding similar substrates.

Description

Detailed Description of the Invention

[0001]

INDUSTRIAL APPLICABILITY The present invention is applicable to control of various movements such as recognition of images and voices, position control of robots, temperature control of air conditioners, orbit control of rockets, etc. The present invention relates to a signal processing device such as a neuro computer.

[0002]

2. Description of the Related Art The function of a nerve cell (neuron), which is a basic unit of information processing of a living body, is mimicked, and further, this "nerve cell mimicking element" (nerve cell unit) is connected to a network to process information in parallel. The aim is a so-called neural network. Although it is easy to perform character recognition, associative memory, motion control, etc. in a living body, there are many things that conventional Neumann computers cannot easily achieve. There have been many attempts to solve these problems by imitating a neural system of a living body, in particular, a function peculiar to the living body, that is, parallel processing, self-learning, etc. by a neural network.

First, a conventional neural network model will be described. FIG. 27 is a diagram showing a certain nerve cell unit A, and FIG. 28 is a network thereof. A ₁ , A ₂ , and A ₃ each represent a nerve cell unit. One neuronal cell unit is coupled to a number of other neuronal cell units and processes the signals received from them to produce an output. In the case of FIG. 28, the network is hierarchical, and the nerve cell unit A ₂ receives a signal from the nerve cell unit A _{1 in the} previous layer (left side) and the nerve cell unit A in the next layer (right side). Output to ₃ .

A more detailed description will be given. First, in the nerve cell unit A of FIG. 27, the degree of coupling between another nerve cell unit and its own unit is called a coupling coefficient, and the coupling coefficient of the i-th nerve cell unit and the j-th nerve cell unit. The coupling coefficient is generally represented by T _ij . For the coupling, excitatory coupling in which the larger the signal from the other unit (the unit that sends a signal to the own unit) is, the larger the own unit output is, and the larger the signal from the other unit is, the own unit output Is an inhibitory bond, where T _ij > 0 represents an excitatory bond, and T _ij <0 represents an inhibitory bond. Now, let's say that our nerve cell unit is the jth unit, and the output of the ith nerve cell unit is y.
_{If it is i} , T _ij y _{i obtained} by multiplying this by the coupling coefficient T _ij becomes the input to the own unit. As described above, since one nerve cell unit is connected to many nerve cell units, ΣT _ij y _{i, which} is the result of adding T _ij y _i for these units, is It becomes an input to the unit. This is called the internal potential and is represented by u _j .

[0005]

[Equation 1]

Next, a threshold value is added to this input (internal potential) to perform non-linear processing, and the result is output from the nerve cell unit. The function used at this time is called a nerve cell response function, and the sigmoid function as shown in equation (2) and FIG. 29 is used as the nonlinear function.

[0007]

[Equation 2]

When such a nerve cell unit is constructed in a network as shown in FIG. 28, each coupling coefficient T
By providing _ij and sequentially calculating equations (1) and (2), parallel processing of information becomes possible and a final output is obtained.

In such a hierarchical neural network, a desired neural network is constructed by performing learning such that the coupling coefficient T _ij is updated so that a desired result is output for a certain input. .. The most widely used such learning method is the error back-propagation method, so-called back-propagation method.

When a dedicated integrated circuit (hereinafter referred to as a neuro LSI) is used in hardwareizing such a neural network, the scale of the neural network can be increased. However, even with a neuro LSI, there is a limit to the number of neurons that can be mounted on one chip. Therefore, when a large-scale neural network is hardened, it is necessary to construct a system equipped with a plurality of neuro LSIs.

[0011]

[Problems to be Solved by the Invention] That is, JP-A-2-81.
According to Japanese Patent Publication No. 159, a signal line between processors or between LSIs is deleted. However, when the scale of the neural network is further increased, a neuro LS is used.
The number of neurons that can be mounted is limited by the degree of integration of I. As a result, the number of neuro LSIs that can be mounted on the board is also limited, and it is necessary to use a plurality of boards. However, conventionally, no mention is made of a mounting method in such a case.

[0012]

A signal processing apparatus comprising: a plurality of neural cell mimicking elements with learning functions connected and provided in a hierarchical network; and a control means for controlling these neural cell mimicking elements. In the invention of item 1,
The nerve cell mimetic element was divided and mounted on a substrate provided for each layer.

According to the second aspect of the present invention, the nerve cell mimicking element is divided and mounted on the substrate provided so as to traverse the layers.

According to the third aspect of the present invention, the nerve cell mimetic element is divided and mounted on the substrate provided for each layer and the substrate provided so as to cross the layers.

According to a fourth aspect of the present invention, in the first, second or third aspect of the invention, the input / output signals between the substrates are divided into at least two or more.

Further, in the invention described in claim 5, in the invention described in claim 2, a common signal line is provided in the wiring on all the substrates, and the connection between the signal line and each substrate is switched. Signal switching means was provided on each substrate.

[0017]

According to the first aspect of the present invention, the nerve cell mimicking element is divided and mounted on the substrate provided separately for each layer in the hierarchical network structure. Therefore, the number of layers can be arbitrarily increased by adding the substrate. A neuro computer system can be constructed.

According to the second aspect of the present invention, since the nerve cell mimicking element is separately mounted on the substrate provided separately so as to traverse the layers in the hierarchical network structure,
It is possible to construct a neural computer system in which a neural network can be configured with one board and the number of neurons can be arbitrarily increased by adding a board.

According to the invention of claim 3, claim 1
Since two types of substrates combining the inventions described in 2 are used, a neurocomputer system having a high degree of freedom can be constructed.

According to the fourth aspect of the invention, since the input / output signals between the substrates are divided into at least two or more, two types of paths are used for signal transmission.
It is also easy to construct a neural network with many wirings between boards.

According to the invention of claim 5, a common signal line is provided in the wiring on all the substrates, and
Since the signal switching means for switching the connection between the signal line and each substrate is provided on each substrate, it is possible to use the same structure as all the substrates, and to create by changing the wiring pattern for each substrate. You can avoid the hassle.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of the present invention will be described with reference to FIGS. The present invention provides a nerve cell unit (neuron element) and a neural network using a digital logic circuit having a self-learning function, for example, Japanese Patent Application Nos. 2-412448 and 3-29342.
As proposed by the applicant of the present invention as an issue, etc., it is premised that all signals are digital signals and that calculation in a neural network uses pulse density expression. The contents will be described with reference to FIGS. 3 to 22.

First, the nerve cell unit and the neural network of this embodiment are digitalized and implemented as hardware. The basic idea is to input / output signals, intermediate signals,
The coupling coefficient, the teacher signal, etc. are all represented by a binary pulse train of "0" and "1". The amount of signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed in the memory. Learning is realized by rewriting this pulse train. For learning, the error is calculated based on the given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1". Is being done.

This idea will be described below. First,
Regarding signal processing by a digital logic circuit, signal processing in the forward process will be described. FIG. 3 shows a portion corresponding to one nerve cell unit (nerve cell mimicking element) 20, and the entire neural network is of a hierarchical type as shown in FIG. 4, for example. Input and output are all
The one that is binarized into “1” and “0” and synchronized is used. The value (intensity) of the input signal y _i is expressed by a pulse density, and is expressed by the number of states of “1” within a certain fixed time as in the pulse train shown in FIG. 5, for example. That is, the example of FIG.
4/6, and the signal "1" is 4 in 6 sync pulses.
There are two "0" s. At this time, it is desirable that the arrangement of "1" and "0" is random.

On the other hand, the coupling coefficient T _ij indicating the degree of coupling between the nerve cell units 20 is similarly expressed by pulse density,
The pulse train of "0" and "1" is prepared in advance in the memory. The example of FIG. 6 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of "1" and "0" is random.

Then, this coupling coefficient pulse train is sequentially read from the memory in response to the synchronous clock, and AND gates 21 and the input signal pulse train are ANDed with each other as shown in FIG. 3 (y _i ∩ T _ij ). This is used as an input to the nerve cell j. In the case of the above example, the input signal is “10
When input as "1101", the pulse train is called from the memory in synchronism with this, and "101000" as shown in FIG. 7 is obtained by sequentially performing AND, which means that the input y _i is the coupling coefficient T _ij. It is shown that the pulse density becomes 2/6 by the conversion.

The pulse density of the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This is because the longer the signal train is, the more "1"
The more random the arrangement of "0" and "0", the closer to the product of numerical values it has. When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no data to be read, the head of the data may be returned to and the reading may be repeated.

Since one nerve cell unit 20 has multiple inputs, there are many "ANDs of the input signal and the coupling coefficient" described above, and the OR circuit 22 then takes the logical sum of these. Since the inputs are synchronized, for example, the first data is "101000" and the second data is "01000".
In the case of “0”, the OR of both is “111000”. If multiple inputs (m pieces) are calculated at the same time and they are output, it becomes as shown in FIG. 10, for example. This corresponds to the sum calculation and the non-linear function (sigmoid function) part in the analog calculation.

When the pulse density is low, the pulse density of its OR is approximately equal to the sum of the respective pulse densities. As the pulse density increases, the OR circuit 22
Since the output of is gradually saturated, it does not match the sum of pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, although the connection has excitability and inhibitory property, it is assumed in the present embodiment that both of the nerve cell units 20 can have both of them. First, depending on whether the coupling coefficient is the excitatory coupling coefficient T _{ij (+)} or the inhibitory coupling coefficient T _{ij (-)} , each coupling is divided into an excitatory coupling group and an inhibitory coupling group. Then, the OR of the AND outputs of the input signal and the pulse train of each coupling coefficient is calculated in each group. Then, "1" is output only when the OR result of the excitatory coupling group is "1" and the OR result of the inhibitory coupling group is "0", and "0" is output otherwise. This output is used as an output pulse of this nerve cell unit 20 (see FIG. 9).

When expressed by a logical expression, it is expressed by the following expressions (3) to (5).

[0032]

[Equation 3]

The network of nerve cell units 20 is
Hierarchical type similar to backpropagation (ie, Figure 4)
And If the entire network is synchronized, each layer can be calculated by the functions described above.

Next, the signal calculation processing in learning (back propagation) will be described. Basically, the error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method of c.

A. Error signal in the final layer In the final layer, the error signal in each nerve cell unit is calculated from the output signal and the teacher signal. Here, a desired output with respect to the input at that time is given by a pulse train as a teacher signal. Generally, if the error is expressed by a numerical value, it can take both positive and negative values, but since it cannot be expressed simultaneously by the pulse density, an error signal can be obtained by using two kinds of signals, one representing a + component and the other representing a − component. Express. That is, the error signal of the jth nerve cell unit is shown in FIG. That is, the + component of the error signal is a pulse existing on the teacher signal side in the portion (1, 0) or (0, 1) where the teacher signal pulse and the output pulse are different, while the − component is similarly output. It is a pulse that exists on the side. In other words, if the error signal + pulse is added to the output pulse and the error signal−pulse is removed, it becomes a teacher pulse. That is, when these positive and negative error signals δ _{j (+)} and δ _{j (-)} are expressed by logical expressions, the expressions (6) and (7) are obtained. The coupling coefficient is changed based on such an error signal pulse as described later.

[0036]

[Equation 4]

B. Error Signal in Intermediate Layer First, the above-mentioned error signal is back-propagated to change not only the coupling coefficient between the final layer and the layer immediately before it but also the coupling coefficient of the layer before that. Therefore, it is necessary to calculate the error signal in each nerve cell unit in the middle layer. Signals were propagated from the neuron unit with an intermediate layer to each neuron unit in the next layer, which is exactly the reverse of the procedure of collecting error signals in each neuron unit in the next layer. And use it as its own error signal. This means that the above-mentioned arithmetic expressions (3) to (5) in FIG.
~ It can be performed in the same manner as the case shown in FIG.

However, the difference from the above-mentioned processing in the nerve cell unit is that y is one signal, whereas
δ has two signals as a signal representing positive and negative, and it is necessary to consider both signals. Therefore, it is necessary to divide into two cases depending on whether the coupling coefficient T is positive or negative.

First, the case of excitatory coupling will be described. In this case, for a nerve cell unit with an intermediate layer, the error signal + at the jth nerve cell unit of the layer immediately before (final layer in FIG. 4), the nerve cell unit and self (intermediate layer in FIG. 4) For each nerve cell unit, an AND (δ _{j (+)} ∩ T _ij ) of the coupling coefficient with a certain nerve cell unit) is obtained, and the OR between them is taken {∪ (δ _{j (+)} ∩ T _ij )}. This is the error signal + of this nerve cell unit. That is, it becomes as shown in FIG.

Similarly, the error signal of the nerve cell unit of the previous layer is ANDed with the coupling coefficient, and the OR of these is taken to obtain the error signal of this nerve cell unit. That is, it becomes as shown in FIG.

Next, the case of inhibitory binding will be described. In this case, the AND of the error signal in the nerve cell unit of the layer one layer ahead and the coupling coefficient of the nerve cell unit and the self is taken, and further the OR between them is taken. This is the error signal + of this nerve cell unit. That is, it becomes as shown in FIG.

Further, by ANDing the preceding error signal + and the coupling coefficient, and further ORing them, the error signal − of this nerve cell unit is similarly obtained. That is, it becomes as shown in FIG.

Furthermore, the error signal + of the excitatory connection and the error signal + of the inhibitory connection of this nerve cell unit are ORed, and this is output to the nerve cell unit in the layer immediately before this unit. The error signal is Δ _{i (+)} . Similarly, the OR between the error signal of excitatory coupling and the error signal of inhibitory coupling
Is taken as the error signal δ _{i (-)} output to the nerve cell unit in the layer immediately before this unit.

The above is summarized as shown in equation (8).

[0045]

[Equation 5]

Further, the same or different learning rates (learning constants) may be provided for the input error signals. This can be realized by thinning out the pulse train. For example, considering a counter-like way of thinking, FIG.
It may be as shown in. In this example, while the learning rate η = 0.5, the pulse train of the original signal is thinned out every other pulse, but even if the pulses of the original signal are not evenly spaced, they can be thinned out with respect to the original pulse train. In FIGS. 15 and 16,
When η = 0.5, every other pulse is thinned out, and η =
In the case of 0.33, every other pulse is left and η = 0.6.
The case of 7 indicates that every other pulse is thinned once.
In this way, the function of the learning rate is provided by thinning out the error signal.

C. Updating of Coupling Coefficients Each coupling coefficient is changed by the error signal resulting from the above processing. The output y _i from the immediately preceding neuron unit and the error signal δ of its own neuron unit corresponding to the line to which the coupling coefficient to be changed belongs (see FIG. 4)
AND with _{j (+)} or δ _{j (-)} (δ _j ∩ y _i ) (Fig. 1
7, see FIG. 18). The two signals thus obtained are designated as ΔT _{ij (+)} and ΔT _{ij (-)} , respectively.

Then, based on this ΔT _ij , a new T
_ij is obtained. Since this T _ij is an absolute value component, it is classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the component of ΔT _{ij (+)} is increased with respect to the original T _ij , and ΔT
Reduce the components of _{ij (-)} . That is, it becomes as shown in FIG. On the contrary, in the case of the suppressive property, the component of ΔT _{ij (+)} is reduced and the component of ΔT _{ij (−)} is increased with respect to the original T _ij . That is, FIG.
As shown in 0.

The above can be summarized as in equation (9).

[0050]

[Equation 6]

The network is calculated based on the above learning rule.

Next, an actual circuit configuration based on the above algorithm will be described. 21 and 22 show the circuit configuration example, the configuration of the entire network 2 is the same as that of FIG. 21 shows a circuit of a portion corresponding to a line (connection) in FIG. 4, and FIG. 22 shows a circuit of a portion corresponding to a circle (each nerve cell unit 20) in FIG. These Figure 2
22. A self-learning digital neural network can be realized by forming the circuit of FIG. 1 and FIG. 22 into a network as shown in FIG.

First, FIG. 21 will be described. In the figure, 25 is an input signal to the nerve cell unit and corresponds to FIG. The value of the coupling coefficient as shown in FIG. 6 is stored in the shift register 26. The shift register 26 has an outlet 26a and an inlet 26b, but may have the same function as a normal shift register, for example, R
It may be a combination of an AM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are ANDed by a logic circuit 28 having an AND gate 27 and performing the processing shown in FIG. The output of the logic circuit 28 must be divided into groups depending on whether the coupling is excitatory or inhibitory. However, it is better to prepare the outputs 29 and 30 for each group in advance and switch which one is output. It is highly versatile. For this reason, in this embodiment, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31, and the information is used to switch by the switching gate circuit 32. The switching gate circuit 32 includes two AND gates 32a and 32b.
And an inverter 32c interposed at one input.

As shown in FIG. 22, gate circuits 33a and 33b having a plurality of OR gates for processing each input (corresponding to FIG. 8) are provided. Furthermore, as shown in the figure, the excitatory coupling group shown in FIG. 11 is “1”.
In addition, a gate circuit 34 including an AND gate 34a and an inverter 34b that outputs an output "1" only when the inhibitory coupling group is "0" is provided.

Next, the error signal will be described. Error signal 3 created from the output from the final layer and the teacher signal
8 and 39 are input to the circuit shown in FIG. The processing shown in FIGS. 11 to 14 for calculating the error signal in the intermediate layer is performed by the gate circuit 42 having the AND gate configuration shown in FIG.
44 is obtained. As described above, it is necessary to classify the connection depending on whether it is excitatory or inhibitory. In this case, the information on excitatory or inhibitory stored in the memory 31 and the +,-signals 45 and 46 of the error signal are used. The gate circuit 47 having AND and OR gate configurations is used. Further, the calculation formula (6) for collecting the error signals is performed by the gate circuit 48 having the OR gate structure shown in FIG. Further, the processing of FIGS. 17 and 18 corresponding to the learning rate is performed by the frequency dividing circuit 49 shown in FIG. Finally, the part for calculating a new coupling coefficient from the error signal, that is, the part corresponding to the processing of FIGS. 17 to 20, is AND, inverter, O shown in FIG.
This is performed by the gate circuit 50 having the R gate configuration, and the contents of the shift register 26, that is, the value of the coupling coefficient is rewritten. This gate circuit 50 also needs to be classified depending on the excitability and inhibitory property of the coupling, but the gate circuit 47 performs this.

A circuit that realizes the above operation, that is,
Neuro L is a collection of nerve cell units 20.
SI (neuronal cell mimicking element), which is basically incorporated into a system as shown in FIG. This system
A control board 51 as a control means configured of a CPU, a RAM, and a ROM for controlling the entire system, and an input random number generation board 5 for generating random numbers with respect to input data.
2. A teacher random number generation board 53 that generates a random number for teacher data and generates an error signal according to equations (6) and (7), an output board 54 that counts output pulse trains, and a neuro L
It is composed of a neuro board (substrate) 55 on which SI is mounted, and these are connected to each other by a parallel board 56.

The present embodiment is intended for a neural network having a hierarchical network structure as shown in FIG. 4, in which the nerve cell units (neuro LSI) 20 are not connected in the same layer and the layers are not connected. Are joined by. On this occasion,
Neural network does not necessarily have one neuro LS
I20 cannot be realized with one substrate 55, so it will be divided appropriately.

In this embodiment, the method of dividing the substrate 55 and the wiring are devised. For example, in the case of a neural network configuration including four input layers A ₁ , three intermediate layers A ₂ , and two output layers A ₃ , as shown in FIG. 1, the neuro LSI 20 belonging to the intermediate layer A ₂ Is mounted on the intermediate layer substrate 55a, and the neuro-LSI 20 belonging to the output layer A ₃ is mounted on the output layer substrate 55b. That is, the substrates 55a and 55b are provided in layer units. For the neurons belonging to the input layer A ₁ , the input data is simply output as it is.
It is not necessary to have a neuro LSI configuration.

It is assumed that the neuro LSI 20 shown in FIG. 1 is one chip and one neuron and has a sufficient number of synapses so that the above-mentioned pulse density logical operation can be performed. That is, when a plurality of input pulse trains are given, an output pulse train is obtained, and when an error signal is given, the error signal of the previous layer is output. Further, in the figure, an arrow → indicates an input / output (forward) connection, and an arrow ← indicates a learning connection. Further, as shown in the drawing, the signal lines are branched so that the connections between the substrates 55a and 55b are reduced as much as possible.
Designed to do on b.

In such a structure, first, the forward process operation will be described. The input binary signal is converted by the input random number generation board 52 into a signal of a pulse train in which the number of pulses is proportional to the binary signal. At this time, the position at which the pulse train is set is determined by the generation of random numbers. The random number may be generated according to, for example, an M-series random number. In this way, the input pulse train signal is input to the intermediate layer substrate 55a via the parallel board 56. Then, calculation is performed in the neuro LSI 20 mounted on the intermediate layer substrate 55a, and the output pulse signal is output to the parallel board 56. This pulse train is input to the output layer substrate 55b, and the output layer substrate 55b is input.
Arithmetic is performed in the neuro LSI 20 mounted on the top. This calculation result is also a pulse signal for the parallel board 5
6 is output onto the output board 54. The output board 54 converts the pulse density signal into a binary signal by a built-in counter and outputs the binary signal.

On the other hand, the operation during learning will be described. First, when a teacher signal based on a binary signal is given, the teacher random number generation board 53 converts the signal into a pulse train signal in which the number of pulses is proportional to the binary signal, and (6)
An error signal is generated according to the equation (7) and is input to the output layer substrate 55b via the parallel board 56, and the neuro LSI
The bond strength is updated within 20. This error signal is also a pulse train signal, and propagates in the opposite direction to the signal transmission of the forward process. That is, it propagates from the output layer substrate 55b side to the intermediate layer substrate 55a side, and the learning as described above is performed.

Therefore, according to this embodiment, the neuro LS is
The board 55 for mounting the I20 is a board 55 in units of layers.
Since it is provided as a and 55b, it is possible to construct a system in which the number of layers of the neural network can be arbitrarily increased by adding a substrate.

Next, an embodiment of the invention described in claim 2 will be described with reference to FIG. The same parts as those shown in the above-mentioned embodiment are designated by the same reference numerals. In this embodiment, for example,
Input layer A _1, the intermediate layer A _2, if the neural network configuration output layer A ₃ consisting of any of four neurons, as shown in FIG. 23, 2 belonging to the intermediate layer A ₂ and the output layer A ₃
Each neuro LSI 20 is mounted on the first substrate 55c, and the remaining two neuro LSIs 20 belonging to the intermediate layer A ₂ and the output layer A ₃ are mounted on the second substrate 55d. .. That is, the substrate 5 so as to traverse the layers
5c and 55d are provided. Since the neurons belonging to the input layer A ₁ only output the input data as they are, it is not necessary to have a neuro LSI configuration. Also in this case, the branching of the signal line is designed to be performed on each of the substrates 55c and 55d as much as possible so that the number of connections between the substrates 55c and 55d is reduced as much as possible.

In such a structure, first, the forward process operation will be described. Basically, it is similar to the case of the above-mentioned embodiment, but the input binary signal is converted by the input random number generation board 52 into a pulse train signal in which the number of pulses is proportional to the binary signal. In this way, the input pulse train signal is input to the first and second substrates 55c and 55d via the parallel board 56. Then, the computation is performed in the neuro LSI 20 of the intermediate layer A ₂ mounted on the first and second substrates 55c and 55d, and the output pulse signal is output to the other substrates 55d and 55c. Then, calculation is performed in the neuro LSI 20 of the output layer A ₃ mounted on the first and second substrates 55c and 55d, and the obtained output pulse signal is output to the parallel board 56. This pulse train is taken into the output board 54, and the pulse density signal is converted into a binary signal by the built-in counter and output.

On the other hand, the operation during learning will be described. First, when a teacher signal based on a binary signal is given, the teacher random number generation board 53 converts the signal into a pulse train signal in which the number of pulses is proportional to the binary signal, and (6)
An error signal is generated according to the equation (7), is input to the output layer substrates 55c and 55d via the parallel board 56, and the coupling strength is updated in the neuro LSI 20. This error signal is also a pulse train signal, which is propagated from the output layer A ₃ side to the intermediate layer A ₂ side, contrary to the signal transmission of the forward process, and the learning as described above is performed.

Therefore, according to this embodiment, the neuro LS is
Since the board 55 for mounting the I20 is provided as the boards 55c and 55d in a state of traversing the layers, it is possible to construct a system in which the number of neurons of the neural network can be arbitrarily increased by adding the boards. It is also possible to configure the neural network with only one board 55c or 55d.

In this embodiment, each of the substrates 55c, 55
Although the number of neurons on d is the same in the intermediate layer A ₂ and the output layer A ₃ , it does not necessarily have to be equal.

Next, an embodiment of the invention described in claim 3 will be described with reference to FIG. This embodiment is a combination of the above two embodiments. For example, when the standard system configuration is such that the number of neurons in the intermediate layer A ₂ and the number of neurons in the output layer A ₃ are both 2, When it is desired to increase the number, a system unit is constructed by adding a substrate of a layer unit according to the first aspect of the invention. The example shown in FIG. 24 shows a configuration in which the number of neurons in the intermediate layer A ₂ is set to be 5 with respect to the standard system configuration. The first substrate 55c of a type traversing layers and the intermediate layer in layer units are provided. Substrate 55a
It is configured by the combination with.

In such a configuration, first, the forward process operation will be described. Basically, it is similar to the case of the two embodiments described above, but the input binary signal is converted by the input random number generation board 52 into a signal of a pulse train in which the number of pulses is proportional to the binary signal. .. In this way, the input pulse train signal is input to each of the substrates 55c and 55a via the parallel board 56. Then, calculation is performed in the neuro LSI 20 of the intermediate layer A ₂ mounted on these substrates 55c and 55a, and these output pulse signals are sent to the neuro LSI 20 belonging to the output layer A ₃ of the substrate 55c side. These neuro LSIs
The calculation is performed in 20 and the obtained output pulse signal is output to the parallel board 56. This pulse train is taken into the output board 54, and the pulse density signal is converted into a binary signal by the built-in counter and output.

On the other hand, the operation during learning will be described. First, when a teacher signal based on a binary signal is given, the teacher random number generation board 53 converts the signal into a pulse train signal in which the number of pulses is proportional to the binary signal, and (6)
An error signal is generated according to the equation (7), is input to the output layer substrates 55c and 55d via the parallel board 56, and the coupling strength is updated in the neuro LSI 20 of the output layer A ₃ .
This error signal is also a pulse train signal, which is propagated from the output layer A ₃ side to the intermediate layer A ₂ (substrates 55a, 55c) side, contrary to the signal transmission of the forward process, and the learning described above is performed.

Therefore, according to this embodiment, the neuro LS is
Since the substrate 55 for mounting the I20 is a combination of a layer unit and a type traversing the layers, a neural network system having a large degree of freedom can be constructed. That is, the configuration is not limited to the combination of the illustrated examples, and may be a configuration of, for example, a combination of the substrate 55c and the output layer substrate 55b. Further, the substrate structure of the invention method described in claim 1 may be a standard structure, and the substrate structure of the invention method described in claim 2 may be added thereto.

Next, an embodiment of the invention described in claim 4 will be described with reference to FIG. This embodiment is a neuro LSI.
The signal transmission between the substrates 55 on which the 20 are mounted is devised. In the embodiment described above, the signal is parallel board 56.
The parallel board 56 may not be sufficient, considering that control signals and address signals are required as system signals. Therefore, in the present embodiment, the input and output signals are divided so that the output and the learning signal as the neural network can be directly transmitted between the substrates 55.

For example, in the example shown in FIG. 25, in a configuration using the substrates 55c, 55d and 55e according to the second aspect of the present invention, a learning signal is sent to each substrate 55 by using a cable 57.
c, 55d, and 55e are directly transmitted and received, and the parallel board 56 transmits and receives an output signal. Since the signal transmission is divided into two systems in this way, it can be easily configured even in the case of a neural network configuration having many wirings between substrates.

The cable 57 side may be used for the output signal and the parallel board 56 side may be used for the learning signal.

An embodiment of the invention described in claim 5 will be described with reference to FIG. This embodiment makes the implementation of the invention according to claim 2 more advantageous. That is, when the substrates 55c to 55e are provided so as to traverse the layers, the input / output signal and the learning signal for the neuron are 1:
Since it is necessary to correspond to No. 1, the wiring pattern must be changed for each of the substrates 55c to 55e, which is disadvantageous in production. Therefore, in this embodiment, each of the substrates 55c to 55e is
All the substrates 55c to 55e have a common structure with the same wiring pattern by adding common signal lines 58 and 59 to the respective substrates 55c to 55e.
The signal switching means provided on c to 55e, for example, switches 60 and 61, are arranged so that the connection to the own substrate can be arbitrarily selected.

Here, the signal line 62 of the parallel board 56 is
From the board to each of the boards 55c to 55e by a connector or the like. The wiring of the input signal to the output layer A ₃ and the wiring of the learning signal to the intermediate layer A ₂ are different for each of the boards 55c to 55e, so that the switch 60 selects the input signal necessary for the own board from the common signal line 58. Then, the switch 61 selects a learning signal required for the own board from the common signal line 59.

As a result, it is possible to provide a necessary network configuration while using a common structure for all the substrates 55c to 55e.

The switch 6 is used as the signal switching means.
It is not limited to 0 and 61, but may be a jumper line connecting only necessary lines or a logic circuit.

[0079]

Since the present invention is configured as described above, according to the invention of claim 1, the neural cell mimicking element is separately mounted on the substrate provided for each layer in the hierarchical network structure. Then, it becomes possible to construct a neuro computer system capable of arbitrarily increasing the number of layers by adding a substrate.

According to the second aspect of the present invention, the neural cell mimicking elements are separately mounted on the substrate provided separately so as to traverse the layers in the hierarchical network structure.
It becomes possible to construct a neural computer system which can configure a neural network with one board and can increase the number of neurons arbitrarily by adding a board.

According to the invention of claim 3, claim 1,
Since two types of substrates combining the inventions described in 2 are used, a neurocomputer system having a high degree of freedom can be constructed.

According to the invention described in claim 4, since the input / output signals between the substrates are divided into at least two and two kinds of paths are used for signal transmission, the wiring between the substrates is arranged. It is also possible to easily construct a neural network with a large number.

Further, according to the invention of claim 5, a common signal line is provided in the wiring on all the substrates, and
Since the signal switching means for switching the connection between the signal line and each substrate is provided on each substrate, it is possible to use the same structure as all the substrates, and to create by changing the wiring pattern for each substrate. You can avoid the hassle.

[Brief description of drawings]

FIG. 1 is a conceptual connection diagram showing an embodiment of the invention described in claim 1.

FIG. 2 is a block diagram showing the overall configuration of the system.

FIG. 3 is a logic circuit diagram for performing basic signal processing.

FIG. 4 is a schematic diagram showing a network configuration example.

FIG. 5 is a timing chart showing an example of logical operation.

FIG. 6 is a timing chart showing an example of logical operation.

FIG. 7 is a timing chart showing an example of logical operation.

FIG. 8 is a timing chart showing an example of logical operation.

FIG. 9 is a timing chart showing an example of logical operation.

FIG. 10 is a timing chart showing an example of logical operation.

FIG. 11 is a timing chart showing an example of logical operation.

FIG. 12 is a timing chart showing an example of logical operation.

FIG. 13 is a timing chart showing an example of logical operation.

FIG. 14 is a timing chart showing an example of logical operation.

FIG. 15 is a timing chart showing an example of logical operation.

FIG. 16 is a timing chart showing an example of logical operation.

FIG. 17 is a timing chart showing an example of logical operation.

FIG. 18 is a timing chart showing an example of logical operation.

FIG. 19 is a timing chart showing an example of logical operation.

FIG. 20 is a timing chart showing an example of logical operation.

FIG. 21 is a logic circuit diagram showing a configuration example of each unit.

FIG. 22 is a logic circuit diagram showing a configuration example of each unit.

FIG. 23 is a conceptual connection diagram showing an embodiment of the invention according to claim 2;

FIG. 24 is a conceptual connection diagram showing an embodiment of the invention according to claim 3;

FIG. 25 is a conceptual connection diagram showing an embodiment of the invention according to claim 4;

FIG. 26 is a conceptual connection diagram showing an embodiment of the invention according to claim 5;

FIG. 27 is a conceptual diagram showing one unit configuration showing a conventional example.

FIG. 28 is a conceptual diagram of the neural network configuration.

FIG. 29 is a graph showing a sigmoid function.

[Explanation of Codes] 20 Neuron Mimicking Element 51 Control Means 55 Substrate 58, 59 Common Signal Line 60, 61 Signal Switching Means

Claims (5)

[Claims] 1. A signal processing device comprising a plurality of neural cell mimicking elements with a learning function, which are connected in a hierarchical mesh, and which has a control means for controlling these neural cell mimicking elements. A signal processing device, characterized in that is mounted separately on a substrate provided for each layer. 2. A signal processing device comprising a plurality of neural cell mimicking elements with a learning function, which are connected in a hierarchical network and provided with control means for controlling these neural cell mimicking elements. A signal processing device, wherein the signal processing device is characterized in that the device is divided and mounted on a substrate provided so as to traverse layers. 3. A signal processing device comprising a plurality of neural cell mimicking elements with a learning function, which are connected in a hierarchical network and which has a control means for controlling these neural cell mimicking elements. A signal processing device, characterized in that the signal processing device is mounted separately on a substrate provided for each layer and a substrate provided so as to cross the layers. 4. The signal processing apparatus according to claim 1, wherein an input / output signal between each substrate is divided into at least two or more. 5. A common signal line is provided in the wiring on all the substrates, and a signal switching means for switching the connection between the signal line and each substrate is provided on each substrate. Item 2. The signal processing device according to item 2.