JP5393589B2 - Electronic circuit - Google Patents

Description

  The present invention relates to an electronic circuit.

  An electronic circuit that mimics the structure and function of the brain is called a neural network or neurocomputer, and many artificial neurons (hereinafter referred to as neurons, unless there is a risk of confusion) are connected in parallel. It is a computer that operates. Hereinafter, an electronic circuit imitating the structure and function of the brain or a software simulator is referred to as a neural network.

  The Neumann computer is suitable for sequential processing using a fixed algorithm. On the other hand, since neural networks can mimic the characteristics of the brain, it is thought that they can be used for applications where it is difficult to accurately describe their control algorithms, such as metal welding control and large-scale plant operation control. . It is also considered that it can be used for parallel processing (for example, image recognition, voice recognition, etc.) of a large amount of data that is ambiguous and has a large amount of loss or fluctuation. If a neural network with the same scale and function as a human brain can be realized, not only robots and cars can be greatly enhanced, but also an interface function with high compatibility with humans can be added to all machines, and past data can be used. It is expected to be able to predict the future.

  There are two main methods for realizing a neural network. One is a method of building a simulator by software on a normal Neumann computer. In this method, a precise model that makes use of knowledge obtained from brain research can be created, and the neural network can be operated while observing or controlling the internal states of the neurons and the neural network. However, since it is a simulation method using a normal computer, it is difficult to move a large-scale (10,000 neurons or more) neural network in real time (all neurons calculate in about 4 ms). It is. In addition, the advantage of small size / low power consumption, which is a feature of the neural network, cannot be enjoyed. Therefore, this method is suitable when a small-scale neural network is mainly used for research purposes.

  The other realization method is a method in which a neuron is modeled by an electronic circuit and is implemented as hardware in which a large number of the neurons are mounted. In this method, there is a possibility that a large number (about 10,000) neurons can be operated in parallel in real time.

As an example of a conventional neural network, a part of the neural network will be described with reference to FIG. FIG. 15 is a schematic diagram showing a part of a neural network. The L1 layer is a data input layer. For example, neurons 210 _i (i is an integer from 1 to 1,000) for converting 4-bit input data into a pulse train having a period of 4 [ms] are represented by 1, There are 000. The L2 layer is a data processing layer, and there are, for example, 900 neurons 212 _j (j is an integer from 1 to 900) that performs arithmetic processing on data.

Each neuron 210 _i has one output signal line 211 _i . Since the number of neurons 212 _j is 900, the output signal line 211 _i branches to a maximum of 900 and is connected to the neuron 212 _j via the synapse connection 213 _{i, j} . That is, a maximum of 1,000 synapse connections 213 _{i, j} exist in each neuron 212 _j . Data a _i (output of the i-th neuron 210 _{i in} the L1 layer) transmitted through each signal line 211 _i is a 1-bit signal.

Each synapse connection 213 _{i, j} holds, for example, 8-bit width weight data w _{i, j} (the weight of the synapse connection of the _jth neuron 212 _{j in} the L2 layer that receives a signal from the neuron 210 _{i in the} L1 layer). and which calculates the product _{a i} × _{w i, j} and _{a i.} Each neuron 212 _j calculates the sum of a maximum of 1,000 products a _i × w _{i, j} calculated by the synaptic connections attached to each neuron 212 _j . That is, the sum is calculated for i of the products a _i × w _{i, j} .

When the total value exceeds a predetermined threshold value θ, the output signal line 214 _{j is set} to 1, and when the total value does not exceed the threshold value θ, the output signal line 214 _{j is set} to 0. Ideally, a series of these operations is completed within 4 [ms]. Since the neural network illustrated in FIG. 15 has a small number of neurons of 1,900, even if the neural network is simulated by software or the neural network is implemented by an electronic circuit, the neural network is within 4 [ms]. It is possible to operate.

As is clear from the above description, the 1,000 neurons 210 _i in the L1 layer create synapse connections in total with the 900 neurons 212 _{j in} the L2 layer, so the number of synapse connections 213 _{i, j} Reaches a maximum of 900,000, and the total number of locations where the wiring 211 _i is wired reaches a maximum of 900,000.

Here, since an unnecessary synapse connection is removed depending on an application used in an actual neural network, a learning result, and the like, there are various kinds of neurons 212 _j and synapse connections 213 _{i, j} that an output of each neuron 210 _i generates. It is. That is, a mechanism is required that allows the neuron 210 _i to selectively create the synapse connection 213 _{i, j} with the neuron 212 _j , which makes the wiring problem between neurons more difficult.

  In the human brain, it is said that one neuron has 1,000 to 10,000 synaptic connections. Therefore, when the number of artificial neurons increases, the number of wirings required is 1, Below 000, it is considered to be approximately proportional to the square, and above 10,000, it is considered to be approximately proportional to the first power. Therefore, it can be predicted that about 100 billion to 1 trillion wirings will be required to realize a neural network having a brain level of 100 million neurons. Such enormous wiring needs to be appropriately performed under appropriate rules or environments.

  In order to realize a large-scale neural network, several methods for reducing the wiring between neurons have already been devised because wiring between neurons is a problem. For example, there is Patent Document 1. In this technique, the output data of all neurons in the L1 layer are time-division multiplexed and sent to one common signal line, which is received by all neurons in the L2 layer and used for computation. When this method is used, there is a remarkable effect that only one signal line connecting the two layers is required.

  On the other hand, a medium-scale (about 10,000 neurons) neural network is often made of a digital circuit from the viewpoint of circuit stability and calculation accuracy. In a neural network of this scale, AER (Address Event Representation) devised by a group of Professor Mead of California Institute of Technology is often used for data transfer between neurons (for example, Non-Patent Document 1).

  The format of AER will be described with reference to FIG. FIG. 16 shows a data format used for data transfer between neurons in AER. In the AER, a number (Local address) is assigned to a neuron, and a number (Node address) is assigned to an LSI (Large Scale Integration), a PCB (Printed-Circuit Board), or a device in which the neuron is mounted.

  For example, assuming a PCB on which four LSIs incorporating 256 neurons are mounted as the L1 layer, 1,024 neurons can be identified by the following method. It combines 10 bits to 1,024 neurons by combining 2 bits Node address to distinguish 4 LSIs and 8 bits Local address to distinguish 256 neurons in each LSI. It is to do.

Similarly for the L2 layer, assuming a PCB on which four LSIs incorporating 256 neurons are mounted, the neurons in the L2 layer can be identified by 10-bit address information.
Considering the case of sending data from a certain neuron 210 _{i in} the L1 layer to a certain neuron 212 _j in the L2 layer, 10 bits are required for addressing the neuron 212 _j . Payload can be used in two ways.

The first method is a method of transmitting the identification number ID _i (10 [bit]) of the neuron 210 _i as Payload. The neuron 212 _j uses the received identification number ID _i as index information to access a memory that stores synapse connection weights _, thereby determining the synapse connection weights u _{i, j} . In this method, two pieces of address information are sent for one data transmission, so one data amount is 20 [bit] (= 10 [bit] +10 [bit]).

  On average, if data is sent from the L1 layer to half the neurons in the L2 layer, the total is 20 [bit] × (1,024 ÷ 2) × 1,024 = 10,485,760 [bit] = 1.3. [MB] data transmission is required. In order to execute this in 4 [ms], a data transfer rate of 1.3 [MB] ÷ 4 [ms] = 328 [MBps] is required.

  Further, in the L1 layer, it is necessary to store which neuron in the L2 layer to transfer data, so 10 [bit] × (1,024 ÷ 2) × 1,024 = 5,242,880 [bit]. = 655 [KB] memory is required.

The second method is a method of calculating and sending a _i × u _{i, j} as Payload. When a _i is 1 bit and u _{i, j} is 8 bits, a _i × u _{i, j} is 8 bits, so one data amount is 18 [bit] (= 10 [bit] +8 [bit]). It is. Therefore, the total data transfer amount is 18 [bit] × (1,024 ÷ 2) × 1,024 = 9,437, 184 [bit] = 1.2 [MB]. The total data transfer amount is slightly smaller than the transfer amount of the first method. However, the need for 655 [KB] memory in the L1 layer remains the same.

Japanese Patent Publication No. 5-47870

Serrano-Gotarredona R, Oster M, Lichtsteiner P, Linares-Barranco A, Paz-Vicente R, Gomez-Rodriguez F, Camunas-Mesa L, Berner R, Rivas-Perez M, Delbruck T, Liu SC, Douglas R, Hafliger P , Jimenez-Moreno G, Civit Ballcels A, Serrano-Gotarredona T, Acosta-Jimenez AJ, Linares-Barranco B, "CAVIAR: a 45k neuron, 5M synapse, 12G connects / s AER hardware sensory-processing- learning-actuating system for high-speed visual object recognition and tracking. ", IEEE Transactions on Neural Networks, Vol. 20, No.9, September, pp.1417-38, 2009.

  Within the brain of a living body, neuron axons grow on a daily basis, creating new synaptic connections between the target neurons' dendrites and cell bodies, and synaptic connections that are no longer used for information transmission. It has been disassembled and disappeared. This process of forming a neural network effective for information processing is called “self-organization”.

  In addition, a route through which a signal frequently passes in a neural network becomes a signal that passes more well, and a route through which the signal does not pass becomes difficult, and this is called “learning”. In order to realize a neural network imitating the brain as an electronic circuit, how to implement a self-organization and learning mechanism is an important technical issue.

  To simplify the self-organization problem, it is a wiring problem between neurons. A neural network originally has a problem that if the number of neurons incorporated therein increases, the amount of wiring between neurons becomes enormous and it becomes difficult to realize a large-scale neural network. Therefore, until now, no method has been shown for implementing a large-scale neural network having a self-organizing function as an electronic circuit.

  Therefore, the present invention has been made in view of the above problems, and provides an electronic circuit that realizes a neural network having a self-organizing function while reducing the number of wirings necessary to connect a large number of neurons. The task is to do.

In order to solve the above-described problem, the invention described in claim 1 includes a plurality of layers, a plurality of artificial neurons existing in the layer, and a synapse connection that connects the artificial neurons and artificial neurons of other layers. Among the plurality of layers, the output data of the artificial neuron of the first layer is multiplied by the weight information of the synapse connection and output to the artificial neuron of the second layer, and information processing is performed in the second artificial neuron An electronic circuit that realizes a neural network, and in a virtual space of two or more dimensions, an arbitrary number of arbitrary polytopes (for example, the cell CN in the embodiment) are disposed in each of the polytopes. A signal is transmitted between the circuit unit (for example, the circuit unit 10 in the embodiment) and the circuit unit arranged in the polytope adjacent to each other. The circuit unit is configured to output the output data of the first layer artificial neuron to the second layer artificial neuron in the same circuit unit, and a circuit unit arranged in a polytope adjacent to its own polytope. A first output unit (for example, the first output unit 1 in the embodiment) that outputs to the second layer of artificial neurons;
An input unit that receives the output data of the artificial neuron of the first layer from a circuit unit arranged in its own polytope and a circuit unit arranged in a polytope adjacent to its own polytope (for example, the input unit in the embodiment) 2) and a weight storage unit (for example, the weight storage unit 3 in the embodiment) that holds weight information of synapse connections for determining a ratio of transmitting information from the first layer artificial neuron to the second layer artificial neuron. ), The output data of the artificial neurons of the first layer and the weight information of the synapse connection (for example, the multiplication unit 4 in the embodiment), and the addition unit that calculates the sum of the products (For example, the adding unit 5 in the embodiment) and the output data of the artificial neurons of the second layer are determined based on the comparison result between the sum of the products and a predetermined threshold value. A comparison determination unit (for example, the comparison determination unit 6 in the embodiment), an output unit (for example, the second output unit 7 in the embodiment) that outputs the output data of the artificial neurons of the second layer, and the first A weight changing unit (for example, weight changing unit 8 in the embodiment) that changes the weight information of the synaptic connection based on the output data of the artificial neuron of the first layer and the output data of the artificial neuron of the second layer; An electronic circuit comprising:

  Thereby, since the data output to the adjacent polytope can be time-division multiplexed, it is possible to make one output signal line to the adjacent polytope. Further, the weight information of the synapse connection can be changed based on the output data of the first neuron and the output data of the second neuron.

According to a second aspect of the present invention, in the first aspect of the present invention, the adder includes weight information on all synaptic connections of the artificial neurons of the second layer for each circuit unit arranged in the polytope. The summation is calculated, and the weight changing unit further changes the weight information of the synapse connection so that the sum of the weight information of the synapse connection falls within a range between a first predetermined value and a second predetermined value. It is characterized by.
Thereby, it is possible to control the weight information of the synapse connection to change within a predetermined range.

According to a third aspect of the present invention, in the second aspect of the present invention, a pseudorandom number generation unit (for example, a pseudorandom number generation unit 70 in the embodiment) that generates a pseudorandom number for each synapse connection, and the weight information A learning signal generation unit that determines increase / decrease (for example, the learning signal generation unit 12 in the embodiment), and the learning signal generation unit serves as a criterion for determining whether to increase the weight information of the synapse connection. A first storage unit that stores a reference value of 1 (for example, the up / down counter 202 of the probability enable signal generator 71 in the embodiment) and a determination criterion for determining whether to reduce the weight information of the synapse connection. A second storage unit that stores a reference value of 2 (for example, the up / down counter 202 of the probability enable signal generator 72 in the embodiment), and the sum of the weight information The first reference value is increased when the first predetermined value is less than or equal to the first predetermined value, and the first reference value is decreased when the sum of the weight information is greater than the first predetermined value. 1 reference value changing unit (for example, the magnitude comparator 201 of the probability enable signal generator 71 in the embodiment) and the second reference value when the sum of the weight information is equal to or greater than the second predetermined value. A second reference value changing unit that increases and decreases the second reference value when the sum of the weight information is smaller than the second predetermined value (for example, the probability enable signal generator 72 of the embodiment). A magnitude comparator 201) and a first magnitude comparator (for example, generating a first control signal that permits an increase in the weight of the synapse connection based on the magnitude comparison result of the first reference value and the pseudo-random number (for example, , Probability rice in the embodiment And a second control signal for permitting a decrease in the weight of the synapse connection based on the magnitude comparison result of the second reference value and the pseudo-random number. A second magnitude comparator (e.g., the 8-bit magnitude comparator 203 of the probability enable signal generator 72 in the embodiment), and the learning signal generator includes output data of the artificial neurons of the first layer, The increase / decrease in the weight information of the synapse connection is determined based on the output data of the artificial neurons of the second layer, the first control signal, and the second control signal.
Accordingly, the synaptic connection weight information can be changed probabilistically so that the sum of the synaptic connection weight information falls within a predetermined range.

According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, when the value of the weight information of the synapse connection is 0, the weight change unit is weight information of the synapse connection. The value of is changed from 0 to a predetermined value.
As a result, the weight information of the synapse connection can be changed from 0 to a predetermined value, so that a new synapse connection can be generated.

According to a fifth aspect of the present invention, in the invention of the fourth aspect, the weight changing unit includes the number of pieces of the synaptic connection weight information whose value is 0 for each circuit unit arranged in the polytope. And the value of the weight information of the synapse connection is changed from 0 to a predetermined value so that the number of 0 falls within a predetermined range.
As a result, the number of synaptic connection weight information 0 is calculated, and the number of newly generated synaptic connections can be adjusted so that the number of 0 falls within a predetermined range.

According to a sixth aspect of the present invention, in the fifth aspect of the present invention, the weight changing unit is a third criterion for determining whether to change the weight information of the synapse connection from 0 to a predetermined value. A third storage unit for storing the reference value (for example, the up / down counter 202 in the probability enable signal generator 73 in the embodiment) and a pseudo random number generation unit for generating the pseudo random number (for example, the random number generation unit 11 in the embodiment). The weight changing unit increases the third reference value when the number of synaptic connection weights of 0 is equal to or less than a third predetermined value, and the number is a third predetermined value. The third reference value is decreased when the value is larger than the value of, and the pseudo-random number generator generates a pseudo-random number for each synapse connection. The weight changing unit generates a third control signal that permits the value of the synapse connection to be changed from 0 to a predetermined value based on a magnitude comparison result between the third reference value and the pseudo-random number. .
The weight changing unit changes the synaptic connection weight information from 0 to a predetermined value based on the third control signal when the synaptic connection weight information is zero.
Accordingly, the number of newly created synapse connections can be adjusted stochastically, and the probability increases or decreases according to the number of 0s of synapse connection weight information. Therefore, the number of 0s of synapse connection weight information is the third number. It can be adjusted to be close to a predetermined value.

The invention described in claim 7 is the invention according to claim 4, wherein the weight changing unit includes a pseudo-random number generating unit (for example, the pseudo-random number generating unit 11 in the embodiment) for generating a pseudo-random number, and the weight The changing unit counts the number of the synaptic connection weight information whose value is 0 for each circuit unit arranged in the polytope, and based on the comparison between the number of 0 and the pseudo-random number, The synapse connection weight information is changed from 0 to a predetermined value. Thus, the number of 0 included in the synapse connection weight information value is determined based on a comparison between the predetermined number and a pseudo-random number. Is changed from 0 to a predetermined value, the number of newly created synapse connections can be adjusted stochastically.

According to an eighth aspect of the present invention, in the invention according to any one of the first to seventh aspects, the circuit unit is configured such that the artificial layer of the first layer is a synaptic connection of the artificial neurons of the second layer. The weight information of synapse connection that does not require input of neuron output data is set to 0.
As a result, even for synapse connections that do not require transmission of information from the first layer artificial neurons to the second layer artificial neurons, the first layer artificial neurons can be obtained without losing the wiring corresponding to the synapse connections. To the second layer of artificial neurons can be blocked.

  According to the first aspect of the present invention, it is possible to receive data output from an artificial neuron of an adjacent polytope while keeping one wiring for a signal output from a polytope, thereby reducing the number of wirings between neurons. Can be reduced. In addition, since the synaptic connection weight information held in the second memory can be changed, the neural network can have a learning function and a self-organizing function.

  According to the second aspect of the present invention, the change in the weight information of each synapse connection can be controlled so that the sum of the weight information of the synapse connection for each predetermined region falls within a predetermined range. It can mimic changes in the weight of synaptic connections of neurons in the brain.

  According to the third aspect of the present invention, the synaptic connection weight information can be changed probabilistically so that the sum of the synaptic connection weight information falls within a predetermined range. It can mimic the fluctuations in the weight of synaptic connections.

  According to the invention described in claim 4, since the number of newly created synapse connections can be autonomously adjusted, the circuit unit can adjust the number of synapse connections without using an external control device. it can.

  According to the invention described in claim 5, since the rate at which synapse connections are formed within a predetermined region can be autonomously adjusted, the circuit unit can determine the number of synapse connections without using an external control device. Can be adjusted.

  According to the invention described in the claims, since the rate at which synaptic connections are formed within a predetermined region can be autonomously adjusted under an appropriate probability condition, the circuit unit can be controlled by an external control device. Without adjusting the number of synaptic connections.

  According to the invention described in claim 7, since the weight information of the synapse connection can be changed and the number of newly created synapse connections can be adjusted stochastically, the circuit unit does not depend on the external control device. The number of synaptic connections can be adjusted.

  According to the eighth aspect of the present invention, since it is not necessary to physically eliminate the wiring in the portion where the synaptic connection is lost, first, all the neurons in the L1 layer and all the neurons in the L2 layer in the predetermined region If the wiring corresponding to the synaptic connection in combination with is prepared, there is no need to perform maintenance such as replacing the circuit later.

FIG. 3 is a schematic diagram of a neural network according to an embodiment of the present invention and a diagram for explaining a signal obtained by time-division multiplexing the outputs of neurons in an L1 layer. It is a figure for demonstrating the wiring between the circuit units by one Embodiment of this invention. It is a figure for dividing a virtual plane by one embodiment of the present invention, and explaining wiring between circuit units. It is a figure for divide | segmenting the virtual space by one Embodiment of this invention by a cube, and demonstrating the wiring between circuit units. It is a block block diagram of the circuit unit 1 by one Embodiment of this invention. It is a block block diagram of the weight change part 8 by one Embodiment of this invention. 1 is a circuit diagram of a circuit unit 10 in which a circuit unit 1 according to an embodiment of the present invention is simulated by a neuron group using a pulse neural network. It is a figure for demonstrating the data conversion table hold | maintained at the data conversion memory. 1 is a circuit diagram of a 1 increase / decrease circuit 66. FIG. It is a circuit diagram of an 8-bit pseudo random number generator (Galois LFSR). FIG. 6 is a circuit diagram of the learning signal generation unit 12 when an L1 layer neuron 20 _i is an excitatory neuron. It is a circuit diagram of a probability enable signal generation circuit. FIG. 6 is a circuit diagram of the learning signal generation unit 12 when an L1 layer neuron 20 _i is an inhibitory neuron. FIG. 6 is a circuit diagram of the learning signal generation unit 12 when an L1 layer neuron 20 _i is an excitatory neuron or an inhibitory neuron. It is a schematic diagram showing a part of the neural network. It is the figure which showed the data format used for the data transfer between neurons in AER.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, the configuration of the neural network of the present invention will be described. FIG. 1A is a diagram showing the configuration of a neural network. The neural network is composed of two layers, an L1 layer and an L2 layer.

The neural network of the present invention includes a first layer (L1 layer) neuron 20 _i (i is an integer from 0 to 4,863) and a second layer (L2 layer) neuron 22 _j (j is 0 to 4). , an integer) of 863, and the wiring 21 _i to neuron 22 _j of the L2 layer from the neuron 20 _i of the L1 layer, synaptic connections _{23 i} to the output of the neuron 20 _i of the L1 layer is made in the neuron 22 _j of the L2 _{layer, j} And the output terminal 24 _j of the neuron 22 _{j in} the L2 layer.

The synaptic connection includes synaptic connection weight information w _{i, j} indicating how much the output information of the L1 layer neurons is transmitted to the L2 layer neurons. Synaptic connections are uniformly formed between the neurons in the L1 layer and the neurons in the L2 layer, but the synaptic connections that do not require information transmission from the specific neurons in the L1 layer to the specific neurons in the L2 layer By setting the weight information w _{i, j of the connection} to 0, it can be substantially invalidated.

The neurons 20 _{i in the} L1 layer are equally divided into 19 groups (hereinafter referred to as cells). Then, the number of L1 layer neurons 20 _i existing per cell is 256. In each cell, the number i of the neuron 20 _i is reassigned to an integer from 0 to 255. Similarly, the L2 layer neuron 22 _j is equally divided into 19 cells. Then, there will be 256 L2 layer neurons 22 _j per cell. In each cell, the number j of the neuron 22 _j is reassigned to an integer from 0 to 255. By the above operation, the neural network of FIG. 1A is divided into 19 cells CN (N is an integer from 0 to 18).

FIG. 1B is a diagram for explaining a signal obtained by time-division multiplexing output data of neurons in the L1 layer of the cell CN. The circuit of the cell CN time-division multiplexes the output data of 256 neurons in the L1 layer of its own cell as shown in FIG. 1 (b), and outputs the signal x _{i, t} from the P0 terminal.

The signal x _{i, t} (i is an integer from 0 to 255) is the output of the t-th (t is an integer greater than or equal to 0) round of the i-th neuron. That is, x _{0, t} is the output of neuron 20 ₀ , x _{1, t} is the output of neuron 20 ₁ ,..., X _{255, t} is the output of neuron 20 ₂₅₅ . Here, a round is a processing unit for processing output data of 256 neurons in the L1 layer once. Assuming that the processing time of one round is 4 [ms], 256 data x _{i, t} are transmitted at 4 [ms], so the data transmission speed of this signal line is 64 (= 256 [bit] ÷ 4 [ms]. ]) [Kbps].

FIG. 2 is a diagram for explaining wiring between circuit units according to an embodiment of the present invention. Neurons in the brain basically make synaptic connections only with physically close neurons. The wiring problem between neurons is solved by referring to this feature.
FIG. 2A is a diagram in which a plurality of cells are arranged on a virtual plane. In FIG. 2A, each cell has a regular hexagonal shape. FIG. 2B is a diagram showing wiring between cells. There is one circuit unit for each cell.

Since there are 19 cells in FIG. 2A, there are 4,864 (= 256 × 19) neurons 20 _i and 22 _{j in} the L1 layer and the L2 layer, respectively. Each cell has one output terminal P0 and six input terminals P1 to P6 (FIG. 2B).

  As shown in FIG. 2B, an output terminal P0 of a circuit unit of a certain cell CN is connected only to an input terminal of a circuit unit of an adjacent cell (hereinafter referred to as an adjacent cell). Further, there are certain rules regarding the input terminal of a certain circuit unit and the position of the circuit unit connected to the input terminal. Here, the cell C0 will be described as an example. As shown in FIG. 2B, the input terminal P1 of the cell C0 is connected to the output terminal P0 of the adjacent cell C1 directly above the cell C0.

  Similarly, the input terminal P2 of the cell C0 is connected to the output terminal P0 of the adjacent cell C2 on the upper right side of the cell C0. The input terminal P3 of the cell C0 is connected to the output terminal P0 of the adjacent cell C3 on the lower right side of the cell C0. The input terminal P4 of the cell C0 is connected to the output terminal P0 of the adjacent cell C4 directly below the cell C0. The input terminal P5 of the cell C0 is connected to the output terminal P0 of the adjacent cell C5 at the lower left of the cell C0. The input terminal P6 of the cell C0 is connected to the output terminal P0 of the upper left adjacent cell C6 of the cell C0.

  That is, the data received by the input terminal P1 of a certain circuit unit is the data output from the output terminal P0 of the circuit unit adjacent immediately above. Similarly, data received by the input terminal P2 of a certain circuit unit is data output from the output terminal P0 of the circuit unit adjacent to the upper right. Data received by the input terminal P3 of a certain circuit unit is data output from the output terminal P0 of the circuit unit adjacent to the lower right. The data received by the input terminal P4 of a certain circuit unit is data output from the output terminal P0 of the circuit unit adjacent immediately below. Data received by the input terminal P5 of a certain circuit unit is data output from the output terminal P0 of the circuit unit adjacent to the lower left. The data received by the input terminal P6 of a certain circuit unit is data output from the output terminal P0 of the circuit unit adjacent to the upper left.

  In the L2 layer of a certain cell, the signals input from the L1 layer of the cell and the six signals input from the adjacent cells to the input terminals P1 to P6 are used to Create synaptic connections with all neurons. That is, for example, each neuron in the L2 layer of the cell C0 in FIG. 2A is synaptic with 1,792 (= 256 × 7) neurons in the L1 layer of the cell C0 and six adjacent cells C1 to C6. Make a bond.

  Similarly, for example, each neuron in the L2 layer of the cell C1 forms a synaptic connection with all the neurons in the L1 layer of the cell C1 and six adjacent cells C8, C9, C2, C0, C6, and C7. As described above, each neuron in the L2 layer of the cell CN can make a synapse connection with 1,792 neurons in the L1 layer of its own cell and adjacent cells, but some of them are unnecessary.

Each neuron in the L2 layer forms a synapse connection with the neurons in the L1 layer of its own cell and adjacent cells, that is, the neurons in the vicinity of the L2 layer, in the virtual plane, so there are few unnecessary synapse connections, and thus the synaptic connection weight information w _There are few places where _{i and j} are set to 0.

  In the embodiment shown in FIG. 1, the number of neurons in the L1 layer and the L2 layer included in one cell is 256. However, the present invention is not limited to this, and any number of 2 or more is generally applicable. it can. Further, the number of neurons in the L1 layer and the L2 layer need not be the same. In the present embodiment, as shown in FIG. In addition, this invention is not restrict | limited to this, It is also possible to extend to three or more layers.

  In this embodiment, regular hexagonal cells are arranged on a virtual plane on which neurons are arranged. However, the present invention is not limited to this. FIG. 3 is a diagram for explaining wiring between circuit units by arranging cells other than regular hexagons on a virtual plane according to an embodiment of the present invention. For example, squares can be arranged as shown in FIG. 3A, and equilateral triangles can be arranged.

  Further, instead of arranging the same polygons, it is possible to arrange a plurality of types of polygons as shown in FIG. Furthermore, since the virtual plane is virtual, there is no particular problem even if the polygons overlap each other or a gap is generated between the polygons. Connect the input terminal to which no signal is input to the ground.

  The polygonal shape is related to the maximum number of synaptic connections that one neuron in the L2 layer can form. For example, in the case of a regular hexagon, since the number of adjacent cells is 6, if 256 neurons are included in the L1 layer of one cell, a maximum of 256 × 7 synaptic connections are formed in the neurons of the L2 layer. .

  On the other hand, in the case of FIG. 3A, since the number of adjacent cells is 8, a maximum of 256 × 9 synapse connections are formed. In the case of FIG. 3B, it can be interpreted that three types of cells having different maximum synapse coupling numbers are mixed, and gaps other than regular pentagons can be ignored. Furthermore, the present invention can be similarly applied even when an arbitrarily shaped figure is arranged on the virtual plane instead of a regular polygon.

  In the description of the present embodiment, the virtual two-dimensional plane is considered and the neurons are distributed and arranged there. However, the present invention is not limited to this. When imitating the human brain, a two-dimensional plane is sufficient, but the present invention can be similarly applied to a three-dimensional space as a structure exceeding this.

  FIG. 4 is a diagram for explaining wiring between circuit units by arranging cubes in a virtual space according to an embodiment of the present invention. For example, as shown in FIG. 4, a cube is arranged in a virtual three-dimensional space, and an L1 layer neuron of an arbitrary cube makes a synaptic connection with an L2 layer neuron in the cube and a cube adjacent to the cube. I will decide.

  As shown in FIG. 4, for each cell, there is one output terminal on the top of the cube and one input terminal on each surface. A cell adjacent to a certain cell is a total of six cells adjacent in a front-back, up-down, left-right direction. One input terminal on each of the front, back, top, bottom, left and right surfaces of a cell is connected to the output terminal of an adjacent cell.

  In another embodiment, there is one output terminal for each cell on the top of the cube, one input terminal on each side (6 in total), 1 on each side (12 in total), and 1 on each corner. (8 in total), that is, 26 in total. A cell adjacent to a certain cell is a total of 26 cells adjacent to each other in terms of surface, side, and corner. Twenty-six input terminals existing in a certain cell are connected to output terminals of twenty-six adjacent cells, respectively.

  In addition, an arbitrary polyhedron can be arranged in a three-dimensional space instead of a cube. As described above, the present invention is generally applicable when an arbitrary polytope is arranged in a space of an arbitrary order of two or more dimensions. The polytope represents a polygon in the case of two dimensions, a polyhedron in the case of three dimensions, and a polytope in the case of four dimensions or more. A polytope is a figure obtained by generalizing a two-dimensional polygon or three-dimensional polyhedron into a general dimension. However, in the case of a four-dimensional or larger space, the illustration is omitted because it cannot be illustrated.

<Description of Example 1 of Circuit Unit>
In order to realize a situation where learning and self-organization always occur like the brain of a living body with a neural network, the weight w _{i, j} of the synapse connection is changed under an appropriate rule during the operation of the neural network. I must continue. If this operation is performed by software by adding, for example, a CPU (Central Processing Unit), the data processing time becomes significantly longer as the scale of the neural network increases, making it very difficult to learn while operating the neural network. . In addition, when a high-speed CPU is added, there is a problem that power consumption increases.

The present invention provides a hardware technology that changes the weights w _{i, j} of synapse connections under appropriate rules, thereby suppressing the increase in power consumption and performing neural learning and self-organization in real time. The purpose is to realize the Internet.

Next, an embodiment of the circuit unit of the present invention will be described. FIG. 5 is a block diagram of the circuit unit 10 according to an embodiment of the present invention. The entire circuit unit 10 corresponds to one cell CN shown in FIG.
The circuit unit 10 includes a first output unit 1, an input unit 2, a weight storage unit 3, a multiplication unit 4, an addition unit 5, a comparison determination unit 6, a second output unit 7, and a weight change. And the unit 8.

The first output unit 1 holds input data of the neurons 20 _{i in} the L1 layer. Here, the input data is data representing the firing frequency of the neuron 20 _{i in} the L1 layer. For example, the input data is represented by 4 bits, and indicates that the smaller the value L1 layer low firing rates of neurons 20 _i of the firing rates of neurons 20 _i of the larger the value of the L1 layer high.

The first output unit 1 converts the input data of the neurons 20 _{i in} the L1 layer from data representing the firing frequency into a pulse train representing a time series of firing states, and sequentially outputs the pulse train. For example, the data representing the time series of the firing state is represented by 15-bit data in which 15 pieces of 1-bit data are arranged, and the 1-bit data represents the firing state of the neuron 20 _{i in} the L1 layer. That is, if the 1-bit data is 0, the L1 layer neuron 20 _i is not fired, and if the 1-bit data is 1, the L1 layer neuron 20 _i is fired.

The first output unit 1 selects one bit at a time from the converted data of each neuron 20 _i and outputs it to the input unit 2 so that the neurons 20 _{i in the} L1 layer output in order. For example, the first output unit 1 sequentially outputs 256 pieces of data at specific bit positions of the 15-bit data of the neurons 20 _{i in} the L1 layer to the input unit 2. When the 256 pieces of data are output, the operation of changing the specific bit position of the 15-bit data and outputting different 256 pieces of data to the input unit 2 is repeated.
The input unit 2 receives the converted output data output from the first output unit 1. The input unit 2 outputs the received output data to the multiplication unit 4 and the weight change unit 8. The input unit 2 always outputs 1 data to the multiplication unit 4 instead of the output data.

The weight storage unit 3 holds weight information of synapse connection from the neuron 20 _{i in the} L1 layer to the neuron 22 _{j in the} L2 layer. The weight storage unit 3 receives a clock signal from an external device (not shown). The weight storage unit 3 calculates a count value from the clock signal, and reads the synaptic connection weight information held at the address of the count value. The weight storage unit 3 outputs the weight information of the synapse connection to the multiplication unit 4 and the weight change unit 8.

  Further, the weight storage unit 3 receives new synapse connection weight information output from the weight change unit 8. The weight storage unit 3 changes the weight information of the synapse connection held at the address of the count value to the new synapse connection weight information.

  The multiplication unit 4 receives the output data of the first output unit output from the input unit 2 or always 1 data, and the synaptic connection weight information output from the weight storage unit 3. The multiplication unit 4 multiplies the output data or always 1 data by the synaptic connection weight information and calculates a product of them. The multiplying unit 4 outputs the product to the adding unit 5.

The adder 5 receives the product output from the multiplier 4. When the input unit 2 outputs the output data of the first output unit, the adder unit 5 adds the products for each neuron 22 _{j in the} L2 layer, calculates a sum, and compares the sum with the comparison determination unit 6. Output to. When the input unit 2 always outputs 1 data, the adding unit 5 adds all the data of the product to calculate the sum, and outputs the sum to the weight changing unit 8.
The comparison determination unit 6 receives the sum output from the addition unit 5 and the threshold value θ input from an external device (not shown). Here, the firing data is data indicating that the L2 layer neuron 22 _j has fired (for example, 1 for 1-bit data) or data indicating that the L2 layer neuron 22 _j has not been fired (for example, 1 bit). It is defined as 0) in the data.

When the sum is larger than the threshold θ, the comparison determination unit 6 outputs 1 to the second output unit 7 as data indicating that the neuron 22 _{j in the} L2 layer has fired. On the other hand, when the sum is equal to or smaller than the threshold θ, the comparison determination unit 6 outputs 0 to the output unit 7 as data indicating that the neuron 22 _{j in the} L2 layer has not fired.

The second output unit 7 receives the ignition data output from the comparison determination unit 6. The second output unit 7 holds the ignition data. The second output unit 7 outputs the firing data as output data of the L2 layer neuron 22 _{j to} an external device (not shown) and the weight changing unit 8.

The weight changing unit 8 includes a clock signal input from an external device (not shown), output data output from the input unit 2, weight data output from the weight storage unit 3, and a sum total output from the adding unit 5. The ignition data output by the second output unit 7 is received.
The weight changing unit 8 calculates the number of data having a weight of 0 stored in the weight storage unit 3 from the weight data, and increases or decreases the probability of changing the data having a weight of 0 to other than 0 from the coefficient result. Further, the weight changing unit 8 increases or decreases the probability of increasing the weight data according to the sum. Further, the weight changing unit 8 increases or decreases the probability of decreasing the weight data according to the sum.
Whenever the weight data is 0, the weight changing unit 8 changes the weight information of the new synapse connection to a value other than 0 according to the probability of changing the weight data to a value other than 0 every time the clock signal is received. Each time the weight change unit 8 receives a clock signal, if the weight data is other than 0, the weight change unit 8 increases the output data, the ignition data, the probability of increasing the weight data, and the probability of decreasing the weight data. Based on this, new synaptic connection weight information is generated. The weight changing unit 8 outputs the new synaptic connection weight information to the weight storage unit 3.

  The weight changing unit 8 controls to change the weight information of the synapse connection currently held in the weight storage unit 3 to the weight information of the new synapse connection, so that the synapse connection held in the weight storage unit 3 is changed. Weight information can be changed. In other words, neural network learning and self-organization are possible.

  FIG. 6 is a block diagram of the weight changing unit 8 according to an embodiment of the present invention. The weight changing unit 8 includes a pseudo random number generating unit 11, a learning signal generating unit 12, a weight increasing / decreasing unit 13, a synapse creation command unit 14, and a switching unit 15.

The pseudorandom number generator 11 receives a clock signal from an external device not shown. The pseudo random number generation unit 11 generates a pseudo random number based on the clock signal, and outputs the pseudo random number to the learning signal generation unit 12 and the synapse creation command unit 14. Note that the pseudo-random numbers generated by the pseudo-random number generator 11 may not have the same probability of appearance frequency of each number out of a predetermined number of predetermined ranges.
The learning signal generation unit 12 outputs the output data output from the input unit 2, the sum total output from the addition unit 5, the firing data output from the second output unit 7, and the pseudo random number output from the pseudo random number generation unit 11. Receive.

  The learning signal generation unit 12 determines whether the sum is within the first predetermined range, and changes the count value for increasing the weight of the synapse connection according to the determination result. The learning signal generation unit 12 outputs a synaptic weight increase / decrease signal based on the comparison result between the increase count value and the pseudo random number. The learning signal generation unit 12 generates a synaptic weight increase instruction signal based on the increase / decrease signal, the output data, and the firing data. The learning signal generation unit 12 outputs the weight increase instruction signal to the weight increase / decrease unit 13.

  Similarly, the learning signal generation unit 12 determines whether or not the sum is within the second predetermined range, and changes the synaptic connection weight reduction count value according to the determination result. The learning signal generation unit 12 outputs a weight reduction enable / disable signal for synapse connection based on the comparison result between the decrease count value and the pseudo random number. The learning signal generation unit 12 generates a weight reduction instruction signal for synaptic connection based on the decrease enable / disable signal, the output data, and the firing data. The learning signal generation unit 12 outputs the weight reduction instruction signal to the weight increase / decrease unit 13.

  The weight increase / decrease unit 13 receives the weight increase instruction signal output from the learning signal generation unit 12, the weight decrease instruction signal, and the weight information output from the weight storage unit 3. Based on the weight increase instruction signal and the weight decrease instruction signal, the weight increase / decrease unit 13 generates weight information of the synapse connection changed from the weight information.

  Specifically, when the weight increase instruction signal is 1, the weight increasing / decreasing unit 13 increases the weight information of the synapse connection by a certain amount to obtain the changed weight information of the synapse connection. When the weight decrease instruction signal is 1, the weight increase / decrease unit 13 decreases the weight information of the synapse connection by a certain amount to obtain the changed synapse connection weight information. When both the weight increase instruction signal and the weight decrease instruction signal are 0, the weight increase / decrease unit 13 sets the weight information of the synapse connection as the weight information of the synapse connection that is changed as it is. The weight increase / decrease unit 13 outputs the changed synaptic connection weight information to the switching unit 15.

The synapse creation command unit 14 receives the weight information of the synapse connection output from the weight storage unit 3 and the pseudo random number output from the pseudo random number generation unit 11. The synapse creation instruction unit 14 generates a synapse creation instruction signal based on the weight information of the synapse connection and the pseudo random number.
Specifically, the synapse creation command unit 14 checks the weight information of the synapse connection, and sets the zero determination signal to 1 when the value is 0. The synapse creation command unit 14 counts the number of times the zero determination signal becomes 1. The synapse creation command unit 14 determines whether or not the count value is within a third predetermined range, and changes the synapse creation instruction count value according to the determination result. The synapse creation command unit 14 outputs a synapse creation enable / disable signal based on a comparison result between the synapse creation instruction count value and the pseudo-random number.

  The synapse creation command unit 14 generates a synapse creation instruction signal based on the creation permission / inhibition signal and the zero determination signal. For example, the synapse creation instruction unit 14 sets the synapse creation instruction signal to 1 when the creation enable / disable signal is 1 and the zero determination signal is 1. The synapse creation command unit 14 outputs the synapse creation instruction signal to the switching unit 15.

  The switching unit 15 receives the changed weight information of the synapse connection output from the weight increase / decrease unit 13 and the synapse creation instruction signal output from the synapse creation command unit 14. Based on the synapse creation instruction signal, the switching unit 15 selects either predetermined fixed value data or the changed synapse connection weight information.

  Specifically, for example, when the synapse creation instruction signal is 1, the switching unit 15 selects the predetermined fixed value data. On the other hand, when the synapse creation instruction signal is 0, the switching unit 15 selects the changed weight information of the synapse connection. The switching unit 15 outputs the selected information to the weight storage unit 3.

  FIG. 7 is a circuit diagram simulating a circuit unit according to an embodiment of the present invention with a group of neurons used in a pulse neural network. The entire circuit unit 10 in FIG. 7 corresponds to one regular hexagon shown in FIG. In this embodiment, each neuron is not an individual electronic circuit, but is a virtual excitatory neuron, and 256 neurons are mounted together for each layer.

  A method for implementing 256 neurons in each layer is as follows. Input data (4 [bit] data) of 256 neurons in the L1 layer is stored in the memory 21. Weight data (8 [bit] data) of the synaptic connection from the L1 layer neuron to the L2 layer neuron is stored in the memory 65. On the other hand, the output data of 256 neurons in the L2 layer output to the terminal 46 is represented as a 1 [bit] signal of 0 or 1.

As shown in FIG. 7, the circuit unit 10 includes a first output unit 1, an input unit 2, a weight storage unit 3, a multiplication unit 4, an addition unit 5, a comparison determination unit 6, and a second determination unit 6. The output unit 7 and the weight changing unit 8 are used.
The first output unit 1 is configured using an 8-bit counter 20, a memory 21, and a pulse train modulation circuit 22.

The input unit 2 includes an output terminal P0, an input terminal Pq (q is an integer from 1 to 6), an 8-bit counter 23A, an 8-bit counter 23B, a memory 24A, a memory 24B, and an OR gate 60. It is configured using.
The weight storage unit 3 includes a 19-bit counter 64 and a memory 65.

The multiplication unit 4 is configured using an AND gate 26.
The adder 5 is configured using an adder 27 and an AND gate 28.
The comparison determination unit 6 is configured using a size comparator 29.
The second output unit 7 is configured using an 8-bit counter 61 and a memory 62.

The weight changing unit 8 includes a pseudo random number generating unit 11, a learning signal generating unit 12, a weight increasing / decreasing unit 13, a synapse creation command unit 14, and a switching unit 15.
The pseudo random number generator 11 is configured using a pseudo random number generator 70.
The weight increase / decrease unit 13 is configured using a 1 increase / decrease circuit 66.
The synapse creation command unit 14 includes a zero detector 68, a 19-bit counter 69, and a probability enable signal generator 73.
The switching unit 15 is configured using a three-input selector 63.

Each time the count-up signal is input to the input terminal 41, the 8-bit counter 20 increments the value of the terminal Q, which is a count value, by 1 and outputs the count value to the memory 21. When the count value is counted up to 255, the 8-bit counter 20 returns the count value to 0 when the next count-up signal is input.
The 8-bit counter 20 sets the count value to 0 when a reset signal (not shown) is input.

  The memory 21 stores input data of the L1 layer neurons. The L1 layer neuron data stored in the memory 21 is a number from 0 to 15. Since the number of neurons in the L1 layer is 256, the memory 21 has a capacity of 1,024 (= 4 [bit] × 256) [bit]. The input data of the i-th neuron in the L1 layer is stored at address i of the memory 21.

  The memory 21 reads the 4-bit data at the address indicated by the count value output by the 8-bit counter 20 and outputs the data to the pulse train modulation circuit 22. For example, when the input count value is 0, the memory 21 reads the data at address 0 in the memory 21 and outputs the data to the pulse train modulation circuit 22.

  The pulse train modulation circuit 22 converts the 4-bit data output from the memory 21 into a 15-bit pulse train. The pulse train modulation circuit 22 includes a data conversion memory, and holds a data conversion table in the data conversion memory. FIG. 8 is a diagram for explaining a data conversion table held in the data conversion memory. In FIG. 8, 4-bit data is data received by the data conversion memory. The 15-bit data is data output from the data conversion memory. One 15-bit data is associated with each of the 16 types of 4-bit data. The pulse train modulation circuit 22 converts the 4-bit data output from the memory 21 into corresponding 15-bit data using a data conversion memory that stores the data conversion table shown in FIG.

The pulse train modulation circuit 22 outputs only the LSB (Least Significant bit) of the 15-bit data train to the memory 24A and the memory 24B of its own cell. The above operation is repeated until the count value of the counter 20 reaches 255. This series of processing is called round.
As described above, since the output data rate of the pulse train modulation circuit 22 is 64 [kbps], the count-up signal of the counter 20 is 64 [kHz].

  In addition, the pulse train modulation circuit 22 outputs the data to the input terminal Pq of the adjacent cell via the output terminal P0. Thereafter, in the adjacent cell, the data is output from the terminal Pq to the memories 24A and 24B.

Each time the number of rounds increases, the pulse train modulation circuit 22 increases the bit position of the data extracted from the 15-bit data train by 1, and outputs data x _i at the bit position.
In round 14, when the pulse train modulation circuit 22 finishes outputting the most significant bit data of the 15-bit data to the memory 24A, the memory 24B of its own cell and the input terminal Pq of the adjacent cell, the data stored in the memory 21 The transfer process ends.

  In this embodiment, data is output in ascending order from the least significant bit of the 15-bit data, but may be output in descending order from the most significant bit. Moreover, you may output at random from arbitrary bit positions. Alternatively, the table data shown in FIG. 8 may be appropriately changed while outputting in ascending order from the least significant bit. This is because the output information of the neurons in the L1 layer is represented by the frequency of 1 existing in the 15-bit data string, and therefore the order in which the data is output does not matter.

When the 8-bit counter 23A belonging to the L2 layer writes data to the memory 24A, the count-up signal (64 [kHz]) identical to the count-up signal input to the terminal 41 is input to the terminal 42A. Each time the count-up signal is input, the counter 23A increases the count value by 1 and outputs the count value to the memory 24A. When the counter 23A counts up to 255, it returns the count value to 0 with the next count-up signal.
The counter 23A sets the count value to 0 when a reset signal (not shown) is input.
Similarly, when writing data into the memory 24B, the counter 23B receives the same count-up signal as the count-up signal input to the terminal 41 at the terminal 42B. Each time the count-up signal is input, the counter 23B increases the count value by 1 and outputs the count value to the memory 24B. When the counter 23B counts up to 255, it returns the count value to 0 with the next count-up signal.
The counter 23B sets the count value to 0 when a reset signal (not shown) is input.

  The memory 24A and the memory 24B (each 7 [bit] × 256 capacity) alternately store the data output from the terminal P0 to the terminal P6 transmitted through the seven signal lines for each round. Here, the data output from the terminal P0 is data output from the pulse train modulation circuit 22 of its own cell. The data output from the terminal Pq is data output from the pulse train modulation circuit 22 of six adjacent cells.

On the L2 layer side, 4 [ms], which is the time of one round, is divided into two equal parts, the output data of the L2 layer neurons is calculated in the first half [ms], and the L2 layer neurons in the second half [ms] Learning and self-organizing.
When reading data from the memory 24A, the 8-bit counter 23A counts up to 512 times the count-up signal input to the terminal 41 at the terminal 42A, that is, 32.768 (= 64 [kHz] × 512) [MHz]. A signal is input.
Since the count value output from the counter 23A increases at 32.768 [MHz], the memory 24A reads the stored 7-bit data at a speed of 32.768 [MHz]. Memory 24A is a 1-bit data _{s i} of 229.4 [Mbps] of 7 times the read data is parallel-serial conversion. Here, the subscript i is an integer from 0 to 1,792 (= 256 × 7-1).
Similarly, when the 8-bit counter 23B reads data from the memory 24B, a count-up signal of 32.768 [MHz] is input to the terminal 42B.
The memory 24B reads the stored 7-bit data at 32.768 [MHz] because the count value output from the counter 23B increases at 32.768 [MHz]. Memory 24B is set to 1-bit data _{s i} of 229.4 [Mbps] of 7 times the read data is parallel-serial conversion.
The memory 24A and the memory 24B alternately perform a data write operation and a read operation for each round. For example, when the memory 24A performs a write operation in a certain round, the memory 24B performs a read operation. The memory 24A and the memory 24B output the read data s _i to the OR gate 60 and the learning signal generation unit 12.

The OR gate 60 receives the data s _i output from the memory 24A or the memory 24B and a control signal at the terminal 80. In the first half of the round, 0 is input to the terminal 80, so the OR gate 60 outputs the data s _i . In the second half of the round, since 1 is input to the terminal 80, the OR gate 60 always outputs 1 to the AND gate 26.

The 19-bit counter 64 receives a count up signal of 229.4 [MHz] at its terminal 84. The counter 64 increases the count value by 1 when the count-up signal is input. The counter 64 outputs the count value to the memory 65.
When the count value is 458,751 (= 256 × 7 × 256-1) and the count value is input, the counter 64 returns the count value to zero. The counter 64 sets the count value to 0 when a reset signal (not shown) is input.

The memory 65 receives the count value output from the counter 64. Since the count value increases at 229.4 [MHz], the memory 65 reads the synapse connection weight data w _{i, j} at 229.4 [MHz]. The memory 65 outputs the read weight data w _{i, j} to the AND gate 26, the weight increase / decrease unit 13, and the synapse creation command unit 14.
The AND gate 26 receives the data output from the OR gate 60 and the weight data w _{i, j of} the synapse connection output from the memory 65.
In the first half of the round, the AND gate 26 takes the product of the data s _i output from the OR gate 60 and the corresponding weight data w _{i, j} of the synapse connection _, and the product y _{i, j} (= s _i). Xwi _{, j} ) is output to the adder 27.
In the second half of the round, the AND gate 26 takes the product of the data 1 output from the OR gate 60 and the synaptic connection weight data w _{i, j} and outputs the product w _{i, j} to the adder 27.

In the first half of the round, the adder 27 receives the y _{i, j} output from the AND gate 26 and calculates the sum of Σy _{i, j} with respect to i for each j-th neuron in the L2 layer. In this case, the summation symbol Σ represents 256 × 7 calculations.
Since the AND gate 26 outputs the data y _{0, j} to the adder 27, the signal input to the terminal 44 of the AND gate 28 becomes 0, so that the AND gate 28 has 0 at the input terminal on one side of the adder 27. Is output. As a result _, the initial value of the sum calculation of Σy _{i, j} becomes zero. Since the signal input to the terminal 44 becomes 1 except when the AND gate 26 outputs the data y _{0, j} to the adder 27, the adder 27 performs Σy every j-th neuron of the L2 layer. _i, the sum of i of _{j Y j (= Σy i,} j) is calculated. When the adder 27 finishes calculating the sum _Yj , the adder 27 outputs the sum _Yj to the magnitude comparator 29.

In the second half of the round, the adder 27 receives the w _{i, j} output from the AND gate 26 and calculates the sum Σw _{i, j} for all i and all j. The sum symbol Σ in this case represents 256 × 7 × 256 calculations. That is, Σw _{i, j} is the sum of weight data of all synapse connections of all neurons in the L2 layer.
Since the AND gate 26 outputs the data y _0,0 to the adder 27, the signal input to the terminal 44 of the AND gate 28 becomes 0, so that the AND gate 28 has 0 at the input terminal on one side of the adder 27. Is output. As a result _, the initial value of the total calculation of Σwi _{, j} becomes zero. The signal input to the terminal 44 becomes 1 except when the AND gate 26 outputs the data y _0,0 to the adder 27. When the adder 27 finishes calculating the sum Σwi _{, j} , the adder 27 outputs the sum Σwi _{, j} to the learning signal generator 12.

Magnitude comparator 29 receives the sum Y _j to the adder 27 is output, the sum Y _j, is compared with a threshold value data θ supplied from the register or an external (not shown) to the terminal 43, than the threshold data θ When the sum _Yj is large, 1 is output to the memory 62. The size comparator 29 outputs 0 to the memory 62 in other cases.

When the count up signal of 128 [kHz] is input to the terminal 82, the 8-bit counter 61 increases the count value by one. The 8-bit counter 61 resets the count value to 0 when a count-up signal is input to the terminal 82 when the count value is 255. The 8-bit counter 61 outputs the count value to the memory 62.
The 8-bit counter 61 sets the count value to 0 when a reset signal (not shown) is input.

The memory 62 receives the 1-bit data output by the magnitude comparator 29 and the count value output by the 8-bit counter 61 in the first half of the round. The memory 62 stores the 1-bit data at the address of the count value. Thereby, the memory 62 stores the outputs of the neurons in the L2 layer (total 256 [bit] = 1 [bit] × 256).
The memory 62 receives the count value output from the 8-bit counter 61 in the second half of the round. The memory 62 reads the 1-bit data b _j from the address of the count value, and outputs the 1-bit data b _j to the terminal 46 and the learning signal generation unit.

  The 3-input selector 63 selects the input I2 when the signal input to the terminal 83 becomes 1 before starting the calculation process in the neural network. After the selection, the 3-input selector 63 selects data supplied from the DB bus and outputs the data to the memory 65. As a result, the initial value of the synaptic connection weight data can be written in the memory 65 in advance by an external circuit (not shown) through the DB bus.

The 3-input selector 63 receives the signal Con output from the AND gate 67. The 3-input selector 63 receives the output signal of the 1 increase / decrease circuit 66.
The 3-input selector 63 selects the input I0 when the signal Con is 0. After the selection, the three-input selector 63 selects an update value of the synapse connection weight data, which is an output of the 1 increase / decrease circuit 66, and outputs the update value to the memory 65.

  The 3-input selector 63 selects the input I1 when the signal Con is 1. After the selection, the three-input selector 63 selects a fixed value 'D0' supplied from a register (not shown) or from the outside, and outputs the fixed value 'D0' to the memory 65.

The memory 65 receives the count value output from the 19-bit counter 64 and the data output from the 3-input selector 63.
In the first half of the round, the memory 65 reads the data at the address of the count value and outputs the data to the AND gate 26 and the zero detector 68.

  In the second half of the round, the memory 65 reads the data at the address of the count value and outputs the data to the AND gate 26, the 1 increase / decrease circuit 66, and the zero detector 68. Further, the memory 65 stores the updated value output from the three-input selector 63 at the address of the count value as a new synaptic connection weight.

Next, the 1 increase / decrease circuit 66 will be described. When the input terminal I is 1, the 1 increase / decrease circuit 66 increases the value of the input terminal A by 1 and outputs the increased value from the output terminal Y. However, when the value of the input terminal A is 255, if it is increased by 1, it becomes 0, so that it is output as 255. Further, when the value of the input terminal A is 0, if it is increased by 1, it becomes 1, and a new synaptic connection is generated, so that 0 is output as it is.
When the input terminal D is 1, the 1 increase / decrease circuit 66 decreases the value of the input terminal A by 1 and outputs the reduced value from the output terminal Y. However, when the value of the input terminal A is 0, if it is decreased by 1, it becomes 255, so that it is output as 0.
When both the input terminal I and the input terminal D are 0, the 1 increase / decrease circuit 66 outputs the value of the input terminal A as it is from the output terminal Y.

  FIG. 9 is a circuit diagram of the 1 increase / decrease circuit 66. In FIG. 9, the 1 increase / decrease circuit 66 includes an x′FF ′ detector 101, an x′00 ′ detector 102, a 3-input AND gate 103, an AND gate 104, an OR gate 105, and an 8-bit adder / subtractor 106. And is configured using.

  The x′FF ′ detector 101 is, for example, an 8-input NAND gate. The x′FF ′ detector 101 receives the signal at the terminal A and inputs it to the terminal In. When all the bits of the signal at the terminal In are 1, the x′FF ′ detector 101 sets the signal at the terminal F to 0 and outputs the signal 0 to the 3-input AND gate 103. The x′FF ′ detector 101 sets the signal at the terminal F to 1 when at least one bit of the signal at the terminal In is 0.

  The x′00 ′ detector 102 is, for example, an 8-input OR gate. The x′00 ′ detector 102 receives the signal at the terminal A and uses it as an input at the terminal In. When all bits of the signal at the terminal In are 0, the x′00 ′ detector 102 sets the signal at the terminal Z to 0 and outputs the signal 0 to the 3-input AND gate 103 and the AND gate 104. The x′00 ′ detector 102 sets the signal at the terminal Z to 1 when at least one bit of the signal at the terminal In is 1.

  The 3-input AND gate 103 receives the signal at the terminal F output from the x′FF ′ detector 101, the signal at the terminal Z output from the x′00 ′ detector 102, and the signal at the terminal I. The 3-input AND gate 103 outputs 1 to the OR gate 105 when the signal of the terminal F, the signal of the terminal Z, and the signal of the terminal I are all 1. The 3-input AND gate 103 outputs 0 to the OR gate 105 in other cases.

  In other words, the 3-input AND gate 103 outputs 1 to the OR gate 105 when the data at the terminal A is neither 0 nor 255 and the signal at the terminal I is 1.

The AND gate 104 receives the signal at the terminal Z and the signal at the terminal D. The AND gate 104 outputs 1 to the OR gate 105 and the adder / subtracter 106 when the signal at the terminal Z and the signal at the terminal D are both 1. In other cases, the AND gate 104 outputs 0 to the OR gate 105 and the adder / subtractor 106.
The OR gate 105 receives the signal output from the AND gate 103 and the signal output from the AND gate 104. The OR gate 105 calculates the logical sum of the two signals and outputs the logical sum to the adder / subtractor 106.

  The adder / subtractor 106 receives the signal output from the AND gate 104 as an input signal of the terminal m. Further, the adder / subtractor 106 receives the data of the terminal A as input data of its own terminal A. Further, the adder / subtractor 106 receives the data output from the OR gate 105 as input data of the terminal B.

  The adder / subtracter 106 adds the data of the terminal A and the data of the terminal B when the input signal of the terminal m is 0. The adder / subtractor 106 subtracts the data of the terminal B from the data of the terminal A when the input signal of the terminal m is 1. The adder / subtractor 106 outputs the added value or the subtracted value as data of the terminal S. The data of the terminal S becomes output data of the terminal Y of the 1 increase / decrease circuit 66.

The 1 increase / decrease circuit 66 receives the weight data w _{i, j} of the synapse connection output from the memory 65 at the input terminal A. The 1 increase / decrease circuit 66 receives the signal Inc output from the learning signal generator 12 at the input terminal I. The 1 increase / decrease circuit 66 receives the signal Dec output from the learning signal generation unit 12 at the input terminal D.

  Next, the operation of the 1 increase / decrease circuit 66 will be described. As will be described later, the signals at the terminals I and D do not both become 1 at the same time. Therefore, when the signal at terminal I is 0 and the signal at terminal D is 1, when the signal at terminal I is 1 and the signal at terminal D is 0, the signal at terminal I is 0 and the signal at terminal D is 0. The three cases will be described.

A case where the signal at the terminal I is 0 and the signal at the terminal D is 1 will be described. When the data w _{i, j at the} terminal A is other than 0, the output Z of the x′00 ′ detector 102 is 1. Since the output Z of the x′00 ′ detector 102 is 1 and the signal at the terminal D is 1, the output of the AND gate 104 is 1. When the output of the AND gate 104 becomes 1, the output of the OR gate 105 becomes 1. Accordingly, since the terminal m of the adder / subtracter 106 becomes 1 and the terminal B becomes 1, the adder / subtractor 106 outputs a value obtained by subtracting 1 from the data w _{i, j} of the terminal A to the terminal S.

When the data w _{i, j at the} terminal A is 0, the output Z of the x′00 ′ detector 102 is 0. Since the output Z of the x′00 ′ detector 102 is 0, the outputs of the AND gate 103 and the AND gate 104 are both 0, and the output of the OR gate 105 is also 0. Accordingly, since the terminal m of the adder / subtracter 106 is 0 and the terminal B is 0, the adder / subtracter 106 adds the value wi _{, j} of the terminal A to 0, that is, the input value 0 of the terminal A (as it is). Output to terminal S.

A case where the signal at the terminal I is 1 and the signal at the terminal D is 0 will be described. When the data w _{i, j at the} terminal A is other than x'FF 'and other than x'00', the output F of the x'FF 'detector 101 and the output Z of the x'00' detector 102 are both 1. Therefore, the output of the AND gate 103 becomes 1. Since the output of the AND gate 103 becomes 1, the output of the OR gate 105 becomes 1. On the other hand, since the signal at the terminal D is 0, the output of the AND gate 104 becomes 0. Accordingly, since the terminal m of the adder / subtracter 106 is 0 and the terminal B is 1, the adder / subtractor 106 outputs a value obtained by adding 1 to the data w _{i, j} of the terminal A to the terminal S.
When the data w _{i, j at the} terminal A is x'FF 'or x'00', the output F of the x'FF 'detector 101 or the output Z of the x'00' detector 102 is 0, so that the AND gate The output of 103 becomes 0. Since the output of the AND gate 103 is 0, the output of the OR gate 105 is also 0. Accordingly, since the terminal m of the adder / subtracter 106 is 0 and the terminal B is 0, the adder / subtracter 106 is a value obtained by adding 0 to the data w _{i, j} of the terminal A, that is, the input value x′FF ′ of the terminal A or 0 is output to the terminal S (as it is).

Next, the case where the signal at the terminal I is 0 and the signal at the terminal D is 0 will be described. Since the signal at terminal I is 0, the output of AND gate 103 is 0. Since the signal at the terminal D is 0, the output of the AND gate 104 becomes 0. Since both the output of the AND gate 103 and the output of the AND gate 104 are 0, the output of the OR gate 105 becomes 0. Therefore, since the terminal m of the adder / subtractor 106 is 0 and the terminal B is 0, the adder / subtracter 106 adds the value obtained by adding 0 to the data w _{i, j} of the terminal A, that is, the input value w _{i, j} of the terminal A. , Output to the terminal S as it is.

The zero detector 68 is a detector that detects whether or not the data at the input terminal In is 0, and includes, for example, an 8-input NOR gate. The zero detector 68 receives the weight data w _{i, j} of the synapse connection output from the memory 65. The zero detector 68 sets the signal at the terminal Z to 1 when w _{i, j} is 0, and outputs 1 to the AND gate 67 and the 19-bit counter 69.
When the w _{i, j} is other than 0, the zero detector 68 sets the signal at the terminal Z to 0 and outputs 0 to the AND gate 67 and the 19-bit counter 69.

The 19-bit counter 69 is initialized to 0 by receiving a reset signal (not shown) at the beginning of the round. The 19-bit counter 69 receives the signal Zero output from the zero detector 68 in the first half of the round. The 19-bit counter 69 increases the value of the terminal Q by 1 when the signal Zero is 1, and does not change the value of the terminal Q when the signal Zero is 0. In other words, the 19-bit counter 69 counts the number of synaptic connection weight data w _{i, j} being zero in the first half of the round.
When the 19-bit counter 69 finishes counting the number of synaptic connection weight data w _{i, j} being 0, the 19-bit counter 69 outputs the count value to the probability enable signal generator 73.

The pseudorandom number generator 70 receives the clock signal input to the terminal 84 at the terminal ck. The pseudo random number generator 70 generates a new pseudo random number every time the clock signal is input, and the pseudo random number is generated as a probability enable signal generator 71, a probability enable signal generator 72, and a probability enable signal generator 73. Output to. That is, the pseudo random number generator 70 generates and outputs a new pseudo random number every time the memory 65 outputs a new wi _{, j} .

  FIG. 10 is a circuit diagram of an 8-bit Galois LFSR (Linear Feedback Shift Register) constituting the pseudo random number generator 70. When the 8-bit Galois LFSR starts from an initial value other than 0, the value of 1 to 255 is generated pseudo-randomly once every 254 times of rising of the ck input, and the generated value is output from the terminal Rn. To do. The 8-bit Galois LFSR returns the value of the terminal Rn to the initial value at the 255th rise of the ck input. That is, when the initial state is counted as the 0th time, the value at the 255th time is the same as the initial value.

In FIG. 10, the pseudo random number generator 70 sets the most significant bit 7 to 1 when the reset input R becomes active. Since this is to prevent all bits from becoming 0 as an initial state, any bit other than bit 7 may be set to 1.
Alternatively, for convenience in circuit design, when a set input or reset input is added to a flip-flop other than bit 7 and the reset input R becomes active, all bits are initialized to appropriate fixed values. You may make it.

  In any case, the initial value itself does not affect the effect of the present invention. What is important here is that this pseudo-random number generator makes a round with 255 ck inputs, which will be described later. Since this circuit has been used for the purpose of generating pseudo-random numbers, a detailed description thereof will be omitted.

The probability enable signal generator 73 includes the count value output from the counter 69, the pseudo random number output from the pseudo random number generator 70, a register (not shown) or γ (third predetermined value) supplied from the outside. , And the clock of the terminal 85 are received. The circuit configuration of the probability enable signal generator 73 will be described later.
The probability enable signal generator 73 compares the count value with the γ (third predetermined value), and when the terminal 85 becomes active, increases the internal state variable by 1 based on the comparison result. Decrease by 1. The probability enable signal generator 73 compares the value of the internal variable with the pseudorandom number in the second half of the round, and sets the third control signal En to 1 or 0 based on the comparison result. The probability enable signal generator 73 outputs the third control signal En to the AND gate 67.

The AND gate 67 receives the output signal Zero of the zero detector 68 and the third control signal En of the probability enable signal generator 73. The AND gate 67 calculates a logical product Con of the signal Zero and the third control signal En, and outputs the logical product Con to the 3-input selector 63.
Specifically, the AND gate 67 outputs 1 as a Con signal to the 3-input selector 63 with the probability that the probability enable signal generator 73 is generated when the w _{i, j} output from the memory 65 is 0. . Accordingly, the three-input selector 63 selects the input I1 with the probability, and outputs the fixed value “D0” to the memory 65 when the I1 is selected. That is, when the w _{i, j} output from the memory 65 is 0, the content of the memory 65 is updated to a value other than 0 with the probability, and a new synapse connection is generated.
When the output of the AND gate 67 is 0, the 3-input selector 63 outputs the value output from the 1 increase / decrease circuit 66 to the memory 65.

FIG. 11 is a circuit diagram of a learning signal generation unit of excitatory neurons. When the L1 layer neuron 20 _i is an excitatory neuron, the learning signal generation unit 12 is represented by a learning signal generation circuit 77A in FIG. The learning signal generation circuit 77A includes a probability enable signal generator 71, a probability enable signal generator 72, a 3-input AND gate 74, a 3-input AND gate 75, and an inverter 76.

The probability enable signal generator 71, the probability enable signal generator 72, and the probability enable signal generator 73 have the same circuit configuration, and the circuit configuration will be described with reference to FIG.
FIG. 12 is a circuit diagram of the probability enable signal generation circuit. In FIG. 12, the probability enable signal generation circuit is configured using a magnitude comparator 201, an 8-bit up / down counter 202, and an 8-bit magnitude comparator 203.

  The large / small comparator 201 receives the data of the terminal A and the data of the terminal B. The size comparator 201 outputs 1 to the up / down counter 202 when the data at the terminal B is equal to or greater than the data at the terminal A. The magnitude comparator 201 outputs 0 to the up / down counter 202 when the data at the terminal B is smaller than the data at the terminal A.

Here, the bit width of the large / small comparator 201 is appropriately determined according to the comparison target and the necessary resolution. For example, the probability enable signal generator 71 in FIG. 11 compares the data Σwi _{, j} input to the terminal A with the data α (first predetermined value) input to the terminal B. Σw _{i, j} is the sum of all the weight data w _{i, j} of the synapse connections stored in the memory 65. Since the weight w _{i, j} of the synapse connection is 8 bits and the number of weight data is 256 × 7 × 256, 27 (= 8 + 8 + 3 + 8) bits are required to accurately represent Σw _{i, j} .

However, since the resolution up to that point is unnecessary, only the upper 12 bits of Σwi _{, j} are used here, and the magnitude comparator 201 of the probability enable signal generators 71 and 72 is set to a 12-bit width. Therefore, α (first predetermined value) input to the probability enable signal generator 71 and β (second predetermined value) input to the probability enable signal generator 72 are 15 to the right of the original values. Bit shift, that is, 1 / 32,768.

On the other hand, the probability enable signal generator 73 compares the number of synaptic connection weights w _{i, j} of 0 with γ (a third predetermined value). Since the number of synaptic connection weight data is 256 × 7 × 256, 19 (= 8 + 3 + 8) bits are originally required here, but only the upper 12 bits are used here, and the magnitude comparator 201 is 12 bits wide. And Therefore, γ (third predetermined value) is a value obtained by shifting the original value to the right by 7 bits, ie, 1/128.

When the 8-bit up / down counter 202 receives a reset signal (not shown), the value of the terminal Q, that is, the count value is set to an appropriate value, for example, a half value x′7F ′.
When the output of the magnitude comparator 201 is 1, the up / down counter 202 increments the value of the terminal Q by 1 when the clock input Ne becomes 1, and the magnitude comparator 203 uses the value of the terminal Q as output data p. Output to. However, when the value of the terminal Q of the up / down counter 202 is x'FF ', it is not increased beyond that.

  When the output of the magnitude comparator 201 is 0, the up / down counter 202 decreases the value of the terminal Q by 1 when the clock input Ne becomes 1, and outputs the value of the terminal Q to the magnitude comparator 203 as output data p. Output. However, when the value of the terminal Q of the up / down counter 202 is 0, the value is not further reduced.

  The magnitude comparator 203 receives the data p output from the up / down counter 202 and the pseudo random number Rn output from the pseudo random number generator 70. The magnitude comparator 203 outputs 1 as the output signal En when the data p is greater than or equal to the pseudo-random number Rn. The magnitude comparator 203 outputs 0 as the output signal En when the data p is smaller than the pseudo-random number Rn.

Here, the data p can take an integer value of 0 to 255, and the pseudorandom number Rn can take an integer value of 1 to 255. Therefore, when the data at the terminal Rn changes under the condition that the data p is constant, the value of the output signal En becomes 1 with a probability of p / 255. That is, when the data p increases, the probability that the value of the output signal En becomes 1 increases, and when the data p decreases, the probability that the value of the output signal En becomes 1 decreases.
This is the end of the description of the probability enable signal generation circuit.

The probability enable signal generator 71 receives Σwi _{, j} output from the adder 27 and the input data α (first predetermined value) at the end of the second half of the round. When the signal at the terminal 85 becomes active, the value of the random variable p of the probability enable signal generator 71 is updated according to the magnitude relationship between the Σwi _{, j} and the α (first predetermined value). .
The probability enable signal generator 71 receives the pseudo random number output from the pseudo random number generator 70 in the second half of the round, compares it with the random variable p, and performs the first control with the probability according to the value of the random variable p. The signal En is set to 1, and the first control signal En is output to the AND gate 74.

The probability enable signal generator 72 receives Σwi _{, j} output from the adder 27 and input data β (second predetermined value) at the end of the second half of the round. When the signal at the terminal 85 becomes active, the value of the random variable p of the probability enable signal generator 72 is updated according to the magnitude relationship between the Σwi _{, j} and the β (second predetermined value). .
The probability enable signal generator 72 receives the pseudo random number output from the pseudo random number generator 70 in the second half of the round, compares it with the random variable p, and performs the second control with a probability corresponding to the value of the random variable p. The signal En is set to 1, and the second control signal En is output to the AND gate 75.

The probability enable signal generator 73 receives the count value output from the counter 69 and the input data γ (third predetermined value) at the end of the second half of the round. When the signal at the terminal 85 becomes active, the value of the random variable p in the probability enable signal generator 73 is updated according to the magnitude relationship between the count value and the γ (third predetermined value).
The probability enable signal generator 73 receives the pseudo random number output from the pseudo random number generator 70 in the second half of the round, compares it with the random variable p, and performs the third control with the probability according to the value of the random variable p. The signal En is set to 1, and the third control signal En is output to the AND gate 67.

AND gate 74 has a first control signal En to the probability enable signal generator 71 outputs the output _{s i} of neurons of the L1 layer which memory 24A or 24B is output, the output of the neuron of the L2 layer memory 62 is output b _j .
The AND gate 74 outputs 1 to the 1 increase / decrease circuit 66 as an Inc signal when all of the first control signal En, the s _i, and the b _j are 1. In other cases, the AND gate 74 outputs 0 as the Inc signal to the 1 increment / decrement circuit 66.

The inverter 76 receives the output b _j of the L2 layer neuron output from the memory 62. The inverter 76 generates a signal b ′ _j obtained by inverting the b _j . The inverter 76 outputs the signal b ′ _j to the AND gate 75.

AND gate 75, a second control signal En outputted probability enable signal generator 72, the output _{s i} of neurons of the L1 layer which memory 24A or 24B is output, the signal _b'j the inverter 76 has output receive.
The AND gate 75 outputs 1 to the 1 increase / decrease circuit 66 as a Dec signal when all of the second control signal En, the s _i, and the b ′ _j are 1. In other cases, the AND gate 75 outputs 0 to the 1 increment / decrement circuit 66 as the Dec signal.

<Description of the learning rules of the embodiment>
FIG. 7 shows an example in which the neurons 20 _{i in the} L1 layer are only excitable neurons. Using the technology of the present invention, inhibitory neurons can be implemented in the same way, and both can be mixed. The case where there is an inhibitory neuron in the L1 layer neuron 20 _i will be described later.

  In the present invention, the Hebb algorithm, which is well-known as a learning rule for a neural network, is used by modifying the learning, that is, the change in the weight of the synapse connection. In the Hebb algorithm, there is an input that a neuron in the previous stage has fired at a synaptic connection of a certain neuron, and when that neuron fires, the weight of the synaptic connection is increased, and conversely, the neuron in the previous stage fires. If the neuron does not fire despite the input, the learning is advanced by the operation of reducing the weight of the synapse connection.

In this algorithm, the weight of the synapse connection that happens to have increased in the early stages of learning is likely to increase, so that the learning of the first learning proceeds faster, but the learning of other things gradually becomes difficult. In addition, if learning progresses too much, there is a disadvantage that the learning is excessively learned at the initial stage, and the generalization performance for determining that similar things are the same deteriorates.
Therefore, a probability is introduced into the process of increasing / decreasing the weight, and the probability is increased / decreased by the total sum W of the weights of the synapse connections existing in the certain region.

Specifically, a certain target value α (first predetermined value) is added to the sum W of all synaptic connection weights of all the L2 layer neurons 22 _j existing in one cell of FIG. Value) and β (second predetermined value). The probability enable signal generator 71 gradually increases the probability that the weight of the synapse connection is increased by the Hebb learning rule in a state where the total sum W is smaller than α (first predetermined value). On the other hand, the probability enable signal generator 71 gradually decreases the probability that the weight of the synapse connection increases when the total sum W is larger than α (first predetermined value).

  Further, the probability enable signal generator 72 gradually decreases the probability that the weight of the synaptic connection is reduced by the Hebb learning rule in a state where the total sum W is smaller than β (second predetermined value). On the other hand, in a state where the total sum W is larger than β (second predetermined value), the probability that the weight of the synapse connection is reduced by the Hebb learning rule is gradually increased.

That is, the probability enable signal generator 71 gradually increases the probability that the weight increases when the sum W is smaller than α (first predetermined value), and the probability enable signal generator 72 has the sum W of β (first). If the value is larger than a predetermined value (2), the probability that the weight is reduced is gradually increased.
An external device or register not shown is connected to the terminal B of the probability enable signal generator 71 so that the values of α (first predetermined value) and β (second predetermined value) are α> β. α (first predetermined value) is output, and β (second predetermined value) is output to the terminal A of the probability enable signal generator 72.
Thereby, the total sum W fluctuates in the vicinity of an appropriate value between α (first predetermined value) and β (second predetermined value). This fluctuation mimics the living brain.

  Next, the self-organization algorithm of the present invention will be described. Self-organization is performed by creating a new synapse connection at a place where no synapse connection has existed until then and deleting the existing synapse connection. To create a new synapse connection, probability is introduced as in learning.

  A target value γ (third predetermined value) is set to the number of synaptic connection weights of all the neurons in the L2 layer in one cell of FIG. 2A, that is, the number of unconnected synaptic connections. In the state where the number of unconnected synapses is smaller than γ (third predetermined value), the probability of newly connecting is gradually decreased, and the number of unconnected synapses is γ (third predetermined value). In a state larger than (value), the probability of making a new connection is gradually increased.

When creating a new connection, an appropriate fixed value 'D0' other than 0 is written in the weight memory 65.
On the other hand, deletion of synaptic connections is left to learning. That is, a new synapse connection is generated at random, but once generated, if not used, the weight of the connection gradually decreases and finally disappears. With this algorithm, synaptic connections used for information processing gradually increase, and synaptic connections not used for information processing gradually disappear, and useful neural networks are autonomously constructed.

<Operation of Circuit Unit 10>
Subsequently, the operation of the circuit unit 10 of this embodiment will be described. Before the processing of the circuit unit 10 of the neural network shown in FIG. 7 starts, input data of the L1 layer is written in the memory 21 in advance by an arbitrary method not shown. Further, the input signal to the terminal 83 is set to 1 so that the selector 63 selects the input I2. Thereafter, a predetermined device (not shown) writes the initial value of the synapse connection weight data in the memory 65 in advance through the DB bus.

  Further, when a reset signal (not shown) is input, the up / down counter 202 in the probability enable signal generators 71, 72, 73 is initialized to, for example, x'7F '. The 8-bit counters 20, 23A, 23B, 61 and 19-bit counter 64 all set their count values to zero.

When the calculation starts, the counter 20 outputs the count value 0 to the memory 21. The memory 21 reads 4-bit data from address 0 and outputs it to the pulse train modulator 22. The pulse train modulator 22 converts the 4-bit data into a 15-bit data train.
This conversion is realized by, for example, reading data from the memory in which the data shown in FIG. 8 is written using the data sent from the memory 21 as address information.

  In the first round 0, the pulse train modulator 22 outputs only the LSB of the data shown in FIG. 8 to the memory 24A. Along with this operation, the pulse train modulator 22 outputs the data from the output terminal P0 to the adjacent cell.

  The memory 24A of the adjacent cell receives the data output from the terminal P0. Since the circuit is symmetrical, the memory 24A of its own cell receives data output from the adjacent cell via the input terminals P1 to P6. The memory 24A converts the output data of the pulse train modulator 22 of its own cell and the data input from the terminals P1 to P6 into 7-bit data, and stores the 7-bit data at address 0 of the memory 24A.

Thereafter, the following processing is repeated until the count values of the counter 20 and the counter 23A reach 255.
When the clock signal input to the terminal 41 becomes 1, the counter 20 increases the count value by one and outputs the count value to the memory 21. The memory 21 reads the data at the address of the count value and outputs the data to the pulse train modulator 22. The pulse train modulator 22 converts the data into 15-bit data, and outputs the LSB to the terminal P0.

When the clock signal input to the terminal 42A becomes 1, the counter 23A increases the count value by 1, and outputs the count value to the memory 24A. The memory 24A converts the data from the pulse train modulator 22 of its own cell and the data from the terminals P1 to P6 into 7-bit data, and stores the 7-bit data at the address of the count value in the memory 24A.
In this way, when the processing is performed until the count values of the counter 20 and the counter 23A exceed 255 and return to 0, the round 0 ends.

  In the next round 1, the pulse train modulator 22 outputs the data at bit position 1 in the 15-bit data to the memory 24B.

In this manner, the data stores 24A and 24B alternately receive data, and when the transfer of bit 14 is completed in round 14, the transfer process of the data stored in the memory 21 is completed. When the processing of the circuit unit 10 is continued, an external device (not shown) writes new data in the memory 21 or the memory 21 is formed in a double buffer structure, and in parallel with the processing in the neural network, the external device Are written to the memory 21.
This is the end of the description of the processing of the neuron 20 _i in the L1 layer.

Next, processing of the L2 layer neuron 22 _j will be described. When data is prepared in the memory 24A in round 0, processing in the L2 layer starts from the next round 1. In the L2 layer, the period 4 [ms] of one round is divided into two, the operation of the neurons in the L2 layer is executed in the first half 2 [ms], and learning and self-organization are performed in the second half 2 [ms].
In the first half of the round, since the signal input to the terminal 80 is 0, the OR gate 60 outputs the data s _i output from the memory 24A or the memory 24B to the AND gate 26.

Two subscripts _i of data s _i are originally required. One has a value of 0 to 255 because the number of neurons in the L1 layer is 256, and the other takes a value of 0 to 6 because the total number of cells of its own cell and adjacent cells is 7. However, this makes the notation complicated, and here, it is assumed that the value is set to 0 to 1,791.

In the j-th neuron of the L2 layer, Σ (s _i × w _{i, j} ) is calculated. Here, Σ represents the total sum for i. Since a maximum of 1,792 (= 256 × 7) synaptic connections are formed per neuron in the L2 layer, 1,792 product-sum calculations are performed.

  In addition, there are 256 neurons in the L2 layer. Accordingly, in the entire L2 layer neuron, the product-sum calculation is performed 458,752 (= 1,792 × 256) times. Since these calculations are executed at 2 [ms], the adder 27 and the like operate at 229.4 [MHz].

In the first half of round 1, the memory 24A is in a data read mode. When a clock signal of 32.768 [MHz] is input to the terminal 42A, the counter 23A counts up the count value and outputs the count value to the memory 24A. As a result, the memory 24A reads 7-bit data at 32.768 [MHz]. The memory 24A converts the data into parallel and converts it into a 1-bit signal s _i of 229.4 [Mbps], which is seven times.

At the same time, when a clock signal is input to the terminal 84 at 229.4 [MHz], the 19-bit counter 64 counts up the count value and outputs the count value to the memory 65. As a result, the memory 65 reads the synapse connection weight data w _{i, j} at 229.4 [MHz] and outputs it to the AND gate 26.

The AND gate 26 multiplies the s _i by the w _{i, j} and outputs an 8-bit 0 or w _{i, j} of the multiplication result to the adder 27. The adder 27 calculates the sum of Σ (s _i × w _{i, j} ) for i for each j-th neuron in the L2 layer. At the beginning of the sum calculation, the signal 44 becomes 0, and the data returned from the output of the adder 27 to the one-side input of the adder 27 becomes 0. When the 1,792 sum total calculation for i is completed, the adder 27 outputs the sum calculation result to the magnitude comparator 29.
The magnitude comparator 29 outputs 1 to the memory 62 if the sum calculation result is larger than the threshold θ, and outputs 0 to the memory 62 if the sum calculation result is equal to or less than the threshold θ.
Since the count-up signal input to the terminal 82 is 128 [kHz], the counter 61 counts up the count value at 128 [kHz] and outputs the count value to the memory 62. Therefore, in the first half of the round, the memory 62 receives the data output from the magnitude comparator 29 and sequentially stores the data at the address of the count value.

  In the second half of the round, the memory 62 is in a read mode. Since the counter 61 counts up at 128 [kHz], the memory 62 reads the stored data at 128 [kHz] and outputs the read data to the terminal 46. Therefore, 256 pieces of output data of the neurons in the L2 layer are output at 128 [kbps] in the second half of the round.

<Description of self-organization processing>
Next, self-organization processing (hereinafter referred to as self-organization mode) performed in 2 [ms] in the latter half of the round will be described. Here, self-organization means changing the synaptic connection weight w _{i, j} from 0 to a fixed value 'D0' and changing the synaptic connection weight w _{i, j} from a value other than 0 to 0. is there. The process of changing the weight w _{i, j} of the synapse connection from a value other than 0 to 0 is performed by learning described later. Here, a process of changing the weight w _{i, j} of the synapse connection from 0 to a fixed value “D0” will be described.
In the self-organizing mode, the memory 65 is in a 2-port mode in which data reading and writing operations are performed simultaneously. When a clock signal is input from the terminal 84 at 229.4 [MHz], the 19-bit counter 64 counts up the count value. The 19-bit counter 64 outputs the count value to the memory 65.
The memory 65 reads out one synaptic connection weight w _{i, j} at 229.4 [MHz] and outputs the weight w _{i, j} to the zero detector 68.

The zero detector 68 determines whether or not the received weight w _{i, j} is zero. When the weights w _{i, j} are other than 0, the zero detector 68 outputs 0 (the signal Zero is 0) to the AND gate 67, so that the AND gate 67 outputs 0 to the 3-input selector 63.
When S0 becomes 0, the 3-input selector 63 selects the input I0 as an input signal. As a result, the 3-input selector 63 outputs the data output from the 1 increase / decrease circuit 66 to the memory 65 and controls to change w _{i, j} stored in the memory 65.

When the weights w _{i, j} are 0, the zero detector 68 outputs 1 to the AND gate 67. The pseudo random number generator 70 receives a 229.4 [MHz] clock signal input to the terminal 84. The pseudo-random number generator 70 generates a new pseudo-random number for each synaptic connection weight w _{i, j} and outputs the pseudo-random number to the probability enable signal generator 73.

The probability enable signal generator 73 sets the third control signal En to 1 with a probability corresponding to the state of the internal counter 202, and outputs the signal to the AND gate 67. When the third control signal En is 0, the AND gate 67 sets the output signal Con to 0 and outputs the 0 to the 3-input selector 63.
The 3-input selector 63 receives the 0 and selects the data output from the 1 increase / decrease circuit 66. As already described, when the input / output circuit 66 receives 0 as its input A, its output Y is always 0. Therefore, in this case, the 3-input selector 63 outputs 0 to the memory 65, and the memory 65 writes the 0, so that the weight w _{i, j} remains 0 and does not change.

  When the zero detector 68 outputs 1 and the third control signal En output from the probability enable signal generator 73 is 1, the AND gate 67 sets the output Con to 1 and outputs it to the 3-input selector 63. To do. The 3-input selector 63 selects the input I1 because the input S0 becomes 1, and outputs the fixed value “D0” to the memory 65. The memory 65 writes the received fixed value “D0”.

After all, if Jinapusu coupling weights w _{i, j} is 0, with a probability corresponding to the state of the up-down counter 202 of signal generator 73, the weights w _i that is stored in the memory _{65, j} is a fixed value 'D0 It is rewritten as'. As a result, the value of the weight w _{i, j} is changed from 0 to the fixed value “D0”, and a new synapse connection is formed.

The 19-bit counter 69 has a count value of 0 by a reset signal (not shown) at the beginning of the self-organization mode. The 19-bit counter 69 counts up whenever the signal Zero becomes 1. Therefore, when the self-organization mode ends and all data is read from the memory 65, the 19-bit counter 69 holds the number of weights w _{i, j} stored in the memory 65 being zero.

The 19-bit counter 69 outputs the count value to the probability enable signal generator 73. The probability enable signal generator 73 receives the count value from the 19-bit counter 69 and uses the count value as a signal at the terminal A.
A register or an external device (not shown) outputs a fixed value γ (third predetermined value) to the probability enable signal generator 73. The probability enable signal generator 73 receives a fixed value γ (third predetermined value) from a register or an external device, and uses the fixed value γ (third predetermined value) as a signal at the terminal B.

  At the end of the self-organization mode, the probability enable signal generator 73 receives a data update signal output from an external device (not shown) at a terminal 85, and uses the update signal as an input Ne. The up / down counter 202 in the probability enable signal generator 73 receives the input Ne.

When the up / down counter 202 receives the input Ne and the signal received from the large / small comparator 201 is 1, the up / down counter 202 increments the signal at the terminal Q by 1 and sends the signal at the terminal Q to the large / small comparator 203. Output.
On the other hand, if the up / down counter 202 receives the input Ne and the signal received from the magnitude comparator 201 is 0, the up / down counter 202 decrements the signal at the terminal Q by 1 and sends the signal at the terminal Q to the magnitude comparator. It outputs to 203.
This is the end of the description of the self-organization mode performed in the second half of the round.

<Change of synaptic connection weight>
Next, a process of changing synaptic connection weights (that is, learning in a neural network) (hereinafter referred to as a learning mode) performed in the second half of the round will be described. In the learning mode, the relationship between the outputs b _j of 256 neurons in the L2 layer and the outputs s _i of 1,792 neurons in the L1 layer is comprehensively examined.

In the learning mode, the counter 61 receives a count-up signal of 128 [kHz] output from an external device (not shown) at the terminal 82. Each time the counter 61 receives the signal from the terminal 82, the counter 61 increments the count value by one. The counter 61 outputs the count value to the memory 62. The memory 62 receives the count value from the counter 61, and reads the output b _j of the L2 layer neuron stored at the address of the count value at 128 [kHz].

The memory 62 outputs the read output b _j to the AND gate 74 and the inverter 76.
Inverter 76 receives the output _{b j,} and _b'j inverts the output _{b j.} The inverter 76 outputs the b ′ _j to the AND gate 75.

On the other hand, the memory 24A or the memory 24B reads the stored output data of the L1 layer at 32.768 [MHz], converts the output data into parallel, and seven times 229.4 [Mbps] 1-bit data. Convert to s _i . Memory 24A or memory 24B outputs the _{s i} and the 75 the AND gate 74.

Signal input from the terminal 84 to the ck input of the pseudo-random number generator 70 is the same 229.4 [MHz] as the data transfer rate of the data _{s i} of the memory 24A or the memory 24B is output. Therefore, the probability enable signal generator 71, each time the data _{s i} are input to the AND gate 74, and outputs the new first control signal En to the AND gate 74. Also, the probability enable signal generator 72, each time the data _{s i} are input to the AND gate 75 outputs the new second control signal En to the AND gate 75..

  Here, the first control signal and the second control signal output from each of the probability enable signal generator 71 and the probability enable signal generator 72 are an up / down counter 202 in each signal generator. It becomes 1 with the probability according to the state of.

The AND gate 74 receives the data s _i , the data b _j, and the first control signal En output from the probability enable signal generator 71. When the data s _i is 1, the data b _j is 1, and the first control signal En is 1, the AND gate 74 sets the Inc signal to 1 and outputs the Inc signal to the 1 increase / decrease circuit 66. To do.

The AND gate 75 receives the data s _i , the data b ′ _j, and the second control signal En output from the probability enable signal generator 72. The AND gate 75 sets the Dec signal to 1 when the data s _i is 1, the data b ′ _j is 1 (that is, the data b _j is 0), and the second control signal En is 1. The Dec signal is output to the 1 increment / decrement circuit 66.
Since the data b ′ _j received by the AND gate 75 is the inverted data of the data b _j received by the AND gate 74, the Inc signal and the Dec signal never become 1 at the same time.

The 1 increase / decrease circuit 66 receives the Inc signal and the Dec signal. When the Inc signal is 1, the 1 increment / decrement circuit 66 increments the value of _{wi, j} received from the memory 65 by 1 and outputs the incremented value to the 3-input selector 63.
When the Dec signal is 1, the 1 increment / decrement circuit 66 decrements the value of _{wi, j} received from the memory 65 by 1 and outputs the value decremented by 1 to the 3-input selector 63.

The 3-input selector 63 receives the changed w _{i, j} output from the 1 increase / decrease circuit 66. The 3-input selector 63 selects the changed w _{i, j} and outputs it to the memory 65. The memory 65 writes w _{i, j} after the change.
In this way, the weight w _{i, j} of the synapse connection can be changed by combining the probability with the learning rule of Hebb. Thereby, the so-called learning of the neural network can be realized by the circuit unit 10.

Here, in the learning mode, since the signal input to the terminal 80 is 1, the signal output from the OR gate 60 is always 1, and the AND gate 26 always outputs the data w _{i, j} output from the memory 65.
The signal input to the terminal 44 becomes 0 only once at the beginning of the learning mode. As a result, the output of the adder 27 is initialized to 0 at the beginning of the learning mode.

During the learning mode, the adder 27 calculates Σw _{i, j} . Here, Σ represents the sum total of the weights of all 458,752 synaptic connections. The adder 27 outputs the sum calculation result to the terminal A of the stochastic enable signal generator 71 and the terminal B of the stochastic enable signal generator 72.

  The probability enable signal generator 71 receives α (first predetermined value) from a register (not shown) or an external device and uses it as a signal at the terminal B. Similarly, the probability enable signal generator 72 receives β (second predetermined value) from a register (not shown) or an external device and uses it as a signal at the terminal A.

At the end of the learning mode, a data update signal is output to the terminal 85 from a register (not shown) or an external device. The probability enable signal generator 71 and the probability enable signal generator 72 receive the data update signal and use it as a signal at the terminal Ne.
When the data update signal becomes active, the probability enable signal generator 71 and the probability enable signal generator 72 update the count value of the up / down counter 202 in each of them.

  As described above, the circuit unit 10 shown in FIG. 7 can cause the neural network to perform self-organization and learning using the modified Hebb rule.

  In the processing of the L2 layer, the data reading speed from the memory 65 and the calculation speed of the adder 27 are as high as 229.4 [MHz]. If it is difficult to achieve this operation speed, the memory 65 is divided into, for example, seven 8 [bit] × 256 × 256 memories instead of one 8 [bit] × 256 × 7 × 256 memory. May be. In that case, seven adders 27 and the like may be arranged in parallel, and the operation speed may be set to 32.8 [MHz].

  The 8-bit pseudo random number generator shown in FIG. 10 makes a round when the clock signal is inputted 255 times to its ck input. On the other hand, the number of data stored in the weight memory for synapse connection is 256 × 7 × 256. Accordingly, when one round is over and the next round is entered, the order of pseudorandom numbers is shifted by 7 when the first synaptic connection data is observed.

Since the least common multiple of 255 and 7 is 255 × 7, it takes 255 rounds to return to the original value when they are shifted by 7. That is, when the pseudo-random number generated corresponding to the synapse connection is observed for the first synapse connection data, the values of 1 to 255 appear once every 255 rounds.
The same can be said for all the synaptic connection data, so the weight of all the synaptic connections is increased or decreased with the same probability. When the synaptic connection weight is 0, a new fixed value “D0” is set with the same probability for every synaptic connection weight.

Here, the same probability means that if the value of the up / down counter 202 in the probability enable signal generator is constant, the probability is equal. Actually, the value of the up / down counter 202 is updated every round.
In the embodiment of the present invention, the 8-bit Galois LFSR is used as the pseudo-random number generator. However, the present invention is not limited to this, and other types of pseudo-random number generators such as Fibonacci LFSR can be used. . In the embodiment of the present invention, since the pseudo random number generator is 8 bits, the probability resolution is 1/255. The present invention is not limited to this, and the Galois LFSR can be set to 9 bits, for example, to increase the probability resolution.

  However, it is necessary to pay attention to the relationship between the number of rounds of the pseudorandom number and the number of synaptic connection weight data, and it is desirable that all synaptic connection weight data be changed with the same probability.

<Description of Example 2>
Next, a learning signal generation circuit 77B that generates a learning signal for an inhibitory neuron when an inhibitory neuron exists in the neuron 20 _i in the L1 layer will be described.
First, a case where only inhibitory neurons exist in the L1 layer will be described.

FIG. 13 is a circuit diagram of the learning signal generation unit 12 when the L1 layer neuron 20 _i is an inhibitory neuron.
The inhibitory neuron learning signal generation circuit 77B includes a probability enable signal generator 71B, a probability enable signal generator 72B, an AND gate 74B, an AND gate 75B, and an inverter 76B.

In the synaptic connection of the L2 layer neuron that receives the signal output from the inhibitory neuron 20 _i of the L1 layer, when the L1 layer neuron fires and the L2 layer neuron does not fire, the weight of the synaptic connection is increased. .
Therefore, as shown in FIG. 13, before the data b _j output from the memory 62 is input to the AND gate 74B, the data b _j is inverted.

Inverter 76B receives the signal _{b j} memory 62 has output inverts the data _{b j,} and outputs the obtained by inverting the data _b'j to AND gate 74B.

Conversely, when the inhibitory neuron 20 _i in the L1 layer fires and the neuron 22 _j in the L2 layer fires, the weight of the synapse connection is reduced.
Therefore, as shown in FIG. 13, the data b _j output from the memory 62 is input to the AND gate 75B as it is.

Since the learning speed of the synaptic connection that receives the signal output from the excitatory neuron is different from the learning speed of the synaptic connection that receives the signal output from the inhibitory neuron _, the probability that the weight w _{i, j} of the synaptic connection changes is different. There must be.

  Accordingly, in FIG. 13, the probability enable signal generator 71 </ b> B receives the value δ supplied from an external circuit or register (not shown) at the terminal B. The probability enable signal generator 71B outputs a signal En corresponding to the first control signal to the AND gate 74B. Since the internal processing of the signal probability enable signal generator 71B is the same as the internal processing of the probability enable signal generator 71, description thereof is omitted.

  Similarly, the probability enable signal generator 72B receives a value ε supplied from an external circuit or a register (not shown) at the terminal B. The probability enable signal generator 72B outputs a signal En corresponding to the second control signal to the AND gate 75B. Since the internal processing of the signal probability enable signal generator 72B is the same as the internal processing of the probability enable signal generator 72, description thereof is omitted.

The AND gate 74B receives the data b ′ _j , the data s _i, and the signal En corresponding to the first control signal output from the signal probability enable signal generator 71B. The AND gate 74B calculates a logical product of the data b ′ _j , the data s _i, and the signal En corresponding to the first control signal. The AND gate 74B outputs the logical product as a signal Inc to the terminal I of the 1 increase / decrease circuit.

The AND gate 75B receives the data b _j , the data s _i, and a signal En corresponding to the second control signal output from the signal probability enable signal generator 72B. The AND gate 75B calculates the logical product of the data b _j , the data s _i, and the signal En corresponding to the second control signal output from the signal generator 72B. The AND gate 75B outputs the logical product as a signal Dec to the terminal D of the 1 increase / decrease circuit.

As described above, even when only inhibitory neurons exist in the L1 layer, the weight of the synaptic connection from the neuron 20 _{i in the} L1 layer to the neuron 22 _{j in the} L2 layer is changed using the modified Hebb learning rule. be able to.

<Description of Example 3>
Next, a case where both the excitatory neurons and the inhibitory neurons are included in the neurons 20 _{i in} the L1 layer will be described. FIG. 14 is a circuit diagram of the learning signal generation unit 12 when both the excitatory neuron and the inhibitory neuron are included in the L1 layer neuron 20 _i .

 The mixed learning signal generation circuit 78 includes an excitatory neuron learning signal generation circuit 77A, an inhibitory neuron learning signal generation circuit 77B, a selector 79A, and a selector 79B.

The selector 79A includes an output signal IncA of the AND gate 74 of the excitatory neuron learning signal generation circuit 77A, an output signal IncB of the AND gate 74B of the inhibitory neuron learning signal generation circuit 77B, and a suppression supplied from a circuit (not shown). The sex neuron identification signal INH _i is received. The inhibitory neuron identification signal INH _i becomes 0 when the neuron 20 _{i in the} L1 layer is excitable and becomes 1 when it is inhibitory.
Selector 79A, the identification signal INH _i is in the case of 0, and outputs to the terminal I 1 decrease circuit the signal IncA as Inc signals. Selector 79A, the identification when the signal INH _i is 1, and outputs the signal IncB to terminal I 1 decrease circuit as Inc signals.

The selector 79B receives the output signal DecA of the AND gate 75 of the excitatory neuron learning signal generation circuit 77A, the output signal DecB of the AND gate 75B of the learning signal generation circuit 77B of the inhibitory neuron, and the identification signal INH _i. .
Selector 79B, the identification signal INH _i is in the case of 0, and outputs to the terminal D of 1 increasing or decreasing circuit of the signal DecA as Dec signal. Selector 79B, the identification when the signal INH _i is 1, and outputs to the terminal D of 1 increasing or decreasing circuit of the signal DecB as Dec signal.

As described above, even if excitatory neurons and inhibitory neurons are mixed in the L1 layer, the weights w _{i, j} of the synaptic connections from the neurons 20 _{i in the} L1 layer to the neurons 22 _{j in the} L2 layer are transformed into the Hebb Can be changed using the following learning rules.

In the present embodiment, a so-called pulse neural network in which information is transmitted from the L1 layer to the L2 layer by a pulse train has been described. However, the present invention is not limited to this, and the same applies to a neural network that transmits information by a plurality of bits. it can.
In the embodiment of the present invention, the bit width of the comparator 201 in the probability enable signal generation circuit is set to 12. However, the present invention is not limited to this, and a comparator having an arbitrary bit width can be used depending on the necessary resolution. Can be used.

In the embodiment of the present invention, the initial value of the up / down counter 202 in the probability enable signal generation circuit is x'7F ', but the present invention is not limited to this, and an arbitrary initial value can be set.
In the embodiment of the present invention, the number of neurons included in one cell is 256 in both the L1 layer and the L2 layer. However, the present invention is not limited to this, and one or more neurons exist in each layer. The same applies. Further, the number of neurons in the L1 layer and the L2 layer need not be the same.

In the embodiments of the present invention, the feedforward type neural network has been described. However, the present invention can be similarly applied to a neural network having a feedback loop.
In the embodiment of the present invention, the neural network has a two-layer structure, but the present invention is not limited to this, and can be applied to each layer even if it is expanded to three or more layers.

 In the embodiment of the present invention, a virtual space is considered, a regular hexagon is disposed therein, and a neuron of each layer is divided and disposed therein, but the present invention is not limited to this, The present invention can be similarly applied to a configuration in which a multi-dimensional space of two or more dimensions is considered, an arbitrary polyhedron is arranged therein, and neurons of each layer are divided and arranged therein.

 In the embodiment of the present invention, α (first predetermined value), β (second predetermined value), γ (third predetermined value) (when inhibitory neurons are mixed, δ and ε are also included. However, the present invention is not limited to this, and these values are larger than the time constant for learning and self-organization. It may be a variable that varies with a time constant.

 For example, a 12-bit counter that counts down at a very slow frequency (for example, 0.0001 Hz) is prepared, and its output is γ. By setting the initial value of this counter to a value close to x'FFF ', creating a state in which almost all synaptic connections become effective immediately, and then proceeding with learning while gradually decreasing γ, only the effective signal transmission path Can be quickly promoted.

<Effect of the present invention>
By using the technology of the present invention, a large-scale neural network can be constructed, and self-organization and learning using the modified Hebb rule can be performed at a speed equivalent to the operation speed of a biological neural network. . The generation of new synaptic connections in self-organization is controlled by probability. This probability increases when the number of unconnected synapse connections is greater than a certain set value γ, and decreases when the number of unconnected synapse connections is less than a certain set value γ. Accordingly, the number of unconnected synaptic connections fluctuates around the set value γ, and a structure close to the brain of a living body is obtained.

 In the learning using the modified Hebb rule, the sum of the weights of synapse connections in a certain region (one cell in the embodiment) is calculated, and this value is set as a set value α (first predetermined value) and Compared with β (second predetermined value), the probability of increasing or decreasing the weight of the synapse connection is increased or decreased depending on the magnitude relationship, so the stability of the neural network increases and generalization by excessive learning It has the effect of preventing a decline in ability.

 In the technique of the present invention, values such as α (first predetermined value), β (second predetermined value), and γ (including δ and ε when inhibitory neurons are mixed) are set for all cells. This information is supplied in common. Also, all the information necessary for self-organization and learning is completed within a certain area (in a cell). That is, since there is no need to exchange information between cells, even if the number of cells is increased and the scale of the neural network is increased, information communication between cells does not become a problem.

 Accordingly, in principle, any number of large neural networks can be constructed by combining with the wiring between cells shown in FIG. 2B, and the neural network can be self-assembled at a speed equivalent to the operation speed of the actual neural network. It can have the function of organizing and learning.

As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

CN cell (polytope)
DESCRIPTION OF SYMBOLS 1 1st output part 2 Input part 3 Weight storage part 4 Multiplication part 5 Addition part 6 Comparison determination part 7 2nd output part 8 Weight change part 10 Circuit unit 11 Pseudorandom number generation part 12 Learning signal generation part 13 Weight increase / decrease part 14 synapse creation command unit 15 switching unit 20 8-bit counter 21 memory 22 pulse train modulation circuit 23A 8-bit counter 23B 8-bit counter 24A memory 24B memory 26 AND gate 27 adder 28 AND gate 29 large / small comparator 60 OR gate 61 counter 62 memory 63 3-input selector 64 19-bit counter 65 Memory 66 1 Increase / decrease circuit 67 AND gate 68 Detector 69 19-bit counter 70 Pseudo random number generator 71 Probability enable signal generator 72 Probability enable signal generator 73 Probability enable signal generator 74 AND gate 74B AND gate 75 AND gate 75B AND gate 76 Inverter 76B Inverter 77A Excitatory neuron learning signal generation circuit 77B Inhibitory neuron learning signal generation circuit 78 Mixed learning signal generation circuit 79A Selector 79B Selector 101 x'FF 'detector 102 x'00 'detector 103 3-input AND gate 104 AND gate 105 OR gate 106 8-bit adder / subtractor 201 magnitude comparator 202 up / down counter 203 8-bit magnitude comparator

Claims (8)

A plurality of layers, a plurality of artificial neurons existing in the layer, and a synaptic connection that connects the artificial neuron to another layer of artificial neurons. An electronic circuit that multiplies the output data by the weight information of the synapse connection and outputs it to a second layer artificial neuron, and realizes an information processing neural network in the second artificial neuron,
In a virtual space of two or more dimensions, an arbitrary number of arbitrary polytopes are arranged in the virtual space, and signals are transmitted between the circuit units arranged for each polytope and the circuit units arranged in adjacent polytopes. Wiring to be provided, and
The circuit unit outputs the output data of the first layer artificial neuron to the second layer artificial neuron in the same circuit unit and the second layer artificial neuron arranged in the polytope adjacent to its own polytope. Output data of the artificial neuron of the first layer from a first output unit that outputs to a neuron, a circuit unit arranged in its own polytope, and a circuit unit arranged in a polytope adjacent to its own polytope A receiving input unit; a weight storage unit holding weight information of a synapse connection for determining a ratio of transmitting information from the first layer artificial neuron to the second layer artificial neuron; and the first layer artificial A multiplier for calculating a product of neuron output data and the synaptic connection weight information; an adder for calculating a sum of the products; and a sum of the products and a predetermined value Based on the comparison result with the threshold value, a comparison / determination unit that determines output data of the second layer of artificial neurons, a second output unit that outputs the output data of the second layer of artificial neurons, and the second layer An electronic circuit comprising: a weight changing unit that changes weight information of the synaptic connection based on output data of the artificial neuron of the first layer and output data of the artificial neuron of the second layer.   The adding unit calculates a sum of weight information of all the synapse connections of the artificial neurons of the second layer for each circuit unit arranged in the polytope, and the weight changing unit calculates the sum of the weight information. 2. The electronic circuit according to claim 1, wherein weight information of the synapse connection is changed for each of the circuit units so as to be in a range between a predetermined value of 1 and a second predetermined value. The weight changing unit includes a pseudo random number generating unit that generates a pseudo random number for each synapse connection, and a learning signal generating unit that determines increase or decrease of the weight information,
The learning signal generation unit is configured to store a first reference value as a criterion for determining whether to increase the weight information of the synapse connection, and to decrease the weight information of the synapse connection. A second storage unit that stores a second reference value serving as a determination criterion, and when the sum of the weight information is equal to or less than the first predetermined value, the first reference value is increased, A first reference value changing unit that decreases the first reference value when a sum of weight information is larger than the first predetermined value; and a sum of the weight information is greater than or equal to the second predetermined value A second reference value changing unit that sometimes increases the second reference value and decreases the second reference value when the sum of the weight information is smaller than the second predetermined value; 1 based on the comparison result of the reference value of 1 and the pseudo-random number, the weight of the synapse connection A first magnitude comparison unit that generates a first control signal that permits addition, and a second magnitude that permits a decrease in weight of the synapse connection based on a magnitude comparison result between the second reference value and the pseudo-random number. A second magnitude comparator for generating a control signal, wherein the learning signal generator is output data of the first layer of artificial neurons, output data of the second layer of artificial neurons, and the first layer 3. The electronic circuit according to claim 2, wherein increase / decrease in weight information of the synapse connection is determined based on one control signal and the second control signal.   The weight change unit changes the value of the weight information of the synapse connection from 0 to a predetermined value when the value of the weight information of the synapse connection is zero. An electronic circuit according to the above.   The weight changing unit counts the number of synaptic connection weight information whose value is 0 for each circuit unit arranged in the polytope, and the number of 0 falls within a predetermined range. 5. The electronic circuit according to claim 4, wherein the value of the synaptic connection weight information is changed from 0 to a predetermined value. The weight changing unit includes a third storage unit that stores a third reference value serving as a determination criterion for determining whether or not to change the weight information of the synapse connection from 0 to a predetermined value, and a pseudo-random number that generates a pseudo-random number. A random number generator,
The weight changing unit increases the third reference value when the number of synaptic connection weights equal to or smaller than a third predetermined value, and the number is greater than the third predetermined value. Sometimes the third reference value is decreased,
The pseudo-random number generator generates a pseudo-random number for each synapse connection, and changes the value of the synapse connection from 0 to a predetermined value based on a comparison result between the third reference value and the pseudo-random number. A third control signal that permits
The weight changing unit, when the synaptic connection weight information is 0, changes the synaptic connection weight information from 0 to a predetermined value based on the third control signal. 5. An electronic circuit according to 5. The weight changing unit includes a pseudo random number generating unit that generates pseudo random numbers,
The weight changing unit counts the number of synaptic connection weight information whose value is 0 for each circuit unit arranged in the polytope, and based on a comparison between the number of 0 and the pseudorandom number 5. The electronic circuit according to claim 4, wherein the weight information of the synapse connection is changed from 0 to a predetermined value.   The synaptic connection weight information which does not require input of output data of the first-layer artificial neuron is set to 0 in the synaptic connection of the second-layer artificial neuron. The electronic circuit according to claim 1.

Images