JP2690702B2 - Self-organizing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a self-organizing device having a neural network structure, and more particularly to a self-organizing device having a network structure with cells having neighbor connections.

[0002]

2. Description of the Related Art Conventionally, this type of neural network learning method has been used for self-organizing the structure of a neural network. As an example of a conventional neural network learning method, an artificial intelligence society research group material JSAI Technical Report SI
G-PPAI-9401-6 (4/9) p. 32-40, "CAM Brain Project: Genetic Programming of Artificial Brain consisting of 1 billion Neurons Growing and Evolving at Electronic Speed Using Cellular Automata (The "CAM-
Brain ”Project: The Genetic
Programming of a Billion
Neuron Artificial Brain w
hitch Grows / Evolves at Ele
ctronic Speed ina Celluia
r Automata Machine) "is known.

The learning method described in the above material uses a two-dimensional or three-dimensional cell auto machine. Each cell changes the next state according to the states of neighboring cells and its own state based on the rules of the state transition table. The structure of the neural network is coded as a gene (a signal sequence given from the outside). By driving the states of the output end cell and the input end cell of the neuron with the pattern of this gene, the cellular automaton changes the state and , Axons and dendrites grow, and the neural network structure self-organizes. How axons and dendrites grow is completely determined by the gene patterns prepared in advance. In order for the structure of the completed neural network to function as a neural network, the state transition table used for self-organization is switched to another one. After that, the performance when functioning as a neural network was evaluated, and the gene with the highest evaluation was evaluated by the genetic algorithm GA (Genetic Al).
better neural network structure.

[0004]

SUMMARY OF THE INVENTION This conventional neural network learning method has the following problems in order to obtain a neural network structure. Coding of gene pattern, 2. 2. Self-organization of neural network structure using the above genes. Evaluation of the completed neural network function, 4. The steps of changing the genetic pattern by GA need to be repeated until sufficient function is obtained. In the above step 2, the structure of the neural network previously coded in the gene pattern is self-organized so that the neural network is adapted to the problem area (for example, pattern recognition, information processing, etc.) to be used. There is no purposeful self-organization. Also, in step 3, the function of the completed neural network structure is evaluated. Here, we have to evaluate according to the problem area,
In a complicated system, it takes a lot of time to evaluate it. For the pattern of the gene expressing the highly evaluated neural network structure, in step 4, G
Operation A applies.

As described above, in the conventional method, the pattern of the gene has to be corrected for many generations until the required function is obtained, and in the large-scale and complicated neural network, There is a problem that learning of a large-scale neural network is difficult because the time required to evaluate the function becomes longer depending on the scale.

Further, in the conventional neural network learning method, since each cell is realized as a cellular automaton, the information propagation speed is a distance of at most one cell per one state update (one clock). Due to the self-organization of the neural network structure, even if the neurons were once connected, the signal propagation was one cell per clock, so that it was difficult to realize a high-speed neural network.

It is an object of the present invention to provide an efficient learning method of a neural network structure according to a problem area and a signal propagation method for realizing a high speed neural network.

[0008]

In order to solve the above problems, according to the present invention, a cell provided with an input control unit, an output control unit, an arithmetic processing unit, and an internal state storage unit is provided as the input / output control unit. In the self-organizing device of the neural network structure, which connects a plurality of cells through each cell and changes the internal state of itself in accordance with the internal states of neighboring cells to perform self-organization, each cell is Accordingly, it functions as four different cells. That is, as cell types, (a) a glial cell that functions as a field for self-organization of a neural network structure, (b) a neuron cell that takes charge of the function of the neuron body, and (c) a tree structure having the neuron cell as a root An element is a search cell that functions as an information propagation path between the neuron cell and the synapse cell; (d) a synapse cell that transmits information between the tree structures having different neuron cells as roots and is responsible for the function of weighted connection; It was set. Then, the neuron cell and the search cell issue a germination request to the adjacent glial cell,
The glial cells were adapted to function as search cells in response to the germination request. The above operations are performed under the control of the arithmetic processing unit in the cell.

[0009]

[Function] In the field of glial cells, a search cell extends from a neuron cell to form a tree structure, and tree structures of different neurons are connected via synapse cells. In this way, the neural network is efficiently self-organized.

[0010]

Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overall configuration diagram showing an embodiment of the present invention. Although FIG. 1 shows an example of cells connected in a two-dimensional mesh for simplification, the following description is the same even in the case of a generally multidimensional cell.

There are three types of cells: glial cell 3, neuron cell 4, search cell 5, germination cell 6, and synapse cell 7, and the function differs depending on the cell type. Each cell has an internal state according to the type. Also, each cell can refer to the internal state of the cell in the vicinity of Neumann. Each cell changes its cell type and internal state according to its own state and the internal states of neighboring cells.

FIG. 2 is a diagram for explaining the connection between cells and neighboring cells. Each cell has 4 pairs (6 pairs in the case of 3 dimensions) for connecting to the cells on the top, bottom, left and right sides (in case of 2 dimensions).
Has an input port 21 and an output port 22. The input port of one cell and the output port of another cell are connected by a plurality of connecting lines, but in FIG. 2, one connecting line is shown for simplification.

FIG. 3 is a block diagram for explaining a general structure of the cell 2. The cell 2 includes an input control unit 23, an arithmetic processing unit 24, an output control unit 25, an internal state storage unit 26, and a cell type storage unit 27. The arithmetic processing unit 24 is composed of, for example, a microprocessor and a memory, or an FPGA (Field Programmable Gate Array) whose function can be changed by a program.
Other configurations may be used. The cell type storage unit 27 stores the cell type (one of glial, neuron, search, germination, and synapse). Although all the cells have the same structure as shown in FIG. 3, the operation of the arithmetic processing unit 24 differs depending on the type of the cell, and its operation program is stored in the memory in the arithmetic processing unit 2. That is, the cell 2 functions as the neuron cell 4, the search cell 5, the germination cell 6, the synapse cell 7, or the glial cell 3 depending on the cell type.

The neuron cell 4 receives the output from another neuron cell 4 via the neighboring search cell 5 and changes the activity and the output, and is responsible for the function of the neuron body. The search cell 5 functions as a propagation path of information between the neuron cell 4, the germination cell 6, and the synapse cell 7. The germination cell 6 is the tip of the tree structure formed by the search cell 5 with the neuron cell 4 as the root. Synapse cell 7
Transmits information between the search cell 5 and the germination cell 6 having different neuron cells 3 as roots, and is responsible for the function of weighted connection.

The glia cell 3 does not participate in the function itself of the neural network, but functions as a field for self-organizing the structure of the neural network.
FIG. 4 is a block diagram illustrating the function of the glial cell 3. The glial cell 3 has a proximity counter 361, a Δp register 362, and a potential p register 363 in its internal state storage unit 36. The proximity counter 361 holds the number of cells in the vicinity of the glial cell 3, and the potential p register 363 holds the value of the potential of the glial cell 3 itself. The arithmetic processing unit 34 sums up the values of the potentials of the neighboring cells, divides the total value by the number of neighboring cells to obtain an average value of the potentials of the neighboring cells, and calculates the difference between the average value and the potential of the glial cell 3 itself. Let Δp be Δp
It is stored in the register 362. Further, the value of the potential p register 363 is changed using the value of the Δp register 362 and the value of the potential p register 363.

The neuron cell 4, the search cell 5, and the germination cell 6 determine the direction of germination according to the potential value of the adjacent glial cell 3 and probabilistically (with a certain probability) issue a germination request to the adjacent glial cell 3. The reason why the germination request is probabilistically issued is that the probability of the germination request is changed so that the rate of self-organization can be controlled. The germination cell 6 is formed at the tip of the search cell 5 and issues a germination request with a higher probability than the search cell 5. The germination cell 6 is not an essential requirement of the present invention, but when the germination cell 6 is provided as in the embodiment, the tree structure grows from the tip portion of the search cell 5 with a higher probability, so that the self-organization is efficiently performed. Can be converted.

The value of the potential of the neighboring cell depends on the type of the cell. In the case of the glial cell 3, it is the value of the potential p register 363, in the case of the neuron cell 4, the output value, in the case of the search cell 5 it is zero, and in the case of the germination cell 6, it is the output value of the neuron cell 6 at the root of the germination cell 6. .
In FIG. 4 and FIG. 5 described next, the data signal line and the control signal line for the cell type storage unit are omitted.

Potential p-register 3 of glial cell 3
The value of 63 changes according to the potential of the neighboring cells,
A potential field is created on an array of cells connected in a two-dimensional (or three-dimensional) mesh. A potential field spreading around the neuron cell 4 having a large output and the germination cell 6 having the neuron cell 4 having a large output as a root is formed. The neuron cell 4, the search cell 5, and the germination cell 6 stochastically request germination in the direction in which the potential value is large. Therefore, there is a high probability that the search cell 5 and the germination cell 6, which are rooted at the neuron cell 4 having a large output, are close to each other. A search cell 5 having different neuron cells 4 as roots and a glial cell 3 having a germination cell 6 in its vicinity change to a synapse cell 7 to form a signal propagation path between different neuron cells 4.

FIG. 5 is a block diagram for explaining the input / output signal exchange function of the search cell 5. The internal state storage unit 56 has a root direction register 561 and a germination direction register 562. The search cell 5 is adjacent to two or more search cells 5 or germination cells 6. One is the search cell 5 that reaches the root neuron cell 4, and the rest is the search cell 5 or the germination cell 6 that germinated from this cell. Root direction register 561
Direction of the search cell 5 to the root, germination direction register 56
2 stores the directions of the search cell 5 and the germination cell 6 germinated from this cell. The input control unit 53 is the root direction register 5
The output of the adjacent search cell 5 is selected and fetched according to the value of 61. Further, the output control unit 55 determines that the search cell 5 germinated from this cell according to the value of the germination direction register 562,
The value taken into the germination cell 6 is output. At this time, the arithmetic processing unit 54 forms a path that directly connects the input control unit 53 and the output control unit 55. In a normal cellular automaton, information can be propagated for only one cell per one state update (one clock), but as described above, the signal propagation path from the input port 51 to the output port 52 has a root direction register 561. Since the input side and the output side are directly connected by setting according to the value of the germination direction register 562, the signal is propagated at high speed only by the delay time of the multiplexer MUX of the input control unit 53 and the gate of the output control unit 55. It becomes possible to do.

6 to 9 show how the self-organization of the neural network structure progresses. FIG. 6 shows the initial state of self-organization, in which only neuron cells 4 and glial cells 3 are present. FIG. 7 shows a state immediately after the start of self-organization, in which the glial cell 3 adjacent to the two neuron cells 4 is the germination cell 6 and the search cell 5.
FIG. 8 shows a state in which self-organization progresses further from FIG. 7, and a tree structure with the neuron cell 4 as a root grows, but there is no synapse cell 7 yet and a signal is output between the two neuron cells 4. There is no route yet. FIG. 9 shows a state further advanced from FIG. 8, in which a synapse cell 7 is formed between tree structures having two neuron cells 4 as roots, and two neuron cells 4 are formed.
A signal transmission path has just been created between them. When self-organization further proceeds, the neural network structure shown in FIG. 1 is obtained.

FIG. 10 is a flow chart showing the operation of the glial cell, and FIG. 11 is a flow chart showing the operation of the neuron cell, the search cell and the germination cell.

The operation of the glial cell will be described with reference to FIG. 10. First, the average AV of potentials near the glial cell is obtained (step 101). As described above, in FIG. 4, the arithmetic processing unit 34 calculates the average value AV by summing the potentials of neighboring cells and dividing by the number. And from the average value, the potential P of the glial cell itself
To obtain Δp (102), and the Δp is α (0 <α
<1) Multiply and add to the original potential P to obtain a new potential value P (103).

Next, a neuron cell near the glial cell,
Check the number n of germination cells and search cells (step 104),
If the number of neighboring cells is one and the glial cell has a germination request (step 105), the cell type is stochastically changed from the glial cell to the germinated cell (step 106). This operation is, for example, an operation in which the glial cell 3 adjacent to the neuron cell 4 in FIG.

In step 104, if the number of neighboring cells is 2, it is checked whether or not these two cells are rooted in different neurons (107), and if so, the cell type is changed from glial cell to synaptic cell. Do (1
08). This operation is an operation performed by the synapse cell 7 in FIG. 9, for example.

In the cases other than the above, the cell type of the glial cell is not changed and the above operation is repeated.

Next, the operation of the neuron cell, the search cell, and the germination cell will be described with reference to FIG. 11. First, the potentials of the neighboring glial cells are checked to find the direction D of the glial cell having the maximum potential (step 201). . Then, a germination request is probabilistically issued to the glial cells in the D direction. This operation is, for example, an operation of forming the germination cell 6 as shown in FIG. As described above, the potential of the glial cell 3 is determined as shown in steps 101 to 103 of FIG. 10, and the potential of the glial cell 3 around the neuron cell 4 and the germination cell 6 becomes relatively high. The glial cells 3 around the cells 6 are transformed into germination cells 6.

Next, it is checked whether or not the glial cells in the D direction have changed to germination cells (step 203), and if they have changed, the direction D is registered in the germination direction register (FIG. 5). Although the germination direction register has already been described as a constituent part of the search cell, the neuron cell and the germination cell also have the germination direction register. Next, in step 205, it is determined whether the cell is a germination cell and if it is a germination cell, the cell type is changed and the state is changed to the search cell.

In this way, the output of the neuron cell propagates through the glial cell as a medium to form a potential field, and the search cell is germinated from the neuron cell in the direction of the larger potential. The probability that a large neuron can be connected becomes high, and efficient learning of the neural network structure according to the problem area becomes possible. Moreover, by setting the signal propagation paths of the input and output of the search cell according to the values of the root direction register and the germination direction register, signal propagation for realizing a high-speed neural network becomes possible.

As described above, according to the present invention,
By using the glial cell that functions as a place for self-organization, it is possible to provide an efficient learning method of the neural network structure according to the problem area.

[Brief description of the drawings]

FIG. 1 is a block diagram of an overall configuration of a neural network learning method according to the present invention.

FIG. 2 is a diagram for explaining connection with neighboring cells.

FIG. 3 is a block diagram illustrating a general configuration of a cell.

FIG. 4 is a block diagram illustrating the function of a glial cell.

FIG. 5 is a block diagram illustrating an exchange function of input / output signals of a search cell.

FIG. 6 is a diagram showing an initial state of self-assembly.

FIG. 7 is a state diagram immediately after the start of self-assembly.

FIG. 8 is a diagram showing an intermediate state of self-assembly.

FIG. 9 is a diagram showing a state immediately after a synapse is formed.

FIG. 10 is a flowchart illustrating an operation of a glial cell.

FIG. 11 is a flowchart illustrating the operations of a neuron cell, a search cell, and a germination cell.

[Explanation of symbols]

 1 mesh-like neighborhood connection cell (processor) group 2 cell 21 input port 22 output port 23 input / output control unit 24 arithmetic processing unit 25 output control unit 26 internal state storage unit 27 cell type storage unit 3 glial cell 33 glial cell input control 34 glial cell Calculation unit of 35 Gliacell output control unit 36 Gliacell internal state storage unit 361 Neighborhood counter 362 Δp register 363 Potential p register 37 Gliacell cell type storage unit 4 Neuron cell 5 Search cell 51 Search cell input port 52 Search cell Output port 53 Search cell input control unit 54 Search cell arithmetic processing unit 55 Search cell output control unit 56 Search cell internal state storage unit 561 Root direction register 562 Germination direction register 57 Search cell cell type storage unit 6 Germination cell 7 Synapse cells

Claims (4)

(57) [Claims] 1. An input control unit, an output control unit, an arithmetic processing unit,
A neural network in which a plurality of cells each having an internal state storage section are connected through the input / output control section, and each cell changes its own internal state according to the internal states of neighboring cells to perform self-organization. In the self-organizing device having a network structure, the arithmetic processing unit determines, according to the state of each cell, (a) a glial cell that functions as a field for self-organizing the neural network structure, and (b) functions of the neuron body. (C) a search cell that is a component of a tree structure having the neuron cell as a root and functions as an information propagation path between the neuron cell and a synapse cell; (d) the tree having a different neuron cell as a root The neuron cell and the search cell are made to function as synapse cells, which transfer information between structures and take charge of the function of weighted connection. The self-organizing apparatus, wherein a germination request is issued to the adjacent glial cell, and the glial cell changes its function to a search cell in response to the germination request. 2. The self-assembling device according to claim 1, wherein the arithmetic processing unit replaces the cells with the cells (a) to (d).
In addition to (e), a self-organizing device that is formed at the tip of the search cell and can function as a germination cell that issues a germination request with a higher probability than the search cell. 3. The self-organizing apparatus according to claim 2, wherein the glial cell has a potential register, the value of the potential register changes according to the potential of the neighboring cells, and each cell is two-dimensional or three-dimensional. A potential field is created on a mesh-shaped array connected in a mesh shape, and the neuron cell, the search cell, and the germination cell stochastically request the germination in the direction in which the potential value is large. Self-organizing device. 4. The self-organizing apparatus according to claim 1, wherein the search cell has a root direction register and a germination direction register, and the input / output control unit includes the root direction register. Self-organization characterized in that an input / output port is selected according to the value of a register and the germination direction register, and the arithmetic processing unit forms a signal propagation path that directly connects the input control unit and the output control unit. apparatus.