JPH04307654A - Neural network simulation system - Google Patents

Abstract

PURPOSE:To attain the bidirectional transfer of messages via a link processor and at the same time to carry out the simulation with high efficiency in order to attain the bidirectional communication between nodes in a neural network of a large scale and an irregular structure. CONSTITUTION:When the messages are forward transferred to an input node from an output node, a node processor 1 refers to an output link table to send the node messages to plural link processors 2. The processors 2 apply the link processing to the received messages and sends them to an input node processor 1 by reference to an input node identifier. When the messages are backward transferred to the output node from the input node, the processor 1 sends the node messages to plural processors by reference to an input link table. Then the processors 2 apply the link processing to the received messages end sends them to an output node processor by reference to an output node identifier.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to neural network simulation systems.

0002

[Prior Art] Conventionally, the weights of the links in a neural network are expressed in a matrix, divided and assigned to multiple processors, and processed in parallel by the multiple processors, and the node processor that simulates each node is There is a known method in which a plurality of output links are stored as an output link table storing pairs of link weights and destination node processor identifiers, and the link processing is performed by the output side node processor.

0003

[Problem to be Solved by the Invention] However, the method of expressing the weights of links in a neural network in a matrix and dividing the weights is not efficient when the neural network to be simulated has a regular structure such as a fully connected or layered structure. However, in large-scale, irregularly structured neural networks, most of the matrix elements are zero, resulting in sparse matrices, resulting in unnecessary calculations and increased storage capacity.

[0004] Furthermore, in a system in which a pair of the weight of each node processor link and the destination node processor identifier is stored in the form of an output link table, it is difficult to transfer messages bidirectionally between nodes via a single link. there were. It is also conceivable to have a similar pair of link weights and node processor identifiers for input links in the form of an input link table, but in this method, the entity of the link weight is the output link,
Since the input link is stored in two locations, there is a drawback that the link update process during learning becomes complicated.

An object of the present invention is to efficiently perform a simulation involving bidirectional communication between nodes, which is required in many learning algorithms, in a large-scale, irregularly structured neural network.

[0006]

[Means for Solving the Problems] The present invention provides a system for simulating a neural network in which nodes exchange messages with each other via weighted links and change their internal states and outputs according to the content of the messages; a plurality of node processors simulating functions of a weighted link; a plurality of link processors simulating a function of weighted links; and a communication path for the node processors and the link processors to communicate with each other; Each link processor has a different processor identifier,
and a storage device that stores the states of the nodes and links, a simulation processing device that simulates the functions of the nodes and links, and a communication device that performs communication processing with the other node processor and the link processor, respectively. a storage device of the node processor maintains an output link table holding an identifier of the link processor simulating a plurality of output links going out of the node and an identifier of the link processor simulating a plurality of input links entering the node; the storage device of the link processor stores an input link table for simulating an output node that sends a message via a link, and an internal state of the node; It stores the identifier of the node processor that simulates the node processor and the weight of the link; and has a bidirectional message transfer function between the node processors via the link processor.

[0007]

[Operation] In the present invention, in forward message transfer from an output node to an input node, a node processor refers to an output link table and sends a node message to a plurality of link processors. The node processor performs link processing on the received message and sends it to the input node processor by referring to the input node identifier, and when transferring the message in the reverse direction from the input node to the output node, the node processor refers to the input link table. The node message is sent to multiple link processors, and the link processor performs link processing on the message, refers to the output node identifier, and sends the message to the output node processor. Bidirectional message transfer can be achieved. Further, even in a large-scale, irregularly structured neural network, it is sufficient to have input link tables and output link tables for only the necessary links, and unnecessary calculations and storage capacity are not required.

As described above, it is possible to efficiently execute a simulation involving bidirectional communication between nodes even in a large-scale, irregularly structured neural network.

[0009]

[Embodiment] Next, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is an overall configuration diagram of the system, which is composed of a plurality of node processors 1, a plurality of link processors 2, and an inter-processor communication path 3. The node processor 1 includes a storage device 11 and a simulation processing device 12.
and a communication device 13. The link processor 2 includes a storage device 21, a simulation processing device 22, and a communication device 23. Each node processor 1 and link processor 2 has a different processor identifier and is distinguished from other processors. Each processor 1,
2 communicates with other processors 1, 2 via the inter-processor communication path 3.
You can send a message to 2. The destination of the message is specified by the identifier of the destination processor.

FIG. 2 is a detailed diagram of the node processor 1, in which the storage device 11 stores output links that hold identifiers of a plurality of link processors 2 that simulate a plurality of output links going out from a node simulated by this node processor 1. Table 14 and multiple link processors 2 that simulate multiple input links going to nodes
The input link table 16 holding the identifier of the node and the internal state 15 of the node are stored.

FIG. 3 is a detailed diagram of the link processor 2, in which the storage device 21 contains an identifier 24 of the node processor 1 that simulates an output node that sends a message via a link, and an identifier 24 that simulates an input node that receives a message via the link. The identifier 26 of the node processor 1 and the link status 25 are stored.

In the above configuration, in forward message transfer from an output node to an input node, the node processor 1 refers to the output link table 16 and sends the node message to a plurality of link processors 2, and the link processors 2 The sent message is subjected to link processing and sent to the input node processor 1 with reference to the input node identifier 26. In the reverse message transfer from the input node to the output node, node processor 1
refers to the input link table 14 and sends node messages to a plurality of link processors 2, and the link processors 2 perform link processing on the sent messages,
It refers to the output node identifier 24 and sends it to the output node processor 1. As described above, bidirectional message transfer between the node processors 1 via the link processor 2 becomes possible.

[0014] In neural network learning algorithms such as error backpropagation algorithm and Heavian rule, correction of link weights requires information on the states of output nodes and input nodes. , refer to the input link table and link processor 2
By transferring node state information to each link processor 2, information necessary for weight correction can be transferred to each link processor 2. In addition, even in large-scale, irregularly structured neural networks, only the necessary links can be input (Link Table 14).
, output link table 16, and unnecessary calculations and storage capacity are not required. This makes it possible to efficiently perform simulations involving bidirectional communication between nodes even in large-scale, irregularly structured neural networks.

[0015]

As described above, according to the present invention, in forward message transfer from an output node to an input node, the node processor refers to the output link table and transfers the message of the node to the multiple link processors. The link processor performs link processing on the sent message, refers to the input node identifier, and sends it to the input node processor.For message transfer in the reverse direction from the input node to the output node, the link processor performs link processing on the message sent. refers to the input link table and sends the node message to multiple link processors, the link processor performs link processing on the sent message, refers to the output node identifier and sends it to the output node processor. This enables bidirectional message transfer via the link processor.

Further, even in a large-scale, irregularly structured neural network, it is sufficient to have input link tables and output link tables for only necessary links, and unnecessary calculations and storage capacity are not required.

As described above, even in a large-scale, irregularly structured neural network, it is possible to efficiently execute a simulation involving bidirectional communication between nodes.

[Brief explanation of drawings]

FIG. 1 is an overall configuration diagram of the system.

FIG. 2 is a detailed diagram of a node processor.

FIG. 3 is a detailed diagram of the link processor.

[Explanation of symbols]

1 Node processor 2 Link processor 3 Inter-processor communication path 11 Storage device 12 Simulation processing device 13 Communication device 14 Input link table 15 Node internal state 16 Output link table 21 Storage device 22 Simulation processing device 23 Communication device 24 Output node identifier 25 Link state 26 Input node identifier

Claims (1)

[Claims] Claim 1: A system for simulating a neural network in which each node exchanges messages with each other via weighted links and changes its internal state and output according to the content of the messages; a plurality of nodes that simulate the functions of the nodes; a processor, a plurality of link processors for simulating functions of weighted links, and a communication path for the node processors and the link processors to communicate with each other; the node processors and the link processors each have a different processor identifier. a storage device that stores the states of the nodes and links, a simulation processing device that simulates the functions of the nodes and links, and a communication device that performs communication processing with the other node processors and the link processors. composed;
The storage device of the node processor maintains an output link table that maintains an identifier of the link processor that simulates a plurality of output links going out of the node, and an identifier of the link processor that simulates the plurality of input links that enter the node. storing an input link table and the internal state of nodes; the storage of the link processor stores an identifier of the node processor that simulates an output node that sends a message via a link, and an input node that receives a message via a link; A neural network simulation system characterized by storing an identifier of the node processor to be simulated and a link weight; and having a bidirectional message transfer function between the node processors via the link processor.