JPH04319760A - Hierarchical connection type neuro-computer - Google Patents

Abstract

PURPOSE:To provide a neuro-computer having a function for adjusting the number of units in an intermediate layer because work for properly setting up the number of units in the intermediate layer is difficult in a conventional method at the time of constructing hierarchical connection type neural network. CONSTITUTION:Respective outputs from respective units in the intermediate layer in learning are accumulated in a past data accumulating part 2. Whether output existence or no output is continued for a fixed period or not is monitored/judged in each unit by a judging part 3, and when the output existence is continued, the unit concerned is copied by a unit increasing part 6 to increase the number of units, and when no output is continued, the unit is erased by a unit erasing part 7.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neurocomputer equipped with a hierarchically connected neural network.

0002

2. Description of the Related Art Generally, a neurocomputer is a device that constructs a neural network by combining models of neurons (nerve cells) and processes information using this neural network.

[0003] As shown in FIG. 2, a multi-input, single-output type model is usually used for the neuron unit. Input signals x1 to xn are signals input from other units. This unit uses each of the input signals x1 to xn to perform an operation expressed by the following equation, and generates an output signal y.

0004

[Math 1]

y=f(I) (2) Here, w1 to wn are weights given to each input signal x1 to xn, and f is a threshold function (increasing function). These parameters change with learning.

[0006] A neural network is constructed by connecting such units in a predetermined format. The hierarchically coupled neural network consists of an input layer 1, a middle layer 2, and an output layer 3, as shown in FIG. Each hierarchy is composed of a plurality of units 4, which are coupled so that the output of each unit in the previous hierarchy becomes the input to each unit in the subsequent hierarchy.

[0007]

[Problems to be Solved by the Invention] Normally, learning is started after determining the number of units in the input layer 1, intermediate layer 2, and output layer 3. The number of units in the input layer 1 and output layer 3 is naturally determined by the problem to which it is applied, but the number of units in the intermediate layer is not uniquely determined by the problem. For this reason, conventionally, the number of units has been experimentally set by the user. However, it is not easy to determine the appropriate number of units; if the number of units is too small, the desired performance cannot be obtained, and if the number is too large, hardware resources may be lost.

SUMMARY OF THE INVENTION In view of the above circumstances, it is an object of the present invention to provide a hierarchically connected neurocomputer having a function of setting the number of units in the intermediate layer.

[0009]

[Means for Solving the Problems] In order to achieve the above object, the present invention provides a neural network in which a hierarchically connected neural network is constructed by sequentially connecting an input layer, a middle layer, and an output layer each consisting of a plurality of neuron model units. A computer equipped with the following means. In addition,
The above-mentioned intermediate layer is not limited to one layer, but may be an intermediate layer consisting of a plurality of layers.

(1) An intermediate layer output accumulation section that accumulates the output of each unit of the intermediate layer during learning over a predetermined number of learning times.

(2) Referring to the output of each unit of the intermediate layer stored in the intermediate layer output storage section of (1), and determining whether or not output or no output has occurred consecutively for a predetermined number of times for each unit. Department.

(3) When the determination unit (2) determines that the output is continuous, a unit increase unit generates a new unit by duplicating the unit and the combination of the unit and another unit.

(4) A unit erasing unit that erases the unit and the connections between the unit and other units when the determination unit in (2) determines that the output is non-continuous.

[0014]

According to the present invention, the output of each unit of the intermediate layer during learning is accumulated in the intermediate layer output storage section, and the determination section monitors the state of the output of each unit. If there is a unit that has an output for a predetermined number of consecutive times, it is determined that that unit is overloaded, and the unit increaser copies that unit and its connections with other units and adds a new unit. and reduce the load. Further, if there is a unit that continues to have no output a predetermined number of times, that unit is determined to be useless, and the unit erasing section erases that unit itself and the connections between that unit and other units. By repeating this, adjust the number of units to an appropriate number.

[0015]

Embodiment A hierarchically coupled neurocomputer simulator according to an embodiment of the present invention will be described below. This simulator virtualizes and constructs a hierarchically connected neural network using data such as vectors representing the signal patterns of each layer and connection matrices representing the connections between each layer. It has an adjustment function. This function can be understood as a combination of means (blocks) shown in FIG.

In FIG. 1, a network data storage unit 1 stores data for constructing a network, and includes a vector storage unit 1-1 for storing the above-mentioned vectors and a connection matrix storage unit for storing connection matrices. 1-2. Each time the network is trained, the past data storage unit 2 stores the vectors of the intermediate layer, and stores and stores the vectors over a predetermined period of time. The determination unit 3
The output status of each unit is determined by referring to the past data stored in the past data storage section 2. Then, when the presence or absence of output continues for a certain period of time, overload determination 4 or death determination 5 is performed.

If there is an overload determination 4, the unit increase unit 6 is selected, and the unit increase unit 6 adds units by modifying the vector of the intermediate layer and the connection matrix for the layers before and after the intermediate layer. That is, for the intermediate layer vector, the element of the vector is increased by one, and the element corresponding to the determined unit is copied to the new element. Similarly, for the connection matrix, the elements of the matrix corresponding to the determined unit are copied to increase the number of elements. As a result,
As shown in FIG. 2, a unit 4b' that is the same as the unit 4b in which the overload determination occurred is added to the network, and the load on the unit 4b is reduced.

When death is determined, the unit erasing section 7 is selected, and this unit erasing section 7 deletes the element corresponding to the unit for which the death has been determined from the vector of the intermediate layer and the connection matrix before and after it. As a result, as shown in FIG. 3, the unit 4b that has been determined to be dead is deleted from the network. By repeating the above operations, the number of units in the intermediate layer can be reset to an appropriate number.

[0019]

[Effects of the Invention] As explained above, according to the present invention,
During learning, it is possible to adjust the number of units in the middle layer to an appropriate number by duplicating units with a large load and adding new units, as well as erasing useless units that have not output for a long time. Become. Therefore, the strictness in setting the number of units in the middle tier can be relaxed, and the conventional cumbersome work procedure of experimentally setting the number of units in the middle tier can be omitted, ensuring the performance required for the network while reducing human resources. - It is possible to improve the utilization efficiency of hardware resources.

[Brief explanation of drawings]

FIG. 1 is a functional block diagram showing main parts of a hierarchically coupled neurocomputer simulator according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an operation of increasing units.

FIG. 3 is an explanatory diagram showing a unit erase operation.

FIG. 4 is an explanatory diagram showing a neuron model.

FIG. 5 is an explanatory diagram showing a hierarchically connected neural network.

[Explanation of symbols]

2... Accumulation section, 3... Judgment section, 6... Unit increase section, 7... Unit deletion section.

Claims (1)

[Claims] Claim 1: In a neurocomputer in which an input layer, a middle layer, and an output layer each consisting of a plurality of neuron model units are sequentially connected to form a hierarchically connected neural network, the output of the middle layer during learning is calculated over a predetermined number of times of learning. Referring to the intermediate layer output accumulation section to be accumulated and the output of the intermediate layer accumulated in this intermediate layer output accumulation section,
A determination unit that determines whether output or no output has continued for a predetermined number of times for each unit in the intermediate layer, and if the determination unit determines that output is continuous, it determines whether or not the unit has an output or not. A unit incrementing unit that duplicates a connection to generate a new unit, and a unit erasing unit that erases the unit and the connection between the unit and another unit when the determination unit determines that the output is non-continuous. A hierarchically connected neurocomputer characterized by: