JPH03105452A - Weight holding system for network constitution data processor - Google Patents

Abstract

PURPOSE:To attain high speed access as against weight or a weight update quantity by using only a storage capacity required for real network organization and allocating a large weight storage area. CONSTITUTION:Weight or the weight update quantity, which are respectively allocated to respective internal connection in a hierarchy network are stored in the large area of a main storage or RAM. At the time of accessing to weight as against a specified input for a specified unit in a specified layer, the area of a layer corresponding to the internal part of a layer designation data storage means 22 is selected and an area corresponding to a 0-th unit in the layer is instructed in a unit designation data storage means 21 from the internal part of the area. A storage area as against the corresponding unit in the means 21 is selected according to the number pf the unit to be accessed and the storage area of weight as against the input of the unit is obtained in a weight storage 20 from the content of the area, whereby weight as against the input is accessed.

Description

[Detailed Description of the Invention] [Summary] Regarding a weight retention method of a network configuration data processing device for efficiently retaining weights for inputs to each neuron in a neural network, only the storage capacity required for the actual network configuration is provided. The aim is to prevent slowdowns in learning, etc. by allocating a large weight storage area even in network configurations where the number of units in each layer varies. The basic unit is a unit that sums up the products of the inputs and the weights to be multiplied by the respective inputs and obtains an output for the sum, and the intermediate layer consisting of one or more units is one stage, one or One or more intermediate layers consisting of a plurality of units, l
between the input layer and the foremost intermediate layer, between the adjacent intermediate layers,
In a data processing device that constructs a hierarchical network by configuring internal connections between units between the final intermediate layer and the output layer and assigning weights to the internal connections, each internal network of the hierarchical network Weight storage means for storing weights or weight update amounts respectively assigned to the connections in a continuous area for each of the intermediate layer and the output layer and for each unit of all the units inside each layer;
Unit designation data storage means for storing, in the weight storage means, a start address of a storage area for the weight or weight update amount corresponding to each unit of all the units in each layer in a continuous area for every unit in each layer; and a layer designation data storage means for storing each start address of a continuous area for every unit of each layer in the unit designation data storage means, the contents of the layer designation data storage means and the unit designation data storage means. The weight or weight update amount assigned to each internal connection can be accessed by using the start address in the weight storage means of the weight or weight update amount for each unit, which is determined from the above.

[Industrial application field]

The present invention relates to a learning processing method for a network configuration data processing device, and more particularly to a weight retention method for a network configuration data processing device for efficiently retaining weights for inputs to each neuron in a neural network, for example. With conventional sequential processing computers, that is, von Neumann type computers, it is difficult to adjust the computer's data processing function according to changes in the usage method or environment. Therefore, parallel failure processing using hierarchical networks is used as an adaptive data processing method. methods have been proposed. In particular, a processing method called the pack propagation method is attracting attention because of its high practicality. In order to realize a hierarchical network, it is necessary to manage a large amount of weight information. Also, since weights are used frequently, access speed to weights must be as fast as possible. Furthermore, the network configuration not only changes depending on the application, but also may change due to an increase in data during operation, so the weight storage method needs to be flexible enough to accommodate changes in the network configuration. .

[Conventional technology]

A hierarchical network, for example a bersebutron type network, is a hierarchical one consisting of several layers, each layer consisting of an appropriate number of units, there are no connections within the layer, and the connection between the eyebrows is from input N (first layer) to output. 1 towards N (M terminal layer)
It is only a direction combination.

The units in each layer except the input layer receive weighted inputs from the units in the previous layer, calculate the sum, and output an appropriate function value for the sum. As a function, for example, a darkness value function is used, which takes l when the sum is larger than a certain value and takes 0 when it is smaller.

In such a hierarchical network, in order to maintain the weights or the amount of weight updates during learning, Tsuchiki prepares a storage area for the maximum number of layers and the maximum number of units in each layer. Software (C language)
If expressed as , it becomes as follows.

doule W (MAXLAYER) (MAX
UNIT) (MAXUNIT); FIG. 4 shows a conventional example of a weight holding method using hardware. This figure corresponds to a case where the maximum number of layers is 10 and each layer is composed of a maximum of 1000 units. That is, the network is constructed from the 0th layer to the 9th layer, and each layer is controlled by the 0th to 999th units. In FIG. 4, areas 1 to 5 are 17i1N,
That is, it stores the weight for the 0th, that is, the first unit of the first intermediate layer, and represents the weight for the 0th input, 1st input, 2nd input, ...999th input from the input layer. . Next, areas 6, 7, and 8 represent the weights for 9 for the 0th, .
It represents the weight for the 0th, . . . 999th input to the th unit. Similarly, the second layer, the 371st layer, etc.
The weights for the inputs to the units within are stored, and the weights for the inputs to the 9th Wi and 999th units are stored in areas 17, 18, and 19. In order to realize up to a maximum of 1000 units in each layer in this manner, a storage capacity of 10×1000×1000=10M words is required.

[Problem to be solved by the invention]

As mentioned above, in the conventional weight retention method, the storage capacity for the maximum expected number of layers and the maximum number of units in a layer is prepared in advance, so the actual network configuration used is, for example, three layers. If the number of units varies, such as 1000 units in the input layer, 10 units in the middle layer, and 20 units in the output layer, the actual storage area required is t000x10+10x20= Even though the number of words is 10,200, if a storage area for IOM words is prepared as described above, there is a problem that the usage efficiency of the storage area is extremely low.

Moreover, the area actually used will be allocated to dispersed areas in the IOM word space, and in the case of a computer that uses virtual memory, disk access will occur frequently, reducing learning speed and execution speed. There was also the problem that it caused a decrease in The present invention uses only the storage capacity necessary for the actual network configuration, and allocates a large amount of weight storage area even in network configurations where the number of units in each layer varies, thereby preventing slowdowns in learning etc. The purpose is to

[Means to solve the problem]

Figure 1 is a block diagram of the principle of the present invention. The figure shows one or more units that obtain an output from the sum of the products of one or more inputs from the previous layer and the weights to be multiplied by the inputs, such as dark value elements, as a basic unit. an input layer composed of units, one or more intermediate layers composed of one or more units, and an output layer composed of one or more units, the input layer By configuring internal connections between units between and the first hidden layer, between adjacent hidden layers, and between the last hidden layer and the output layer, and assigning weights to each of these internal connections, respectively. FIG. 2 is a principle block diagram of a weight retention method in a data processing device that maintains a hierarchical network.

In FIG. 1, the weight storage 20 is, for example, a random access memory (RAM), and stores the weight assigned to each internal connection of the hierarchical network or the previous weight update amount during learning in the intermediate layer and the output layer. It is stored collectively for each layer and in a continuous area for each unit of all units within each layer. The unit designation data storage means 21 is, for example, a register, and inside the weight storage means 20, the start address of the storage area of the weight corresponding to each unit of all the units in each layer is stored in a continuous area for every unit in each layer. do. The layer designation data storage means 22 is, for example, a register, and stores the start address of each continuous area for every unit of each layer in the unit designation data storage 21.

For example, if the network is made up of 3N layers: an input layer, a middle layer, and an output layer, the layer specification data storage means 22 consists of two areas corresponding to the middle layer and the output layer. 0th to the 0th unit in the layer
This indicates the address of the area within the unit designation data storage means 21 that stores the address of the weight storage means 20 that stores the weight for the input. [Operation] As described above, in the present invention, the weight or weight update amount assigned to each internal connection of the hierarchical network is stored in a large area in the main memory or RAM, for example. For example, when accessing the weight for a specific input to a specific unit in a specific layer, the area of the corresponding layer inside the layer designation data storage means 22 is selected, and the unit designation data storage means 2
1, the area corresponding to the Oth unit of that layer is designated. Then, depending on the unit number of the unit to be accessed, a storage area for the corresponding unit in the unit specification data storage means 21 is selected, and the storage area for the corresponding unit is selected in the weight storage means 20 based on the contents of the area. A storage area for the weight for the input "O number l" is found, and the weight for that input is accessed depending on which input the weight to be accessed corresponds to.

As described above, in the present invention, the storage area necessary for the actual network configuration is collectively secured on the main memory or RAM.

(Embodiment) FIG. 2 is a basic conceptual diagram of an embodiment of the weight retention method of the present invention. In the figure, the layer designation storage device 23 corresponds to the layer designation data storage stage 22 in FIG. 1, the unit designation storage device 24 corresponds to the unit designation data storage stage 21, and the main memory 25 corresponds to the weight storage stage 20. do. The layer designation storage device 23 and the unit designation storage device 24 may be located on the main memory, or may be composed of registers.

FIG. 2 shows a weight retention method for a 3N network consisting of 1000 units in the input layer, 10 units in the hidden layer, and 20 units in the output layer. Layer specified storage device 2
The first intermediate layer in 3 corresponds to the intermediate layer, and the second intermediate layer corresponds to the output layer. The area from the third intermediate layer to the output layer is not used here.

Compared to the conventional example shown in FIG. 4, the difference in the present invention is that the area on the main memory is used as much as is necessary for actual network configuration.

In FIG. 2, weights for internal connections between units in the network are successively assigned to each unit on the main memory. In other words, there are 1000 input layers and IO layers in the middle layer.
In the case of a network consisting of 20 units in the output layer, first the weight i of the 0th unit in the hidden layer is
ooo pieces are stored in main memory. Then, the start address is stored in the first word of the unit designation storage device 24 as data for the Oth unit 1. Then ψ
The 1000 weights of the first unit of the intermediate layer are stored in the main memory, and the start address is stored in the unit designated storage device 2.
It is stored as the -2nd word of 4. Thereafter, the weights for each unit are sequentially stored in the main memory 25 in the same way, and the start addresses of these continuous areas are stored in the unit designation storage device 24.

By using the storage method described above, main memory 2
5 without wasting space, and since the start address of the weight for each unit is stored in the unit designation storage device 24, high-speed access to the weight is possible. Become. Furthermore, by storing the address of the area storing data for the 0th unit 1 of each layer in the unit designation storage device 24 in the layer designation storage device 23, if the layer, unit, and weight numbers are given, the layer The corresponding weight on the main memory 25 can be accessed via the designated storage device 23 and the unit designated storage device 24.

FIG. 3 is a block diagram of the structure of an embodiment of the weight retention method of the present invention. The figure shows a main control circuit (MPU) 26, three selectors 27, 28, 29, and a layer designation storage device 23 in FIG.
A layer designation register 30 corresponding to the unit designation register 30, a unit designation register 31 corresponding to the unit designation storage device 24, and a main memory 25
weight RAM 32, two adders 33, 34,
and the weight data bus 35 between the main control circuit (MPU) 26 and the weight RAM 32.

The main control circuit (MPU) 26 has signal lines for layer numbers, unit numbers, weight numbers, weight read/write (R/W) signals, and weight data. The selector 27 selects one of the layer designation registers 30 based on the layer number designated by the main control circuit 26. Layer specification register 30
For example, in order to realize a network of up to 10 layers, it is configured by 10 registers or 10 words of memory, and its contents are the number of the unit designation register corresponding to the first unit of each layer.

The selector 28 selects one unit designation register 310 based on the unit number given from the main control circuit 26, and the sum of the unit number and the 0th unit designation register number in the layer to be accessed which is output from the layer designation register 30 is The sum is taken by adder 33 and used to select a register specifying the unit to be accessed.

The selector 29 is a selector that selects one area of the weight RAM 32 based on the weight number output from the main control circuit 26. That is, the adder 34 adds the weight number and the start address of the weight for the unit to be accessed in the weight RAM 32, which is the output of the unit designation register 3l, and uses this to access one area of the weight RAM. .

The unit designation register 31 is, for example, the maximum number of units 2.
In order to realize 000 networks, 2000 registers or 2000 words of memory are required. Also, the weight RAM 32 is a dedicated RAM.
may be prepared, or a part of the main memory may be used.

Note that the layer designation register 30 and the unit designation register 3l
, all of the weight RAM 32 can be stored in the main memory,
In that case, the selector function and adder function are performed by MPU2.
6 can also be performed by software. In other words, read the corresponding part in the layer designation register in main memory based on the layer number, calculate the sum of its contents and the unit number, read out the corresponding part in the unit designation register in main memory based on that, and then The final weight address can be obtained from the sum of the content and the weight number and the weight can be accessed. In this way, if the layer designation register 30, unit designation register 31, and weight RAM 32 are all implemented in the main memory, there will be no restrictions on the number of layers and units, and the main memory can be used without wasting the main memory capacity. You can build as large a network as you can.

〔Effect of the invention〕

As described in detail above, according to the present invention, weights for internal connections of a network can be collectively arranged in a continuous area without wasting area on the main memory. Therefore, access to external storage for virtual storage management can be reduced, and high-speed access to weights or weight update amounts is possible.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the present invention. FIG. 2 is a diagram showing the concept of an embodiment of the weight retention method of the present invention. FIG. 3 is a block diagram showing the structure of an embodiment of the weight retention method. 1 is a diagram showing a conventional example of a weight holding method using hardware. 2G...Main control@path (MPU), 27, 28, 29...Selector, 30...Layer specification register, 3G...Unit specification register, 32...Weight RAM, 33, 34 ・Adder. Patent issuer τr Shitsu stock Δ company

Claims (1)

[Claims] The basic unit is a unit that sums up the products of one or more inputs from the previous layer and the weights by which the inputs are respectively multiplied, and obtains an output for the sum. Or one input layer consisting of multiple units, one or more intermediate layers consisting of one or more units, one
between the input layer and the foremost intermediate layer, between the adjacent intermediate layers,
In a data processing device that configures a hierarchical network by configuring internal connections between units between a final stage intermediate layer and an output layer and assigning weights to each internal connection, each internal connection of the hierarchical network is provided with: Weight storage means (20) for storing each allocated weight or weight update amount in a continuous area for each layer of the intermediate layer and the output layer and for each unit of all units inside each layer; and the weight storage means (20) Unit designation data storage means for storing the start address of the storage area of the weight or weight update amount corresponding to each unit of all the units in each layer in a continuous area for each unit in each layer ( 21
), and a layer designation data storage means (22) for storing each start address of a continuous area for every unit of each layer in the unit designation data storage means (21), the layer designation data storage means (22) ) and the weight storage means (20) for the weight or weight update amount for each unit, which is determined from the contents of the unit specification data storage means (21).
1. A weight holding method for a network configuration data processing device, characterized in that the weight or weight update amount assigned to each of the internal connections can be accessed using a first address in the internal connection.