JPH0883261A - Neuro arithmetic unit - Google Patents

Abstract

PURPOSE: To efficiently utilize a data area and speed up neuro operation by composing the knowledge base of a group connection relation table and a data arrangement table. CONSTITUTION: The knowledge base has the group connection relation table and data arrangement table. A table 2 shows the contents of the group connection relation table of a network and binary data showing the relation of connections from units u1 to u4 to an input layer group and respective units u1 to u8 and the relation of connections an output layer group consisting of units u5 to u8 us and the respective units u1 to u8 are entered. A table 3 shows the contents of the data arrangement table. This table 3 is formed by taking only combinations of groups and units which are given '1' out of the table 2 and entering data of addresses where the degree of connections between units included therein are stored.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neuro arithmetic unit for converting input information obtained from a certain object by using a plurality of units for performing a fixed operation.

[0002]

2. Description of the Related Art In recent years, an information processing system using a large number of units modeling information processing of nerve cells in a living body has been applied to control and pattern recognition. For example, in character recognition, a sample character and the actual character that it represents are given as learning data, the degree of connection of the network is learned, and the character actually input from the input device is determined using the network. .

A unit is a multi-input / single-output element that performs an operation on an input and outputs the result. That is, FIG.
, A plurality of data O ₁ , O ₂ , ... O _i , ... O
_Assuming that _n is input to the unit u _j , these data are multiplied by w ₁ , w ₂ , w _i , ... W _n , respectively, and the input value i is obtained by subtracting the threshold value θ _j from the total addition value. Treat as. Then, the output function f (i _j ) as shown in FIG.
The output value O _j is output based on

Each unit is connected to other units to form a network. This bond has a bond degree (weighting coefficient) expressing the bond strength between the units as a positive value, zero, or a negative value. Then, as described above, an operation in which each unit outputs based on the input of a plurality of data is called a neuro operation.

The network structure is roughly classified into two types.
One is a hierarchical network in which each unit forms a layer and input information is converted from the input layer to the output layer one after another. The other is an interconnected network in which each unit is interconnected. There are various networks in the hierarchical network, such as one having a feedback loop in which the output value of the output layer is returned to the input layer, and one having mutual coupling only in each layer.

FIG. 9 is a conceptual diagram showing an example of a three-layer hierarchical network having no intra-layer connection, and FIG. 10 is a conceptual diagram showing an example of a two-layer hierarchical network having intra-layer connection. .

FIG. 11 is a block diagram showing the configuration of a conventional device for realizing such a network. In this figure, a neuro operation unit 1 performs the above-mentioned neuro operation, and has a neuro processor corresponding to each unit. The program storage unit 2 stores the neuro calculation method and learning rule.

The unit state storage unit 3 stores the unit output value and unit state (a specific state represented by binary data) obtained by subjecting a certain input to certain processing. The bond data storage unit 4 stores a bond degree indicating the bond strength between units (a numerical value of 0 to 1 indicating the bond degree).

The knowledge base 5 has a connection relation table 6. This connection relation table 6 stores binary data indicating the presence or absence of mutual connection relations between a plurality of units, and the data arrangement table 7 relates to the degree of connection between the units described in this connection relation table 6. The address storing the data is described.

Next, the contents of the connection relation table 6 will be described. First, the case of the three-layer hierarchical network of FIG. 9 will be described. This input layer is composed of units u ₁ , u ₂ and u ₃ , the intermediate layer is composed of u ₄ , u ₅ and u ₆ , and the output layer is composed of u _7. , U ₈ and u ₉ respectively. Intra-layer bonding is not performed in each layer.

As is apparent from FIG. 9, each of the input layer units u ₁ , u ₂ and u ₃ has a coupling relation with all of the intermediate layer units u ₄ , u ₅ and u _6. , And each of the units u ₄ , u ₅ , u _{6 in} the intermediate layer is
It has a coupling relation with all of the units u ₇ , u ₈ and u ₉ of the output layer. Therefore, if it is indicated by "1" that the units have a coupling relationship, the coupling relationship between the units in FIG. 9 is expressed as in Table 11 in FIG. 12 (a).

By the way, since the neuro calculation repeatedly executes the same calculation, it can be processed in parallel and sequentially, and by doing so, the speed can be increased. The units u ₁ , u ₂ and u ₃ each have a bond to the units u ₄ , u ₅ and u ₆ , and have no bond to the other units. Therefore, FIG.
As shown in Table 12 in (b), only the units u ₄ , u ₅ , and u ₆ can be processed in parallel and sequentially. Similarly, with respect to the units u ₄ , u ₅ and u ₆ , only the units u ₇ , u ₈ and u ₉ can be processed in parallel and sequentially. Therefore, the data of the coupling degree stored in the coupling data storage unit 4 should be only the data of the units actually having the coupling relation, and the data should be retrieved by the data arrangement table 7 similar to the coupling relation. Should be possible. However, in reality, the connection relation table 6 describes the connection relation between all units.

Next, the case of the two-layer hierarchical network shown in FIG. 10 will be described. This input layer is a unit u.
_{1 to} u ₄ and the intermediate layers are units u _{5 to} u _8.
It consists of. Intra-layer bonding is performed in each layer, and the bonding relationship between the units is shown in FIG.
It is represented as in Table 13 of (a).

As is clear from Table 13, the unit u ₁
Has a connection relation with all units u _{2 to} u ₈ other than itself, and thus can perform parallel and sequential processing.
The other units are not connected to all units like the unit u ₁ and therefore cannot perform processing in parallel and sequentially.

Therefore, as shown in Table 14 of FIG. 13 (b), a unit that does not actually have a coupling relation but has a coupling relation as if it actually has a coupling relation is " 0 ", which enables parallel and sequential processing. In the connection relation table 6 in the knowledge base 5, binary data “1”,
"0" is described.

The neuro operation device has a neuron corresponding to each unit (a unit in the neuro operation device is called a neuron in order to distinguish it from a unit on the network structure), and each neuron is a neuro processor. It is realized in. Now let the neuron i be n
When the writing is i, the unit u _1, u ₂ in the network of FIG. 10, ..., u ₈ is neuron n _1, n
_2, ..., will be assigned is to n _8. However, the number of neurons (the number of neuroprocessors) that can be included in the neuro-operation device has a limit in the area and size of the device. Therefore, when the number of units to be processed in parallel is larger than the number of neurons in the device, time division processing is performed. For example, assuming that 64 neurons (consisting of 32 neuroprocessors) are mounted in the apparatus, time division processing is performed when the number of units to be processed in parallel becomes 65 or more.

[0017]

For some input, the network must be trained in order to obtain the desired output or the state of each unit. There are various methods for the learning. Now 1000
In an interconnected network composed of (= 1.0 × 10 ³ ) units, assuming that each unit has connections from all units, the total number of connections is 1.0 × 1.
It will be 0 ⁶ . In order to obtain a desired input / output relationship for all these connections, it is necessary to give input / output data as many as hundreds of sets as teacher data and apply a learning rule to learn the degree of connection. In this case, the number of units,
Depending on the number of connections, the time required for learning and network operation increases exponentially.

On the other hand, as shown in FIGS. 9 and 10, in a so-called loosely coupled network in which all units are not coupled to each other, the meaning of "0" means as the network scale increases. The number of elements that store addresses for data related to a unit to which invalid data is attached increases, resulting in a large amount of useless data area.

For example, if the total number of bonds is 1.
In the case of 0 × 10 ⁶ , if 1% of them are actually unjoined, the number of invalid data is 1.0 × 10 ^4.
Therefore, the useless data area becomes very large.

Also, when the common processing (for example, the time-division processing in the common processing) is performed, the common processing that handles only invalid data is increased, and the ratio of wasted processing time becomes very large.

That is, suppose that the neuron of the neuron is three, that is, n ₁ , n ₂ and n ₃ .
Since the number of units in the network of FIG. 10 is 8, it is necessary to perform time division processing. Now, as shown in Table 15 of FIG. 14, the neurons n ₁ have units u ₁ , u ₄ ,
u _{7 is connected} to the neuron n ₂ by the units u ₂ , u ₅ , and u _8.
, The units u ₃ and u ₆ are assigned to the neuron n ₃ . Then, the total number of time division processing is 2
The number of invalid time-division processing is 11 for 4 and the total 46
% Means wasteful processing. Therefore, it has been a major factor that impedes the speeding up of calculations.

The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a neuro arithmetic unit capable of efficiently utilizing a data area and accelerating neuro arithmetic. .

[0023]

As a means for solving the above-mentioned problems, the invention according to claim 1 has a knowledge having a table in which data concerning mutual connection relations of a plurality of units forming a neural network is described. In a neuro operation device including a base and a neuro operation unit that performs a neuro operation based on the knowledge base table, in the knowledge base, the plurality of units are divided into a predetermined number of groups, and each group A group connection relation table in which binary data indicating a connection relation between each unit and each unit is described, and a combination relation among combinations of each group and each unit described in the group connection relation table. Only for the combination that describes the binary data that means, the same number of units included in the combination. A data arrangement table described data address coupling degree is stored in and is characterized in that it consists of.

According to a second aspect of the present invention, in the first aspect of the present invention, the group of the group connection relation table is an aggregate of units that can perform common processing.

According to a third aspect of the present invention, in the second aspect of the present invention, the common processing is time division processing, and the neuro-operation unit performs time division processing based on the individual connection relation table. It has a control means.

[0026]

According to the first aspect of the invention, the plurality of units are divided into a predetermined number, and the binary data indicating the connection relation between each group and each unit is described in the group connection relation table.

Then, in the data arrangement table, only for a combination of a group and a unit having a connection relationship, the data of the address in which the degree of connection between a plurality of units included in this combination is stored is described. .

Of course, although the invalid data exists in this data arrangement table, the units belonging to the combination of the group and the unit having no connection relation are excluded, so that the number of invalid data becomes smaller than the conventional one. .

As in the second aspect of the invention, if the grouping is performed for each group of units capable of performing common processing, the common processing can be efficiently performed.

Further, when the time division processing is performed as the common processing as in the third aspect of the invention, the time division processing can be efficiently performed.

[0031]

Embodiments of the present invention will be described below with reference to FIGS. However, the same components as those in FIG. 11 are designated by the same reference numerals, and duplicate description will be omitted.

FIG. 1 is a block diagram according to an embodiment of the invention described in claim 1. The knowledge base 5A of this embodiment includes a group join relation table 6A and a data arrangement table 7
Have A.

Table 2 in FIG. 2B shows the contents of the group connection relation table 6A in the network of FIG. 10, which is an input layer group consisting of units u _{1 to} u ₄ and each unit u _{1 to} u _8. And the output layer group consisting of the units u _{5 to} u ₈ and each unit u ₁ to
Binary data showing the binding relationship with u ₈ is described.

Table 3 in FIG. 2C shows the contents of the data arrangement table 7A. This is one in which only the combinations with a unit of “1” are taken out from the combinations of groups and units in Table 2, and the data of the address at which the coupling degree between the units included in them is stored is described. . In the table, a1, b1, c1, ..., G7 indicate address data, and "0" indicates an address of data having a coupling degree of zero.

Table 1 of FIG. 2A shows the contents of the conventional data arrangement table. Since the total number of elements in Table 1 is 8 × 8 = 64 and the number of addresses for valid data is 19, the effective utilization rate is 19/6.
4 = 0.30 or 30%.

The number of elements in Table 2 is 16 and the number of elements in Table 3 is 4 × 11 = 44. Therefore, 64- (44 + 16) = 4, and as shown in Tables 2 and 3, the number of elements can be reduced by 4 by using a table having hierarchical connection relationships.

In Table 2, the units u are shown in the vertical direction.
_{Although 1 to} u ₄ and the units u _{5 to} u ₈ are divided into the input layer group and the output layer group, respectively, there is another way to divide the units that perform parallel processing into the same group.

That is, in Table 4 of FIG. 3A, the units u _{1 to} u ₄ and the units u _{5 to} u _{8 in} the horizontal direction are divided into two groups, and the coupling relation between these groups and each unit is shown. Binary data is shown. And FIG.
In Table 4 of (b), only the combinations with “1” among the combinations of groups and units in Table 4 are taken out, and the data of the address storing the coupling degree between the units included in them is extracted. Has been described.

The number of elements in Table 4 is 16 and Table 5
The number of elements in is 4 × 9 = 36. Therefore,
Since 64- (36 + 16) = 12, the memory area can be reduced by 12 as compared with the case of Table 1.

It is also possible to change the combination between units before grouping a plurality of units.
For example, in Table 13 of FIG. 13A, the units u ₂ and u ₇ in the vertical direction and the units u ₃ and u ₈ in the vertical direction are replaced. Table 6 in FIG. 4A shows this state. Then, the grouping is performed based on the table 6 into the group connection relation table of the table 7, and the content of the data arrangement table corresponding to the table 7 is the table 8.
The number of elements in Table 7 is 16 and the number of elements in Table 8 is 4 × 10 = 40. Therefore, 64- (40 + 16) = 8, and the memory area can be reduced by 8 compared with the case of Table 1.

FIG. 5 is a block diagram according to an embodiment of the invention described in claim 3. The neuro operation unit 1A of this embodiment has a time division control means 8.

Table 9 in FIG. 6 (a) shows the contents of the group connection relation table 6A in the network of FIG. As shown in Table 15, the neuron n ₁
The units u ₁ , u ₄ and u ₇ are assigned to the neuron, the units u ₂ , u ₅ and u ₈ are assigned to the neuron n ₂ , and the units u ₃ and u ₆ are assigned to the neuron n ₃ .
Grouping is performed into time division processing groups u _{1 to} u ₃ in the first cycle, time division processing groups u _{4 to} u ₆ in the second cycle, and time division processing groups u ₇ and u _{8 in} the third cycle. . Table 10 in FIG. 6B is a data arrangement table corresponding to Table 9.

As is apparent from Table 9, the total number of time-division processing is 24, but since only the combination marked with "1" needs to be processed, the actual time-division processing is performed. The number of times is 13. Therefore, the number of times of time-division processing can be significantly reduced, and the speed of neuro calculation can be increased.

[0044]

As described above, according to the present invention, it is possible to efficiently use the data area and to speed up the neuro operation.

[Brief description of drawings]

FIG. 1 is a block diagram according to an embodiment of the invention described in claim 1.

FIG. 2 is a chart showing the contents of a table included in a knowledge base shown in FIG.

FIG. 3 is a chart showing the contents of a table included in a knowledge base shown in FIG.

FIG. 4 is a chart showing the contents of a table included in the knowledge base shown in FIG.

FIG. 5 is a block diagram according to an embodiment of the invention described in claim 2.

FIG. 6 is a diagram showing the contents of a table included in the knowledge base shown in FIG.

FIG. 7 is an explanatory diagram of a unit used in a neural network.

8 is a characteristic diagram showing a function of output according to the unit of FIG.

FIG. 9 is a conceptual diagram of a three-layer hierarchical network having no intra-layer connection.

FIG. 10 is a conceptual diagram of a two-layer hierarchical network having intra-layer coupling.

FIG. 11 is a block diagram of a conventional device.

FIG. 12 is a chart for explaining the problems of the conventional device.

FIG. 13 is a chart for explaining the problems of the conventional device.

FIG. 14 is a chart for explaining the problems of the conventional device.

[Explanation of symbols]

 1 Neuro Operation Unit 2 Program Storage Unit 3 Unit State Storage Unit 4 Joined Data Storage Unit 5A Knowledge Base 6A Group Joined Relationship Table 7A Data Placement Table 8 Time Division Control Means

Claims (3)

[Claims] 1. A knowledge base having a table in which data relating to mutual connection relations of a plurality of units forming a neural network is described, and a neuro operation unit for performing a neuro operation based on the knowledge base table. In the neuro operation device, the knowledge base is configured such that the plurality of units are divided into a predetermined number of groups, and indicates a coupling relationship between each group and each unit.
Only the group connection relation table in which the value data is described, and the combination in which the binary data that indicates that there is a connection relation among the combinations of each group and each unit described in the group connection relation table are described. And a data arrangement table in which the data of the address in which the coupling degree between the plurality of units included in the combination is stored is described. 2. The neuro arithmetic device according to claim 1, wherein the group of the group connection relation table is an aggregate of units capable of performing common processing. 3. The neuro-calculation device according to claim 2, wherein the common processing is time-division processing, and the neuro-calculation unit has time-division control means for performing time-division processing based on the individual connection relation table. A neuro arithmetic unit characterized by: