JPH05346914A - Neuro processor - Google Patents

Abstract

PURPOSE:To provide a neuro processor which performs the inference of a neural network and calculation for learning at a high speed without increasing its hardware. CONSTITUTION:Processor elements 2-10 which can change mutual connections among a multiplier 24, an adder 25, a memory 26, and registers 34-40 by the switching of selectors 27-33 are arranged in a two-dimensional matrix shape, and memories 11-15 and adders 17-22 are added to the respective columns and rows to constitute a matrix arithmetic unit 1 which is flexible in hardware. This neuro processor consists of the matrix arithmetic unit and an auxiliary arithmetic device 23; and the matrix arithmetic unit 1 selectively perform matrix calculation and the auxiliary arithmetic unit 23 perform calculation other than the matrix calculation. Then the matrix calculation and other calculation are carried out in parallel to process the calculation including the learning of the neural network at a high speed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neuroprocessor for performing high-speed inference and learning calculations for simulating the actions of synapses and neurons that make up a neural network.

[0002]

BACKGROUND OF THE INVENTION Neural networks model and imitate the function of nerve cells in the human brain,
The aim is to realize a new computer that is good at recognition, association, optimization problems, speech synthesis, etc., which conventional so-called Neumann-type computers were not good at.

Various structures such as a hierarchical structure and a mutual connection structure have been studied as a model of a neural network depending on the arrangement of neurons. In a hierarchical network, an algorithm called backpropagation makes it easy. Since it can be learned, a wide range of applications are considered for control, character recognition, image recognition, image processing, and the like.

The structure and operation of this hierarchical network will be described below with reference to FIGS. 8 and 9. In FIG. 8, the constituent elements of the hierarchical network are neurons 101 arranged in each layer, synapses 102 representing the connection relationship between neurons, and the input layer, the intermediate layer, and the output layer constituting the hierarchical structure are 103 and 10 respectively.
4, 105. Although an example of a three-layer network is shown in FIG. 8, a network having a hierarchical structure of four or more layers can be configured by using a plurality of intermediate layers. Hereinafter, for simplification, the description will be limited to a three-layer network, but the same can be applied to a network configuration of four layers or more, or even if the number of neurons in each layer is increased or decreased.

In the three-layered neural network, an input signal is first given to each neuron of the input layer 103, and the signal is subjected to arithmetic operations of synapses and neurons constituting each layer, the input layer, the intermediate layer, and the output. The signals of the neurons in the output layer 105 are propagated in the order of the layers and become the output of the network. Such normal propagation in which the input layer, the intermediate layer, and the output layer propagate in this order is called forward propagation. The function of the neuron in this forward propagation will be described with reference to FIG. In FIG. 9, a neuron is indicated by 101, 107, 108, 109, etc., a synapse is indicated by 102, and a characteristic function f of the neuron is indicated by 106. Each neuron receives the output of the neuron of the previous layer via a synapse, and at that time, each synapse is characterized by a value called a connection weight specific to that synapse. The output value of is multiplied by the value of the connection weight of the synapse, and the result is given to the next neuron. For example, the synapse between the i-th neuron 108 and the i + 1-th neuron 101 in FIG. 9 has a connection weight W _lij , and among the neurons arranged in the l-th layer, the i-th neuron 108 The result of multiplying the output O _li by the value of the weight of the connection, that is, W _lji × O _li
Is given to the j-th neuron 101 among the neurons arranged in the (l + 1) th layer, and this neuron 101 adds all the similar inputs given by all the synapses connected thereto, and the addition result of the neuron The function value resulting from the action of the characteristic function 106 is output as the output O _{l + 1, j} of the neuron 101. This can be expressed by the following equation.

[0006]

[Equation 1]

Here, I represents the calculation result of the product-sum operation of ΣW × O. The calculation in the forward propagation of the three-layered network by the action of the above neurons and synapses is performed by the following procedure. First, the outputs in all neurons of the intermediate layer are calculated according to (Equation 2).

[0008]

[Equation 2]

Next, the outputs of the neurons in all output layers using the result are calculated by (Equation 3).

[0010]

[Equation 3]

Next, the learning method by the back propagation algorithm and the action of neurons and synapses in that case will be described. In learning in the backpropagation algorithm, a set of an input and an ideal output corresponding to the input is prepared in advance, and the difference between the actual output and the ideal output (called a teacher signal) for such an input is reduced. As described above, the operation is performed by correcting the synaptic connection weight. Hereinafter, the derivation of the correction value of the synapse connection weight, that is, the values of ΔW _1ji and ΔW _{2ji in} the case of three layers will be described by a specific calculation procedure by taking the three-layer network shown in FIG. 8 as an example. 1. The actual output obtained by the forward propagation is calculated based on (Equation 1) and (Equation 2). 2. A value δ _3j (hereinafter simply referred to as delta) corresponding to the error between the calculation result of each neuron in the output layer and the teacher signal is calculated by ( _Equation 4).

[0012]

[Equation 4]

Δ _lj is the delta for the j-th neuron among the neurons arranged in the l-th layer, t _j is the teacher signal corresponding to the given input of the j-th neuron among the neurons arranged in the output layer, g corresponds to the differential coefficient of the characteristic function f of the neuron expressed by (Equation 5). Since a monotone non-decreasing function is used as the characteristic function f of the neuron, g can be expressed as a function of the function value of the characteristic function.

[0014]

[Equation 5]

3. When the correction amount of the synaptic connection weight between the intermediate layer and the output layer is ΔW _lji , the correction value of this weight is calculated by ( _Equation 6).

[0016]

[Equation 6]

Η in (Equation 6) is called a correction coefficient, and R
Represents a matrix of δ × O. 4. The delta for the neurons in the hidden layer is calculated according to (Equation 7).

[0018]

[Equation 7]

S in (Equation 7) represents the product-sum result represented by Σδ × W. 5. The correction amount of the synaptic connection weight between the intermediate layer and the input layer is calculated using the output of the intermediate layer, and the correction value is calculated according to (Equation 8).

[0020]

[Equation 8]

In the actual learning, since it is necessary to repeat the calculation until the correction is no longer necessary, the above-described calculation procedure is executed for a plurality of combinations of the input and the teacher signal. That is, if the above procedure from 1 to 5 is called one learning, all learning times are given by (the number of pairs of input and teacher signals used for learning) × (number of repetitions), which is an enormous amount. It becomes a calculation amount.

As is clear from the above description, the calculation amount of the neural network has become enormous because of the repetition of the matrix operation accompanying learning. Large hardware such as a large-scale computer was required to execute this calculation. Therefore, in the application of the learning effect when targeting home electric appliances, learning by back propagation with a large-scale computer etc. and calculation of forward propagation performed by transferring the calculation result data to a microprocessor or digital signal processor (DSP) Is used.

[0023]

In the application of the learning effect of the above-mentioned conventional neural network, the use of large hardware is indispensable because of the huge amount of calculation. It has a large number of arithmetic units for parallel processing in order to ensure high speed, but there is a problem that parallel processing cannot be performed in some calculations due to lack of flexibility of hardware. This is because, for example, data cannot be exchanged between the memory and the arithmetic unit or between the arithmetic units depending on the type of calculation, and thus high-speed processing is difficult to perform. Conventionally, in order to deal with this kind of calculation that cannot be performed in parallel, there are two types, such as a method of increasing the number of arithmetic units for sequential processing, or a method of dividing the whole calculation into two groups and providing hardware suitable for the calculation. Method has been adopted. However, in any case, high-speed processing has been realized due to the increase in size of hardware.

From the viewpoint of application, the use of a general-purpose microprocessor or DSP is often an essential condition,
In a microprocessor, a huge amount of calculation time is required for complete sequential processing, so there is a limit to practical applications.
In the DSP, the sum-product operation is pipelined by the parallel operation of the adder and the multiplier, so it is much faster than the microprocessor, but the transfer of data from the memory to the arithmetic unit becomes a bottleneck, and the pipeline cannot be utilized. Due to this, the calculation speed cannot be increased in the calculation such as learning of the neural network. This is the reason why learning and positive operation must be separated even in the application to relatively simple home appliances. According to this conventional method, the operation of the neural network is fixed, and the operation desired by the user cannot be realized.

The present invention solves the above-mentioned problems, and provides a neuroprocessor for efficiently performing high-speed parallel processing of matrix calculations that occupy most of the calculation amount without increasing the hardware by giving flexibility to the hardware. It is intended to be provided.

[0026]

To achieve the above object, in the neuroprocessor of the present invention, a selector,
The function of the processor element composed of a multiplier, an adder, a memory, and a register is multiplied by this selector,
Flexibility is provided by switching the mutual coupling of adders, memories, registers, etc.
It has a matrix operation unit that is dimensionally arranged and has a memory and an adder added to each column and each row, and an auxiliary operation unit that can operate in parallel with this matrix operation unit and that performs calculations other than matrix calculation. It is a high-speed calculation by a neuroprocessor.

[0027]

According to the above construction, each processor element constituting the matrix operation unit of the neuroprocessor is assigned to each processor element by changing the coupling between the multiplier, the adder, the memory and the register by switching the selector. Since the matrix calculation can be performed most efficiently and flexibly, and the hardware can be effectively used by performing the calculation other than the matrix calculation in parallel by the auxiliary calculation device provided separately, the neural network can be executed by the relatively small hardware. It enables high speed processing of a huge amount of computation involved in learning.

[0028]

Embodiments of the present invention will be described below with reference to the drawings. A block diagram of a neuroprocessor that is an embodiment of the present invention is shown in FIGS. In FIG. 1A, the matrix calculation device 1 is a processor element (hereinafter PE).
2 to 10, memories 11 to 16 and adder 17
-22. Each PE is coupled to the adjacent PE in the row direction and the adjacent PE in the column direction, and the data can be transmitted to the PE in the adjacent row or column direction. The memories 11 to 16 are arranged on the input side of each row and each column of the PE arranged two-dimensionally, and the adder 1
Similarly, 7 to 22 are arranged on the output side of each row and each column of PEs arranged two-dimensionally. Each memory on the input side can transfer data to the first PE in each column or row. Further, each adder on the output side is capable of accumulatively adding the data of the calculation result transmitted thereto. Figure 1
The auxiliary arithmetic unit 23 shown in (a) is independent of the matrix arithmetic unit 1 and operates in parallel with the matrix arithmetic unit 1. The auxiliary arithmetic unit 23 has a function of calculating a part of the calculation of the neural network other than the matrix calculation.

Although FIG. 1A shows an example in which PEs are arranged in a two-dimensional matrix of 3 rows and 3 columns, the PE may be arranged in any number of rows and columns.

FIG. 1B is a block diagram showing the configuration of the PE used in the neuroprocessor of the present invention. Figure 1
In (b), the PE of this embodiment includes a multiplier 24, an adder 25, a memory 26, selectors 27 to 33, and a register 34.
˜40. Y _IN is a column-direction input terminal, Y _OUT is a column-direction output terminal, X _IN is a row-direction input terminal, and X _OUT is a row-direction output terminal. By these input / output terminals, the processor element is coupled to the adjacent processor elements in the row direction and the column direction. Further, by switching the selector, it is possible to select whether to output the data in the column direction or in the row direction.

The operation of the neuroprocessor of this embodiment will be described in detail below. In a neural network having one intermediate layer as in the example shown in FIG. 8, (Equation 1)
And (Equation 6) and (Equation 7) are shown in FIG. 2 (I = W × O),
Figure ^{3 (s = W T × δ} ), represented by the matrix equation shown in FIG. ^{4 (R = δ × O T} ). Here, ^T means that it is a transposed matrix. The three matrix calculations of FIGS. 2, 3 and 4 account for a large amount of the enormous amount of calculation of this kind of neural network. In FIGS. 2, 3, and 4, the connection weight W
Matrix and R matrix are 9 rows and 9 columns, and these matrices are further divided into 9 sub-matrices consisting of 3 rows and 3 columns. In addition, the matrix (vector) of O, I, s, and δ is also divided into three blocks according to the matrix of R. By this division, the calculation of the nine small matrices of W is executed corresponding to the nine PEs (FIG. 1A) arranged. That is,
Each sub-matrix w _nm of W is associated with PE _nm , and each element of the sub-matrix of W consisting of nine elements corresponding to the memory W (FIG. 1B) in each processor element is stored. The same applies to the R sub-matrix. Further, the block δ ₁ of δ is stored in the memory 11, the block δ ₂ of δ is stored in the memory 12,
The block δ ₃ of δ is stored in the memory 13 and the memory 1
4 stores the block O ₁ of O, and the memory 15 stores O
Block O ₂ is stored in the memory 16 and a small matrix O ₃ of O is stored in the memory 16.

Although the calculation function as hardware is made flexible by switching the selector, its effect will be explained by the calculation of I = W × O shown in FIG. The contact position of each selector of PE shown in FIG. 1B is D for selector 1, C for selector 2, E for selector 3, A for selector 4, B for selector 6, and selector 7.
Then switch to D. As a result, the actual PE configuration is as shown in FIG. 5, and the arithmetic functions are also the same as those in this figure. According to this configuration, the memory 1 shown in FIG.
4, memory 15 and memory 16 output O data O ₁ ,
O ₂ and O ₃ are transmitted in the row direction (lateral direction), and PE ₁₁
0 is input to Y _IN of PE ₁₂ and PE ₁₃ .
Then, in each PE, the transmitted O data and each P
Multiplies the value of W stored in the internal memory of E, and then Y
This multiplication result is added to the input from _IN and output to Y _OUT . The operation results transmitted in the column direction (longitudinal direction) via each PE are cumulatively added by the last adders 20 to 22 in each column, and as a result, the sum of products operation corresponding to the formula of FIG. 2 is executed. To be done.

Next, the flexibility of the method of adapting to another matrix calculation using the same hardware will be described by taking the calculation of I = W ^T × δ in FIG. 3 as an example. The contact position of each selector of the PE shown in FIG. 1B is switched to E for the selector 1, C for the selector 2, D for the selector 3, A for the selector 4, D for the selector 6, and A for the selector 7. As a result, the actual PE has the configuration shown in FIG. According to this configuration, the memory 11, the memory 12,
Data δ ₁ , δ ₂ , and δ ₃ of δ are transmitted from the memory 13 in the column direction (vertical direction), and 0 is input to X _IN of PE ₁₁ , PE ₂₁ , and PE ₃₁ . Then, each processor element multiplies the transmitted δ data by the value of W stored in the internal memory of each PE, adds the multiplication result to the input from X _IN , and outputs it to X _OUT . The calculation results transmitted in the row direction (horizontal direction) through PE are cumulatively added by the last adders 17 to 22 in each row, and as a result,
The sum of products operation corresponding to the formula of FIG. 3 is executed.

[0034] Finally, similarly illustrating the method of calculating the R = δ × O ^T FIG. The contact position of each selector of the PE shown in FIG. 1B is E for the selector 1, D for the selector 2, C for the selector 3, A for the selector 4, B for the selector 5, B for the selector 6, and A for the selector 7. Switch to. As a result, the processor element will have the same configuration as shown in FIG. In this configuration, the O data O ₁ , O ₂ , and O ₃ are transmitted in the row direction (horizontal direction) from the memory 14, the memory 15, and the memory 16, and the δ data δ ₁ from the memory 11, the memory 12, and the memory 13. ,
δ ₂ and δ ₃ are transmitted in the column direction (vertical direction), and in each PE, the transmitted O and δ data are multiplied to obtain R. When this R is obtained (Equation 6), when η is 1, R becomes equivalent to the weight correction amount ΔW, so PE
It is possible to correct the value of W by switching the selector and adding .DELTA.W to the value of W stored in the memory by the adder.

As described above, three matrix calculations are simultaneously performed in parallel. In this embodiment, the calculation inside each PE is executed via the register, so that it can be pipelined. Further, since the nine PEs can operate in parallel, the matrix calculation can be executed at a very high speed by executing pipeline processing in units of PEs in the row direction or the column direction.

Although an auxiliary arithmetic unit is used for calculations other than matrix calculation, there are mainly the following four kinds of such calculations. They are (1) a process of operating the characteristic function f of the neuron in the calculation of the forward propagation. (2) Calculation to find the difference of t-O when calculating the delta of the output layer.
(3) A process of multiplying the differential coefficient g of the characteristic function when calculating the delta of the intermediate layer. (4) A process of multiplying the differential coefficient g of the characteristic function when calculating the delta of the output layer. In order to perform such processing, the auxiliary arithmetic unit is composed of a memory for a look-up table of the characteristic function f of the neuron and its differential coefficient g, an adder, a multiplier, a data memory and the like.

The operation of the auxiliary arithmetic unit is performed as follows. First, (1) Adders 20 to 2 are added in the process of applying the characteristic function f of the neuron in the calculation of the forward propagation.
The result I of the multiply-accumulate operation output by 2 is taken in, and f (I) is obtained by the look-up table of the characteristic function f of the neuron.
And write the result in the memories 14 to 16. (2) In calculating the difference of t-O when calculating the delta of the output layer, the value of O read from the memories 14 to 16 and the value of t stored in the data memory inside the auxiliary arithmetic unit are The difference t-O is calculated by the adder, and the result is stored again in the data memory. Further, (3) in the process of multiplying the differential coefficient g of the characteristic function when calculating the delta of the intermediate layer, the value of O is read from the memories 14 to 16 and g (O) is read from the lookup table of the differential coefficient g of the characteristic function. Is calculated, the value of s output from the adders 17 to 19 is multiplied by the multiplier, and the value of δ as the multiplication result is written in the memories 11 to 13. (4) In the process of multiplying the differential coefficient g of the characteristic function when calculating the delta of the output layer, the value of O is read from the memories 14 to 16, and g (O) is read from the lookup table of the differential coefficient g of the characteristic function. And the value of t−O written in the data memory are multiplied, and the value of δ as the multiplication result is written in the memories 11 to 13. In these calculations,
The auxiliary computing device can operate in parallel independently of the matrix computing computing device.

As described above, in the neuroprocessor of the present invention, by switching the selector of each PE of the matrix operation unit, the coupling among the multipliers, adders, memories, and registers is changed to select the most efficient calculation configuration. it can. Furthermore, each PE can arbitrarily transmit the calculation result data in the column direction and the row direction, and the adder provided at the end of each column or each row can perform cumulative addition of the calculation results in the column direction and the row direction. , Matrix calculation can be performed at high speed. Furthermore, since an auxiliary computing device for performing calculations other than matrix calculations is separately provided, matrix calculations and other calculations can be executed in parallel, eliminating wasted hardware and enabling large amounts of calculations at high speed. Can be processed. As a result, neural network inference and learning calculations can be performed quickly with relatively small hardware.
It enables the user to expand the field of application of learning for creating a neural network that behaves in his or her preference.

Although the calculation of the neural network having the hierarchical structure has been described as an example in the present embodiment, the PE used can efficiently execute the arbitrary calculation by switching the selectors. It is obvious that matrix calculation can be performed at high speed. Further, it goes without saying that the matrix operation device of the present invention is effective for matrix calculations other than neural network calculations.

The neuroprocessor of the present invention may be a single chip, or may be configured on the same chip as other processors, arithmetic blocks and memories.

[0041]

As is apparent from the above embodiments, in the neuroprocessor of the present invention, various matrix calculations can be performed at high speed by switching the selectors constituting the processor elements, so that the inference and learning of the neural network can be performed. The calculation is effective for fast execution,
This provides a neuroprocessor whose hardware is significantly reduced.

[Brief description of drawings]

FIG. 1A is a block diagram of a neuroprocessor according to an embodiment of the present invention. FIG. 1B is a block diagram of a processor element according to an embodiment of the present invention.

FIG. 2 is a diagram showing a matrix calculation (1) formula of I = W × O4.

FIG. 3 is a diagram showing an equation of matrix calculation (2) s = W ^T × δ.

[4] matrix calculation (3) diagram shows the formula of R = δ × O ^T

FIG. 5 is a diagram showing a configuration of a processor element in the case of matrix calculation (1).

FIG. 6 is a diagram showing a configuration of a processor element in the case of matrix calculation (2).

FIG. 7 is a diagram showing a configuration of a processor element in the case of matrix calculation (3).

FIG. 8 is a diagram for explaining a model of a hierarchical neural network.

FIG. 9 is a diagram for explaining the function of neurons.

[Explanation of symbols]

 1 Matrix operation device 2-10 Processor element 11-16 Memory 17-22 Adder 23 Auxiliary operation device 24 Multiplier 25 Adder 26 Memory 27-33 Selector 34-40 Register

Claims (4)

[Claims] 1. A matrix calculation device for selectively performing matrix calculation of a neural network, and an auxiliary calculation device that operates independently of the matrix calculation device, wherein matrix calculation is focused on the matrix calculation device. , A neuroprocessor characterized in that calculations other than matrix calculations and matrix calculations are allocated to auxiliary arithmetic units and used in parallel. 2. A matrix operation device comprises a multiplier, an adder, a memory, a register, a selector and a plurality of output terminals,
By switching the contact position of the selector, it is possible to change the coupling among the multiplier, the adder, the memory, and the register, and has the function of outputting data from one of the plurality of output terminals selected by switching the selector. The neuroprocessor according to claim 1, wherein the neuroprocessor comprises a processor element. 3. The processor elements are arranged in a two-dimensional matrix, a memory is provided at the input side of each row and each column of the processor element, and the data is transmitted to the processor element. The neuroprocessor according to claim 2, wherein 4. Processor elements are arranged in a two-dimensional matrix form, an adder is provided on the output side of each row and each column of the processor element, and the addition result is cumulatively added in the row direction or the column direction. The neuroprocessor according to claim 2, further comprising a matrix operation device.