KR20040040075A - Hardware of reconfigurable and expandable neural networks - Google Patents

Abstract

PURPOSE: An expandable and reconfigurable neural network hardware is provided to offer a digital neural network hardware having availability as much as software by securing variability and expandability with the interconnecting bus between a ladder bus and a module, realizing a fast neural network operation through a parallel process and a pipeline process, and changing an architecture of a neural network. CONSTITUTION: A processing element includes SPEs(Synapse Processing Element)(10), an AFU(Activation Function Unit), and an LPE(Layer Processing Element)(20). The SPE has a multiplier multiplying a weight of a synapse included in a neuron and an accumulator performing an accumulating operation of the synapses. The AFU receives an operation result of the SPE and outputs an activation function value for the operation result. The LPE has a bus controller controlling a data transfer direction between processing elements. The interconnecting bus(30) connects between the SPEs and expands the synapse by transferring the accumulated synapse value to the near SPE. The ladder bus(40) transfers the operation result of the SPE to the LPE by connecting the SPE and the LPE.

Description

Neural Network Hardware with Reconfigurability and Expansion {HARDWARE OF RECONFIGURABLE AND EXPANDABLE NEURAL NETWORKS}

The present invention relates to neural network hardware having reconfigurability and expandability, and more particularly, to an expandable and reconfigurable neural network architecture (ERN), which is a new structure for hardware implementation of digital neural networks.

Neural networks are computational methods that mimic the human brain, unlike von Neumman-type computer structures that are based on logic and operate by programs. The differences between the human brain and the computer can be summarized in two main ways. The first is to perform operations in a massively distributed and parallel manner, and the second is to solve problems through learning instead of developing proactive programs for problems. The neural network is a computational method that mimics the characteristics of the human brain.

The definition of a general neural network can be summarized as follows.

“A structure composed of numerous interconnections between units (artificial neurons). Each unit has its own input and output characteristics, which are implemented as local operations or functions. Individual unit outputs are defined by their input and output characteristics and are interconnected with external inputs and adjacent units. Occasionally, the neural network can be arbitrarily constructed by hand, but in general, the overall function of the neural network is developed through various forms of learning. ”Fig. 1 shows the basic form of a neural network modeled by visual cells. have.

The development of the hardware neural network has a similar time to the development of digital computers by von Neumann type computers. In other words, the neural network computer was not developed to replace the existing von Neumann type computer, but is a different kind of von Neumann type computer from the effort to imitate the human brain structure. The first hardware implementation of this neural network computer was the MARK I Perceptron, developed in 1958 by F. Rosenblatt. Over the next 40 years, neural networks have evolved and decayed, but digital computers have evolved rapidly with the advent of ICs, while neural networks have not advanced enough to be applied to generalized problems.

Most of these early neural networks were made with analog technology. In general, analog neural networks have the advantage of making logical neural networks intuitive, and by not using a multiplier for weighting, a very fast parallel structure can be obtained while occupying a small area. On the other hand, it has not been successful as a commercial product due to the limitations of general versatility, difficult to store learned weight values, high error rate because it is sensitive to electrical noise and temperature, and difficult to interface with existing digital computers. Since then, the development of neural network chips has been somewhat stagnant until the mid 90's, but in the late 90's, research on neural network chips has begun to revitalize with the rapid development of digital semiconductor technology. With the development of ASIC technology and the emergence of FPGAs, it has become possible to implement fast and high-density digital VLSI, and many neural network chips have been developed using them, and commercial neural network chips have been actively developed. In addition, the emergence of high speed DSP enabled the implementation of neural networks using parallel DSP. Compared to analog neural network chip, it takes up a lot of area and has a slow speed, but it is easy to store weight value and can be easily manufactured by using existing digital processor technology, and it can have relatively free structure compared to analog neural network. There is an advantage.

There are two ways to implement digital neural networks. The first is a microprocessor-based neural network using DSP or CPU, and the second is an ASIC type that implements neural networks by directly constructing synapses and neurons. Digital processor-based neural networks have the advantage that they are smaller in area than the number of synapses that are implemented, and are easy to design because the existing processors are used as they are, and that various types of neural networks can be implemented just by changing the program. However, it is slower than the neural network implemented by ASIC, difficult to construct as a single chip, and the efficiency of parallel processing and distributed processing, which is a characteristic of neural network, is inferior. It is possible to implement various types of neural networks according to the purpose, and it is possible to implement neural networks in a form close to a theoretical model. However, it is difficult to change the shape of the neural network once implemented, and it is difficult to expand neural networks. .

The number of synapses increases in the form of the square of the number of neurons with respect to the increase of neurons in a neural network structure having a general fully connected (Fully Connected) form, in proportion to the amount of multipliers. The area consumption and speed decrease caused by the increase of the multiplier appear to be a greater disadvantage in the digital neural network than in the analog neural network without the multiplier. Research has been conducted to reduce the size of the multiplier while leaving the structure intact, such as a digital neural network using serial data. The other method uses a digital neural network that reuses a multiplier through a bus structure like the SIMD form, and a neural network that linearly increases the number of multipliers by using a neural network and a cellular neural network. As a neural network using Neighborhood Connected, research has been conducted to solve the problem of multiplication by increasing the number of neurons and synapses by implementing a modular architecture instead of physically implementing the existing fully connected structure.

As described above, the neural network has advantages such as parallelism, learning ability, fault tolerance, adaptability, etc., compared to the existing von Neumann type computers. It has the advantage of being able to solve problems that could only be solved by using difficult or complicated algorithms. As such, the neural network is advantageous in realizing functions such as association, inference, and learning because data is processed in parallel and distributedly, but its application range is limited and it is difficult to implement hardware. Therefore, it is not widely used as a practical hardware.

Nevertheless, since the neural network is based on parallel and distributed processing, the simulation in the existing digital computer alone is limited, and thus necessitates the actual hardware implementation of the neural network.

The present invention has been made to solve the above problems, by adopting a ladder bus (Interconnecting Bus) between the module (Ladder Bus) and the module (Interconnecting Bus) to secure variability and scalability, parallel processing and pipeline processing It is possible to provide high speed neural network operation, and to provide digital neural network hardware that is as useful as software by changing the structure of neural network through setting of configuration bit registers.

1 is a view showing the basic form of the neural network modeled on the visual cell,

2 is a block diagram of a basic processing module according to the present invention;

3 is a block diagram of an SPE according to the present invention;

4 is a block diagram of an LPE according to the present invention;

5 is a diagram illustrating expansion of a synapse using an interconnection bus;

6 shows module expansion using a ladder bus,

7 is a configuration diagram of the forward and reverse buses through the configuration bit registers of the LPE;

8 illustrates neuronal expansion within a layer.

<Description of the symbols for the main parts of the drawings>

(10) SPE (Synapse Processing Element): Synapse processing element

(20) Layer Processing Element (LPE): Layer Processing Element

(30) Interconnecting Bus

(40) Ladder Bus

(100) Basic Processing Module

In order to achieve the above object, the present invention provides a neural network hardware having one or more basic processing modules composed of a plurality of processing elements (PEs) and a plurality of data buses, wherein the processing elements are weights of synapses possessed by neurons. A plurality of synaptic processing elements (SPEs) having a multiplier for performing a multiplication operation and an accumulator for performing a cumulative sum operation of synapses; And a layer processing element (LPE) having an activation function unit (AFU) for receiving an operation result of the synaptic processing element and outputting an activation function value for the synaptic processing element and a bus controller for controlling a data transfer direction between the processing elements. The data bus includes: an internally connected bus connected between the synaptic processing elements and extending synapses by transferring the synaptic values accumulated in one synaptic processing element to adjacent synaptic processing elements; And a ladder bus which connects the synaptic processing element and the layer processing element to transfer the calculation result of the synaptic processing element to the layer processing element.

The present invention proposes a method for securing scalability and scalability through a modular neural network based on a variable structure.

The proposed processing module has a ladder bus structure as a whole and expands neurons or synapses by changing the direction of the ladder bus.

Each module is internally composed of two modules, a SPE (Synapse Processing Element) and an LPE (Layer Processing Element), and an interconnecting bus capable of transferring a cumulative sum of neurons synapses between adjacent SPEs. Is connected. By passing the cumulative sum of neurons through the interconnection bus, the number of synapses can be extended not only inside the module but also outside.

Finally, the LPE has the ability to output the neuron's activation function and to redirect the ladder bus. This feature allows neurons and layers to extend out of the module, and enables various forms of neural networks such as monolayers, multilayers and forward and reverse.

Referring to the operation model for the expansion of the synapse according to the configuration of the present invention.

First, in order to express the proposed structure formally, any i-th SPE (Synapse Processing Element) responsible for calculating the sum of products for synapses and any j-th LPE (except the bus controller) A computational model for Processing Element) is shown in Equation 1 below.

At this time, Is the maximum number of synapses that the SPE can handle. On the other hand, if the model for a general neuron is expressed by the equation can be expressed as Equation 2.

Likewise, Is the maximum number of synapses of one neuron, and f (y) in Equations 1 and 2 is an activation function of the neuron. Any within the scope of SPE A set containing Of the same condition A set containing Randomly within the range of the neuron A set containing Of the same condition A set containing In this case, each set may be represented by Equation 3 below.

SPE's relationship to neurons For According to the size of the equations are divided into two cases as shown in equations (4) and (5).

i) When the neuron is constructed using the SPE in the case of Equation 4, when the condition as shown in Equation 6 is satisfied, Equation 2 may be expressed as Equation 7.

,

Substituting the first expression of Equation 1 into Equation 7 results in the following Equation 8.

Eventually neuron output Can be obtained by inputting one SPE output to one LPE.

ii) In the case of Equation 5, the size of synapses of neurons is larger than that of SPEs. In this case, when trying to construct a neuron through the SPE, the condition of the collective relationship between the synapse of the neuron and the synapse of the SPE is expressed by Equation 9 below.

When the condition of Equation 9 is satisfied To If the quotient divided by k is r, the relationship is as shown in Equation 10 below.

By using Equation 2, Equation 2 can be summarized as Equation 11 below.

Substituting the first expression of Equation 1 into Equation 11 is as follows.

Equation (7) may be applied to the remaining terms as follows.

Where f (x) If replaced by the following equation (14).

Thus, the output of neurons Can be obtained by inputting the sum of the outputs generated by k + 1 SPEs into one LPE. Using this method, when the number of synapses of SPEs is smaller than the number of neurons as shown in Equation 5, multiple SPEs can be used to expand the synapses of a single neuron.

Hereinafter will be described in detail with reference to the accompanying drawings the configuration and its operation according to an embodiment of the present invention.

The present invention is based on the neural network of the SIMD (Single Instruction Multiple Data) structure and has a delay of 1 clock latency for data input through the pipeline structure, and the modular hardware design is applied. In order to expand the synapses, neurons, and layers of a neural network by transferring data in a master-slaved manner between adjacent processing elements (PEs) through a distributed control unit in a modular unit, It solves the problem of slowing rate, increase of control signal and nonlinear increase of synapse to neuronal expansion.

In addition, it uses a multi-bus that connects PEs through a ladder bus (40: Ladder Bus) and an interconnecting bus (30: Interconnecting Bus), and the bus direction of the ladder bus applied to each module and the internal connection between the PEs By reconfiguring whether the connection bus is connected through a reconfigurable hardware technique using a field programmable gate array (FPGA) method, the reconfiguration structure enables the structure of the entire neural network to be changed and expanded through simple manipulation of configuration bits. In addition, external expansion is also possible by connecting the chip and the chip as well as the unit inside the chip.

In the present invention, a circuit block capable of implementing one or multiple neurons constituting one layer is referred to as a processing module. The processing module is basically composed of two processing elements (PEs), a synapse function that multiplies the weights of input nodes, a synapse function that has a cumulative sum function of each synapse, and a result of accumulated synapses. This is the Layer Processing Element (LPE) that outputs the value of the activation function and controls the data transfer direction between PEs. 2 shows the structure of a basic processing module composed of four SPEs and one LPE.

Here, the SPE stores a weight value in a built-in memory, performs a multiplication on each synapse by using the stored weight value, and accumulates a multiplication result and receives and adds a result calculated from an adjacent SPE. It has a function to output the result to LPE. The internal block diagram of the SPE is shown in FIG.

As shown in FIG. 3, the calculator of the SPE basically includes an adder multiplier and an accumulator. SPE is designed with speed in mind because it is the dominant part of the computational speed of neurons. As a result, they all process parallel data, have a 4-put through-put to improve speed, and have one clock latency.

In addition, the function of the LPE is largely divided into two. The first is the AFU (Activation Function Unit), which outputs the neuron's activation function, and the second is a bus controller that changes the ladder bus. This LPE allows for the actual expansion of layers or neurons. 4 is a block diagram for an LPE.

The following describes the bus structure.

The overall data of the present invention is delivered through a multi-bus architecture. The data bus can be divided into three types: the first is the input / output bus used in the SIMD structure inside the module, the second is the internal connection bus 30 connected between the respective SPEs inside the module, and the third is between the modules. The ladder bus 40 in charge of the connection.

The interconnection bus 30 may extend the synapse by transferring the synaptic value accumulated in the SPE to the adjacent SPE. This is applicable not only inside the module but also between modules as shown in the processing module structure of FIG. Therefore, synaptic expansion is not limited to the inside of the module, it is possible to process the amount of synapses in excess of the module in one neuron. 5 shows a method for extending synapses using an interconnect bus. FIG. 5A illustrates a method of expanding a synapse in one module, and FIG. 5B illustrates a method of expanding a synapse using two modules.

The ladder bus 40 is capable of implementing various neural networks by changing the connection structure between the modules. The direction and control of the ladder bus is handled by the LPE. The configuration bits stored in the LPE determine the direction of the bus and the function of the activation function module (AFU). By setting the configuration bit, it is possible to use an output value from an own module or another module existing on the same layer as an input, not from an external or other module. This feature makes it possible to implement a backward neural network, such as a hopfield. Figure 6 shows a diagram of a module constructed by a ladder bus, Figure 7 shows the forward (a) and reverse (b) bus configurations by setting the LPE, and Figure 8 shows the neurons in a ladder form by the LPE. Here is an example of extending.

The data format of the present invention is divided into a 24-bit format and a 32-bit format. The 24-bit (ladder-type bus) format is the format of the neuron's input data, weights, and outputs of the neuron. The 32-bit (interconnect-bus) format is the output of the SPE, that is, the cumulative sum value.

In addition, the data bus of the module according to the present invention is a multi-bus type 24-bit data input bus, 24-bit data output bus, 32-bit cumulative sum input bus, 32-bit cumulative sum output bus, and 32 for extending synapses. It consists of six bits: a synaptic expansion input bus with a bit and a synaptic expansion output bus with a 32 bit. Table 1 below shows a summary of the data bus.

Bus name direction size Contents din in 24 Enter data into neural network module passin in 32 Input synaptic accumulation value from adjacent module sumin inout 32 Input SPE outputs from adjacent modules and internal SPEs dout inout 24 Data output or feedback input to neural network module passout out 32 Output Synapse Accumulation to Adjacent Modules sumout out 32 Connect with sumin of adjacent module (direction and function are opposite to connected module)

The control signal is divided into a SIMD type control signal, that is, a control signal input to all modules equally and a master-slaved type control signal affected by its neighboring module. Since each SPE or LPE communicates with the neighbor SPE or LPEs in a master slave mode, each module has control over its slave module. By using this structure, the number of control signals is increased, but there is an advantage that the module can be extended without limit to the number of PEs. Table 2 shows the types of control signals.

Signal name Direction Description clk in Clock rst in Reset ale in Address latch enable ld in Load data bus ndset in New data set cmode in Configuration mode wmode in Weight update mode ovin in Overflow in poe in Previous output enable prev_ae in Previous address enable next_ae out Next address enable ovfl out Overflow out noe out Next output enable

In order to define a configuration form and an extension form of the neural network according to the present invention, configuration bits must first be defined. The constituent bits of ERNA can be divided into constituent bits for SPE and LPE. 9 is a configuration bit of the SPE, Figure 10 is a description of the configuration bit of the LPE SPE module to determine whether to operate through the configuration bit of the SPE, whether to increase or decrease the accumulator, the use of the internal connection bus, LPE module The configuration bit of the LPE determines the data progress direction of the ladder-type variable bus and whether the AFU is operated.

The type of neural network configurable using the structure proposed in the present invention basically depends on configuration bits and a look-up table (LUT). In addition, the shape of the internal connection bus, the complete connection of the synapse and the partial connection affect the configuration. Forms configurable by such elements are SPE configuration bits (2 ⁴ ), LPE configuration bits (2 ² ), internally connected bus types (2 ¹ ), full connection of synapses, partial connections (2 ¹ ), at least 2 ⁸ = In fact, it is possible to combine them into many types of neural networks by the number of SPEs in the module, the output of nonlinear functions, and the configuration of TDNN (Time Delayed Neural Network) by partial feed back.

The present invention is not limited to the above-described specific preferred embodiments, and various modifications can be made by any person having ordinary skill in the art without departing from the gist of the present invention claimed in the claims. Of course, such changes will fall within the scope of the claims.

The present invention configured as described above can secure variability and scalability, enable high speed neural network operation, and can change neural network structure by setting configuration bit registers, thereby implementing neural network hardware having general purpose. It is a useful invention that can contribute to industrial development because it has high commercial value and has the advantage of a hardware chip, and can be used for real-time inference which is widely used in the fields of securities, finance, or intelligent services such as portable mobile devices.

Claims (8)

In a neural network hardware having one or more basic processing modules composed of a plurality of processing elements (PEs) and a plurality of data buses, The processing element includes: a plurality of synaptic processing elements (SPEs) having a multiplier for performing a weighted multiplication operation of a synapse of a neuron and an accumulator for performing a cumulative sum operation of synapses; And a layer processing element (LPE) having an activation function unit (AFU) for receiving an operation result of the synaptic processing element and outputting an activation function value for the synaptic processing element and a bus controller for controlling a data transfer direction between the processing elements. , The data bus includes: an internal connection bus connected between the synaptic processing elements and extending synapses by transferring the synaptic value accumulated in one synaptic processing element to an adjacent synaptic processing element; And a ladder bus which connects the synaptic processing element and the layer processing element to transfer the calculation result of the synaptic processing element to the layer processing element. The method of claim 1, The plurality of synaptic processing elements are arranged in a pipeline structure having a delay of one clock delay (Clock Latency) for the data input, the neural network hardware having a reconfigurable capacity and expansion capability. The method of claim 1, A neural network hardware with reconfigurability and expandability, further comprising: a modular control unit for transferring data between adjacent processing elements in a master-slave manner. The method of claim 1, The interconnection bus is reconfigurable and expandable neural network hardware, characterized in that the synaptic expansion between the main processing module as well as inside the basic processing module. The method of claim 4, wherein The internal connection bus is a neural network hardware having a reconfigurability and expandability, characterized in that for transmitting the result of the synaptic processing element in 32-bit format. The method of claim 1, The layer processing element has a means for storing a configuration bit, and according to the configuration bit to determine the transmission direction of the ladder bus and whether the activation function unit operating neural network with reconfigurability and expansion capability hardware. The method according to claim 1 or 6, The synaptic processing element has a means for storing a configuration bit, and according to the configuration bit reconfigurability characterized in that it determines whether the operation of the synaptic processing element, increase or decrease of the accumulator and the use of the internal connection bus and Neural network hardware with expandability. The method of claim 1, The ladder bus is a neural network hardware having a reconfigurability and expandability, characterized in that for transmitting the input data, the weight value, the output data of the neuron in a 24-bit format.