JP6960700B2 - Multicast Network On-Chip Convolutional Neural Network Hardware Accelerator and Its Behavior - Google Patents

Description

The present invention belongs to the communication technology area of integrated circuit network-on-chip, and is particularly related to a convolutional neural network hardware accelerator based on a multicast network-on-chip and its operation method.

Convolutional neural networks are derived from artificial neural networks. As a multi-layer sensing device, it has strong adaptability to parallel movement of figures, proportional zoom, and deformation of inclination, and can quickly extract the features of figures. Adopting a weighted value sharing network structure, this structure has strong similarity to the biological neural network structure, such a structure reduces the number of weighted values, reduces the complexity of the network model, and is more than the network. Inputting a dimensional image further clarifies this advantage, as the image is directly network input, avoiding the complex feature extraction and data reconstruction processes of traditional identification algorithms.

There are only ten types of numbers that are easy to write, but because the numbers are relatively easy to write, discrimination is relatively small, and the handwritten numbers are highly discriminatory and difficult to identify. The accuracy is relatively low. The identification accuracy is greatly improved by using a convolutional neural network, and the current identification rate has reached 99.33%. However, some identification processes require speed and accuracy of identification.

However, it takes time to identify a figure with a software-like convolutional neural network, the concurrency of the neural network cannot be sufficiently exhibited, and the actual requirements are not satisfied in terms of real-time performance and power consumption. In terms of graphic identification of convolutional neural networks based on hardware, the realization of neural networks with FPGAs is a new direction. In "FPGA-based Convolutional Neural Network Accelerator", Volume 43, Phase 1 of the Computer Engineering Journal, we submitted a method for realizing a convolutional neural network accelerator based on FPGA, and this accelerator has a relatively high speed in neural network calculations. Although acquired, the use of concurrency of convolutional calculation is not so high when calculating neural networks, so there is still a large interval to increase in terms of convolutional calculation, and at the same time, this accelerator does not separate communication and calculation and reuses it. The sex is not strong.

In order to solve the above-mentioned deficiencies in the current technology, the present invention submits a convolutional neural network hardware accelerator based on the multicast network on chip and its operation method, and uses the convolutional neural network hardware accelerator to turn on the multicast network. Expected to accelerate the processing of data transferred on the chip, the parallelism of the neural network is fully demonstrated, a faster numerical identification speed is obtained, and reusability and expandability are enhanced.

The technical method adopted by the present invention to achieve the above object is as follows:
The present invention provides a convolutional neural network hardware accelerator based on a multicast network on chip, said multicast network on chip is composed of an N × M lower layer two-dimensional network and an L × S upper layer network (M and N are 2). The above integers, L and S are one or more integers, and 1 ≦ L <N, 1 ≦ S <M).
The lower two-dimensional network has a total of N × M node routers and is divided into L × S blocks, and each block includes an intermediate node router and a plurality of ordinary node routers. Includes five horizontal forwarding directions, the intermediate node router contains five horizontal forwarding directions and one uplink forwarding direction, the upper network is each forwarding node router, and each of the forwarding node routers is five. Data is transferred in one horizontal transfer direction and one downward transfer direction, and the intermediate node router in each block and the corresponding transfer node router are interconnected, and five horizontal transfer directions are four in front, back, left, and right. One direction and one direction to transfer to the corresponding convolutional neural network hardware accelerator, the features of which are:
Each of the convolutional neural network hardware accelerators is connected to one node router in the lower two-dimensional network, and each of the convolutional neural network hardware accelerators includes a communication interface module, an input processing module, an output processing module, and the like. Includes data memory group, reconstructable compute module, memory control module, activation function module, weighted memory,
The convolutional neural network hardware accelerator is arranged so as to be divided into five modes at the initialization stage, and each mode has at least one convolutional neural network hardware accelerator.
In the first type mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n feature diagram memories (1 ≦ n ≦ N), and a result memory, and the target diagram. The data is written to the target diagram memory, the storage control module writes the first weighted data to the weighted memory, and a kind of activation function is set in the activation function module.
In the second mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n-1 feature diagram memory and a result memory, and the storage control module has a second weighting. Data is written to the weighted memory, the activation function is set in the activation function module, and the activation function is set.
In the third mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n-2 feature diagram memories and a result memory, and the storage control module has a third weighting. Data is written to the weighted memory, the activation function is set in the activation function module, and the activation function is set.
In the fourth mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n-2 feature diagram memories and a result memory, and the storage control module has a fourth weighting. Data is written to the weighted memory, the activation function is set in the activation function module, and the activation function is set.
In the fifth mode, the storage control module of the convolutional neural network hardware accelerator does not use the data memory group, the weighted memory, and the activation function module.
Individual convolutional neural network hardware accelerators are operated by the following method:
(1) Calculation of the first convolution layer:
When the current convolutional neural network hardware accelerator is in the first type mode, the storage control module reads the target diagram data from the target diagram memory, transfers the target diagram data to the output module and the reconstructable calculation module, and outputs the output. The processing module performs packaging processing of the target diagram data and transfers it to the communication interface module, and the reconstructable calculation module convolves and calculates the target diagram data and then transfers it to the activation function module to activate the activation function. The calculation is performed, the first feature diagram data calculated by the corresponding convolutional neural network hardware accelerator is acquired, and the storage control module stores the first feature diagram data in the first feature diagram memory.
When the current convolutional neural network hardware accelerator is in the second type mode, the input processing module receives the package data from the communication interface module in the first type mode, performs depackaging processing, and obtains the target diagram data. It is acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module convolves and calculates the target diagram data and then transfers it to the activation function module to perform the activation function calculation, and the convolutional neural network corresponds to the convolutional neural network. The first feature diagram data calculated by the hardware accelerator is obtained, and the storage control module stores the first feature diagram data in the first feature diagram memory.
(2) Calculation of the second convolution layer:
When the current convolutional neural network hardware accelerator is in first-class mode, the storage control module reads the first feature diagram data from the first feature diagram memory, and the reconstructable calculation module and output processing module. The reconstructable calculation module convolves and calculates the first feature diagram data, then transfers it to the activation function module to perform the activation function calculation, and the convolution neural network hardware accelerator calculates the first. The second feature diagram data is acquired and transferred to the storage control module, the storage control module stores the second feature diagram data in the second feature diagram memory, and the output module is the first feature diagram data. Is packaged and then transferred to the communication interface module.
When the current convolutional neural network hardware accelerator is in the second type mode, the storage control module reads the first feature diagram data from the first feature diagram memory and turns it into the reconstructable calculation module and output module. After transferring, the output processing module sequentially receives the package data in the first type mode and the second type mode, performs the depackaging process, and performs the first feature diagram data and the first feature diagram data in the first type mode. The first feature diagram data in the two modes is acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module convolves and calculates the first feature diagram data and then activates the activation. Transfer to the function module, perform activation function calculation, acquire the second feature diagram data calculated by the convolutional neural network hardware accelerator, transfer to the memory control module, and the memory control module has the second feature. The data is stored in the second feature diagram memory, and the output processing module packages the first feature diagram data and then transfers the data to the communication interface module.
When the current convolutional neural network hardware accelerator is in the third type mode, the input processing module sequentially receives the package data in the first type mode and the second type mode, performs the depackaging process, and performs the first type. The first feature diagram data of the type mode and the first feature diagram data of the second type mode are acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module is the first of the two modes. After each of the feature diagram data of the above is convolutionally calculated, it is transferred to the activation function module to perform the activation function calculation, and the second feature diagram data in the two modes is acquired and transferred to the storage control module. The storage control module stores the second feature diagram data in the two modes in the second feature diagram memory, and stores the second feature diagram data in the second feature diagram memory.
When the current convolutional neural network hardware accelerator is in the fourth type mode, the input processing module sequentially receives the package data in the first type mode and the second type mode, performs the depackaging process, and performs the first type. The first feature diagram data in the seed mode and the first feature diagram data in the second type mode are acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module is in two modes. After each of the first feature diagram data of the above is convolutionally calculated, it is transferred to the activation function module to perform the activation function calculation, the second feature diagram data in the two modes is acquired, and the data is transferred to the storage control module. Then, the storage control module stores the second feature diagram data in the two modes in the second feature diagram memory.
(3) Calculation of the first all connection layers:
When the current convolutional neural network hardware accelerator is in the second type mode, the storage control module reads the second feature diagram data from the second feature diagram memory and transfers it to the reconstructable calculation module. , The reconstructable calculation module transfers the convolutional result of the convolutional calculation of the second feature diagram data to the output processing module, and the output processing module packages the convolutional result and then transfers it to the communication interface module. death,
When the current convolutional neural network hardware accelerator is in the third type mode, the storage control module reads the second feature diagram data from the second feature diagram memory and transfers it to the reconstructable calculation module. , The reconstructable calculation module transfers the convolutional result of the convolutional calculation of the second feature diagram data to the output processing module, and the output processing module packages the convolutional result and then transfers it to the communication interface module. ,
When the current convolutional neural network hardware accelerator is in the fourth type mode, the storage control module reads the second feature diagram data from the second feature diagram memory and transfers it to the reconstructable calculation module. The reconstructable calculation module transfers the convolutional result of the convolutional calculation of the second feature diagram data to the output processing module, and the output processing module packages the convolutional result and then transfers it to the communication interface module.
When the current convolutional neural network hardware accelerator is in the 5th mode, the input processing module sequentially receives the package data in the 2nd mode, the 3rd mode and the 4th mode. The packaging process is performed, the convolution results in the three modes are acquired, and the data is transferred to the reconstructable calculation module. The reconstructable calculation module acquires the cumulative calculated results of the convolution results in the three modes. Then, the data is transferred to the output processing module, and the output processing module processes the cumulative result and then transfers the cumulative result to the communication interface module.
Otherwise, when the current convolutional neural network hardware accelerator is in first-class mode, the storage control module can read the second feature diagram data from the second feature diagram memory and reconstruct it. Transferred to the calculation module, the reconstructable calculation module convolves the second feature diagram data and transfers the convolutional calculation result to the reconstructable calculation module, and the input processing module sequentially performs in the fifth type mode. Receives the package data of, performs depackaging processing, acquires the cumulative result in the fifth type mode, transfers it to the reconstructable calculation module, and the reconstructable calculation module combines the cumulative result and the convolutional result. At the same time, the third feature diagram data cumulatively calculated is acquired and transferred to the storage control module, and the storage control module stores the third feature diagram data in the third feature diagram memory.
(4) Calculation of the second whole connection layer:
When the current convolution neural network hardware accelerator is in the mode of the first kind, the storage control module reads the third feature diagram data from the third feature diagram memory and performs the activation function calculation, respectively. It is transferred to the activation function module, and after obtaining the activation result, it is transferred to the reconstructable calculation module and the output processing module, and the reconstructable calculation module performs a convolution calculation on the activation result, and the first The first rudimentary result is acquired and transferred to the activation function module, the activation function calculation is performed, the first output result is acquired and transferred to the storage control module, and the storage control module is the first. One output result is written to the result memory, the output processing module packages the activation result, and then transfers the activation result to the communication interface module.
When the current convolution neural network hardware accelerator is in the second type mode, the input processing module sequentially receives the package data in the first type mode, performs the depackaging process, and obtains the activation result. The reconstructable calculation module performs a convolution calculation on the activation result, obtains a second elementary result, transfers the result to the activation function module, and performs the activation calculation. The second output result is acquired and transferred to the output processing module, and the output processing module processes the second output result and then transfers it to the communication interface module.
When the current convolution neural network hardware accelerator is in the 4th type mode, the input processing module sequentially receives the package data in the 1st type mode, performs the depackaging process, and obtains the activation result. The reconstructable calculation module performs a convolution calculation on the activation result, obtains a third rudimentary result, transfers the result to the activation function module, and activates the result. The conversion function calculation is performed, the third output result is acquired and transferred to the output processing module, and the output processing module processes the third output result and then transfers it to the communication interface module.
When the current convolutional neural network hardware accelerator is in the first type mode, the input processing module further sequentially receives the package data in the second type mode and the fourth type mode and performs the depackaging process. , The second output result and the third output result are acquired and transferred to the storage control module, and the storage control module sequentially transfers the second output result and the third output result to the result memory. The three output results indicate the probability of belonging to the type corresponding to the target diagram data.

The present invention also provides a method of operating a convolutional neural network hardware accelerator based on a multicast network on-chip. The multicast network-on-chip is composed of an N × M lower layer two-dimensional network and an L × S upper layer network (among them). M and N are integers of 2 or more, L and S are integers of 1 or more, and 1 ≦ L <N, 1 ≦ S <M).
The lower two-dimensional network has a total of N × M node routers and is divided into L × S blocks, and each block includes an intermediate node router and a plurality of ordinary node routers. The router includes five horizontal forwarding directions, the intermediate node router contains five horizontal forwarding directions and one uplink forwarding direction, the upper network is a forwarding node router, and the forwarding node router has five horizontal forwarding directions. Including direction and one downward transfer direction, the intermediate node router in each block and the corresponding transfer node router are interconnected, of which five horizontal transfer directions are front, back, left and right, four directions and the corresponding convolutional neural network hardware. It is one direction to transfer to the wear accelerator, and its features are as follows:
Each of the convolutional neural network hardware accelerators is connected to one node router in the lower two-dimensional network, and the process of the convolutional operation is as follows:
Step 1. Each of the convolutional neural network hardware accelerators is placed and divided into five modes at the initialization stage, and each mode has at least one convolutional neural network hardware accelerator.
Step 2, Convolutional Neural Network in First and Second Modes Perform the first convolutional calculation using the hardware accelerator.
Step 2.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator acquires the target diagram data, performs the convolution calculation, and then performs the activation function calculation. Obtained the first feature diagram data
Step 2.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator performs the activation function calculation after convolving the target diagram data. Acquire feature diagram data and
Step 3, perform the second convolutional layer calculation using the convolutional neural network hardware accelerator in the first, second, third and fourth modes.
Step 3.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator convolves and calculates the first feature diagram data, and further performs an activation function calculation. Acquired the second feature diagram data,
Step 3.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves the first feature diagram data and then performs the activation function calculation. Acquired the second feature diagram data,
Step 3.3, When the current convolutional neural network hardware accelerator is in the third type mode, the convolutional neural network hardware accelerator is in the first feature diagram data in the first type mode and the second type mode. Convolutional calculation of the first feature diagram data of each, and further activation function calculation to acquire the second feature diagram data in the two modes.
Step 3.4, When the current convolutional neural network hardware accelerator is in the 4th type mode, the convolutional neural network hardware accelerator is in the 1st feature diagram data and the 2nd type mode in the 1st type mode. After convolutional calculation of the first feature diagram data of, the activation function calculation is performed, and the second feature diagram data in the two modes is acquired.
Step 4, the first full connection layer calculation is performed using the convolutional neural network hardware accelerator in the first, second, third, fourth and fifth modes.
Step 4.1 When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.3, When the current convolutional neural network hardware accelerator is in the third type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.4, When the current convolutional neural network hardware accelerator is in the fourth type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.5, If the current convolutional neural network hardware accelerator is in mode 5, the convolutional neural network hardware accelerator is in mode 2, mode 3 and mode 4. After accumulating the convolutional results of
Step 4.6, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator further integrates the integration results in the fifth type mode, and then the third feature. Acquire figure data,
Step 5, perform the second full connection layer calculation using the convolutional neural network hardware accelerator in the first, second and fourth modes.
Step 5.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator calculates the activation function in the third feature diagram data and then calculates the activation result. Obtain and then perform a convolutional calculation of the activation result to obtain the first rudimentary result, and finally calculate the activation function of the first rudimentary result to obtain the first activation result. And store
Step 5.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves and calculates the activation result in the first type mode, and then the second type. Obtain the rudimentary result of, activate the second rudimentary result, perform the function calculation, and obtain the second output result.
Step 5.3, When the current convolutional neural network hardware accelerator is in the fourth type mode, the convolutional neural network hardware accelerator convolves and calculates the activation result in the first type mode, and then the third type. Obtain the rudimentary result of, calculate the activation function of the third rudimentary result, and obtain the third output result.
Step 5.4, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator is the second output result in the second type mode and the fourth type mode. And the third output result are stored, and each of the three output results shows the probability of belonging to the type corresponding to the target diagram data.
Each of the three output results shows the probability of belonging to the type corresponding to the target diagram data. Alternatively, the three output results are input to the next convolutional network or the next convolutional layer calculation.
Compared with the existing technology, the beneficial technical effects of the present invention are as follows:
1. The accelerator of the present invention greatly enhances the parallelism of the convolutional neural network calculation, the speed of the convolutional neural network calculation is clearly higher than that of the traditional convolutional neural network accelerator, and the present invention is different due to the reconstructable design. It is possible to make appropriate settings of the present invention for the calculation request of, and it has a reconstruction function which is not provided in the traditional convolutional neural network, and the strategy of separating calculation and communication is an extension of the present invention. The property and reusability are higher than those of traditional convolutional neural network hardware accelerators, and the scale of the calculated convolutional neural network can extend the present invention.

2. The present invention interconnects the reconstruction calculation module of the convolutional neural network hardware accelerator and the multicast network on-chip through the communication interface module and the input / output processing module, and communicates between a large number of convolutional neural network hardware accelerators. Realized, at the same time, the calculations of many convolutional neural network hardware accelerators are independent of each other, and such a design allows many convolutional neural network hardware accelerators to calculate one convolutional neural network at the same time, locally. The result of the calculation can be transferred to many other convolutional neural network hardware accelerators at once through the multicast network on-chip, and the next parallel calculation can be performed, such communication and calculation separation tricks and traditional. Compared with the convolutional neural network hardware accelerator, the concurrency of the calculation of the convolutional neural network is greatly increased, and the calculation speed of the convolutional neural network is increased. At the same time, the calculation and communication of the convolutional neural network are separated through the communication interface module and the input / output processing module, and the calculation and communication become mutually independent parts. Therefore, when the scale of the network on-chip is changed accordingly, it is not necessary to make settings only for the input / output processing module and newly design the reconstruction calculation module, which reduces the difficulty level at the time of expansion. Increased scalability and reusability of modules, overcoming the extensible and reusability flaws of traditional neural network accelerators.

3. The present invention adopts a reconstructable design, and the primitive convolutional neural network hardware accelerator can be set to 5 different operation modes, respectively, the convolutional neural network hardware accelerator in the first type mode, the first. Convolutional Neural Network Hardware Accelerator in 2 Modes, Convolutional Neural Network Hardware Accelerator in 3 Modes, Convolutional Neural Network Hardware Accelerator in 4 Modes, Convolutional Neural in 5 Modes It is a network hardware accelerator. The five operation modes are obtained by setting the primitive convolutional neural network hardware accelerator, and the realized functions are different from each other. For example, the convolutional neural network hardware accelerator in the first type operation mode inputs and convolves the target figure. It is possible to realize the output function of the calculation result of the neural network, and these two functions are not provided in the convolutional neural network hardware accelerator in other modes, and the convolutional neural network hardware accelerators in five different operation modes are mutually used. In combination, a complete convolutional neural network can be calculated, and if you want to calculate different convolutional neural networks with such a design, you only need to set the primitive convolutional neural network hardware accelerator to several different modes of operation. Different types of convolutional neural networks can be calculated with the appropriate combinations. The design of five different modes of operation provides the present invention with the advantage of being reconstructable over traditional convolutional neural network accelerators. At the same time, the functions of the convolutional neural network hardware accelerator with different calculation steps were simply classified by setting five different operation modes, and the operation efficiency of the convolutional neural network hardware accelerator was improved.

4. The activation function module of the present invention can be set to a different type of activation function logic, satisfying the calculation requirements of the corresponding convolutional neural network, and the present invention reiterates the activation function calculation process when calculating the convolutional neural network. Realized the possibility of construction.

Handwriting identification network flow diagram adopted in this embodiment Multicast network on-chip configuration diagram of lower layer 6x6 and upper layer 2x2 scale adopted in this embodiment Multicast network on-chip mapping diagram of this embodiment Reconstructable Convolutional Neural Network Hardware Accelerator Structure Diagram of the Present Invention Neural network hardware accelerator structure diagram in the first type mode Neural network hardware accelerator structure diagram in the second type mode Neural network hardware accelerator structure diagram in the third type mode Neural network hardware accelerator structure diagram in type 4 mode Neural network hardware accelerator structure diagram in the fifth type mode.

The purpose of this embodiment is to increase the concurrency of the convolutional neural network by adopting the multicast network on-chip architecture and to accelerate the speed of identification of handwritten Arabic numerals. The convolutional neural network adopted in this embodiment is a handwritten digit identification network, the structure of which includes a two-layer convolutional layer and two layers of all connecting layers, and the structure of the corresponding handwritten identification network is shown in FIG.

The handwriting recognition function of this embodiment is completed using the MNIST (Mixed National Institute of Standards and Technology database) handwriting library, and MNIST is a handwriting number training library built by Google Love and the Keron Institute of New York University. The entire training library includes a training library and a test library, the training library has 60,000 handwritten numeric images, and the test library has 10,000. The size of the MNIST handwritten numeric image is 28x28, and the features of the image are extended to a size of 29x29 through zero padding to reduce the edge effect of the extraction process. As described above, the number of input layer neural nodes is 29 × 29 equal to 841.

As shown in FIG. 1, the calculation of the corresponding convolutional neural network is a four-layer calculation, the calculation of the first layer is a convolutional layer calculation, the input layer is 841 neural nodes, and the output is six 13 × 13. There are 1041 neural nodes in total in the feature diagram, the second layer is a convolutional layer calculation, and the input layer is a total of 1041 neurals in 16 13 × 13 feature diagrams output by the first convolutional layer. It becomes a node, and the output is 50 5 × 5 feature diagrams with a total of 1250 neurons, the third layer is a total connection layer calculation, and the input layer is 50 5 5 output by the second convolutional layer. In the feature diagram of × 5, there are a total of 1250 neurons, the output is 100 neurons, the fourth layer is the total connection layer calculation, and the input layer is the 100 neurons output by the first all connection layer. Yes, the output is 10 neurons.

The present invention employs the interconnection of a multicast network on-chip and a convolutional neural network hardware accelerator to realize the calculation of a convolutional neural network. In this embodiment, the structure of the multicast network on-chip is shown in FIG. 2, where the circular icon represents the node router and the square icon represents the convolutional neural network hardware accelerator mounted on the node router, 6x6. It consists of a lower two-dimensional network and a 4x4 upper network, and includes a total of 40 node routers, of which the lower two-dimensional network has a total of 36 node routers and 3x3 children. is divided into four multicast area network unit, a four region surrounded by a dotted line in FIG. 2, within each region comprises one intermediate node router R ₃ and a plurality of ordinary nodes router R ₂ , Ordinary node router R ₂ contains five horizontal transfer directions, intermediate node router R ₃ contains five horizontal transfer directions and one uplink transfer direction, and the upper network is transfer node router R ₁ and the transfer node. Router R ₁ includes five horizontal forwarding directions and one downward forwarding direction, _{interconnecting the intermediate node router R 3} and the corresponding forwarding node router R ₁ in the individual regions, and forwarding the multicast network on-chip structure. Information can be transferred from one node router to many node routers at the same time, thus greatly increasing the efficiency of information transfer and increasing the speed of information transfer that can be reused during the convolutional neural network calculation process. , Increases the parallelism of convolutional neural network calculations.

The multicast network on-chip scale adopted in this example is set according to the size of the convolutional neural network used, and since the type of convolutional neural network used in the calculation differs in the actual situation, the multicast network is used according to the actual situation. It is a dual-layer network structure that sets the scale of on-chip and includes, but is not limited to, a 6 × 6 lower layer two-dimensional network and a 4 × 4 upper layer network.

Each convolutional neural network hardware accelerator is connected to a 36-node router in the lower two-dimensional network, and the internal structure of the convolutional neural network hardware accelerator is shown in Fig. 4, which shows the communication interface module, input processing module, output processing module, and data. Includes memory group, restructureable compute module, memory control module, activation function module, weighted memory,
At the initialization stage, the convolutional neural network hardware accelerator is set to 5 modes:
In the first type of mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into the target diagram memory, the three feature diagram memories (feature diagram 1, feature diagram 2, and feature diagram 3, respectively) and the result memory. It is divided, the target diagram data is written to the target diagram memory, the storage control module writes the first weighted data to the weighted memory, and the activation function module sets the activation function to the ReLU (Refectied Liner Units) function. The structure of the accelerator in this mode is shown in FIG.
In the second mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, two feature diagram memories (feature diagram 1 and feature diagram 2, respectively) and a result memory. The storage control module writes the second weighted data to the weighted memory, sets the activation function in the activation function module to the ReLU (Refectied Liner Units) function, and the structure of the accelerator in this mode is shown in FIG.
In the third mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into the target diagram memory, one feature diagram memory (feature diagram 2) and the result memory, and the memory control module is the first. The three weighted data are written to the weighted memory, the activation function is set to the ReLU (Rectifier Liner Units) function in the activation function module, and the structure is shown in FIG.
In the fourth type mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into the target diagram memory, one feature diagram memory (feature diagram 2), and the result memory, and the memory control module is the first. The weighted data of 4 is written to the weighted memory, the activation function is set to the ReLU (Rectifier Liner Units) function in the activation function module, and the specific structure is shown in FIG.
In the fifth mode, the storage control module of the convolutional neural network hardware accelerator does not use the data memory group, the weighted memory and the activation function module, and its structure is shown in FIG.
In this example, the convolutional neural network hardware accelerators in the five modes are mounted on the node routers corresponding to the multicast network on-chip, and the specific mapping method of the multicast network-on-chip is shown in FIG. In addition, the upper number on the node router icon indicates the mode number of the convolutional neural network hardware accelerator mounted on the node router, and if there is no upper number, the corresponding node router and the convolutional neural network hardware mounted on it. Not communicating with the wear accelerator, one convolutional neural network hardware accelerator in its first mode, five convolutional neural network hardware accelerators in its second mode, third mode There are 16 convolutional neural network hardware accelerators in, 4 convolutional neural network hardware accelerators in the 4th mode, and 4 convolutional neural network hardware accelerators in the 5th mode. Among them, the multicast network on-chip upper layer network does not mount the convolutional neural network hardware accelerator. The convolutional neural network hardware accelerator mapping method adopted in this embodiment is set according to the scale of the handwritten identification convolutional neural network hardware accelerator used, and the present invention has five different numbers depending on the different convolutional neural networks. The convolutional neural network hardware accelerator in the mode of is adopted, and the mapping method adopted in this embodiment and the number of convolutional neural network accelerators in various modes are included, and the present invention is not limited to that. Complete convolutional neural network calculations can be achieved by combining neural network hardware accelerators in five different modes of operation, and such a design allows for primitive convolutional neural network hardware accelerators when calculating different convolutional neural networks. Can be combined and calculated for different types of convolutional neural networks by simply setting to a number of different modes of operation.

First convolution layer calculation:
When the current convolutional neural network hardware accelerator is in first-class mode, the storage control module reads the target diagram data from the target diagram memory and transfers it to the output processing module and the restructible calculation module, and the output processing module is the target. The figure data is packaged, the target figure data and the node information for transfer are packaged, transferred to the communication interface module, transferred from the communication interface module to the multicast network on-chip, and the second type through the multicast network on-chip. Transfer to the communication interface module of the node that mounts the convolutional neural network hardware accelerator in the mode, the restructible calculation module receives the target diagram data, and at this time the path selection device of the reconstructable calculation module is the multiplication logic. Connect to the unit, perform multiplication logic calculation on the target diagram data, perform cumulative logic calculation (that is, complete convolution calculation), and the reconstructable calculation module is activated after the convolution calculation is completed on the target diagram data. In this embodiment, two multipliers are built in the multiplication logic unit in order to transfer to the multiplication function module to perform ReLU activation function calculation, acquire the first feature diagram data, and increase the parallelism of the convolution calculation. The multiplication calculation can be performed at the same time, and at the time of the convolution layer calculation of the first layer, the two multipliers perform the multiplication logic calculation on the target diagram data in parallel, and the present invention is a multiplication logic unit. Can incorporate a large number of multipliers and includes the two multipliers of this embodiment, but not limited to that, the parallel processing of a large number of multipliers increases the degree of parallelism during convolution calculations. Increase and increase the speed of convolution calculation. After that, the storage control module receives the first feature diagram data transferred from the activation function, and then stores the first feature diagram data in the first feature diagram memory.
When the current convolution neural network hardware accelerator is in the second type mode, the input processing module receives the package data transferred by the communication interface module in the first type mode, performs depackaging processing, and is in the package data. The target diagram data of the After performing the multiplication logic calculation and the cumulative logic calculation (that is, the complete convolution calculation), the data is transferred to the activation function module to perform the ReLU activation function calculation, the first feature diagram data is acquired, and the storage control module is the activation function. Receives the first feature diagram data transferred from the module, writes the first feature diagram data to the first feature diagram memory, and
Second convolution layer calculation:
When the current convolution neural network hardware accelerator is in the first type mode, the memory control module reads the first target diagram data from the first target diagram memory, and the path selection unit of the reconstructable calculation module is Receives the first feature diagram data, connects it to the multiplication logic unit, performs multiplication logic calculation and cumulative logic calculation (that is, complete convolution calculation) on the first feature diagram data, and then transfers it to the activation function module for ReLU activity. The conversion function calculation is performed, the second feature diagram data is acquired and transferred to the storage control module, and the memory control module receives the second feature diagram data transferred from the activation function module, and then transfers the second feature diagram data. Stored in two feature diagram memories, the output processing module performs packaging processing of the first feature diagram data, packages the first feature diagram data and transfer target node information, transfers it to the communication interface module, and communicates. Transfer from the module onto the multicast network on-chip and transfer through the multicast network-on-chip to the communication interface module of the destination node with the convolutional neural network hardware accelerators of the third and fourth modes mounted.
When the current convolutional neural network hardware accelerator is in the second mode, the memory control module reads the first target diagram data from the first target diagram memory and turns it into an output processing module and a restructurable calculation module. Transfer, the input processing module receives the package data in the first type mode and the other second type mode, performs the depackaging process, and performs the first feature diagram data of the data package and the second type mode. The first feature diagram data is extracted, the first feature diagram in the first type mode and the first feature diagram in the second type mode are acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module The passage selection unit receives the first feature diagram, connects it to the multiplication logic unit, performs multiplication logic calculation and cumulative logic calculation (that is, complete convolution calculation) on the first feature diagram data, respectively, and then turns it into an activation function module. Transfer and perform ReLU activation function calculation, acquire the second feature diagram data and transfer it to the storage control module, the storage control module stores the second feature diagram data in the second feature diagram memory, and the output processing module Packages the first feature diagram data, packages the first feature diagram data and transfer target node information, transfers it to the communication interface module, transfers it from the communication interface module to the multicast network on-chip, and then transfers it to the multicast network on-chip. Transfer to the communication interface module of the target node of the convolutional neural network hardware accelerator in the third and fourth types of modes through
When the current convolution neural network hardware accelerator is in the third type mode, the input processing module sequentially receives the package data in the first type mode and the second type mode, performs the depackaging process, and performs the data. The first feature diagram data of the package and the first feature diagram data in the second type mode are extracted, and the first feature diagram data in the first type mode and the first feature diagram data in the second type mode are extracted. Acquired and transferred to the reconstructable calculation module, the path selection unit of the reconstructable calculation module receives the first feature diagram in the two modes, connects to the multiplication logic unit, and multiplies each of the first feature diagram data. After performing logic calculation and cumulative logic calculation (that is, complete convolution calculation), transfer to the activation function module to perform ReLU activation function calculation, acquire the second feature diagram data in the two modes, and use it as the storage control module. After receiving the second feature diagram data transferred from the activation function, the storage control module stores the second feature diagram data in the two modes in the second feature diagram memory.
When the current convolution neural network hardware accelerator is in the 4th mode, the input processing module sequentially receives the package data in the 1st mode and the 2nd mode, performs the depackaging process, and performs the data package. The first feature diagram data in the first type mode and the first feature diagram data in the second type mode are extracted, and the first feature diagram data in the first type mode and the first feature diagram data in the second type mode are extracted. The first feature diagram data is acquired and transferred to the reconstructable calculation module, and the path selection unit of the restructureable calculation module receives the first feature diagram data in the two modes, connects to the multiplication logic unit, and the first feature. After performing multiplication logic calculation and cumulative logic calculation (that is, complete convolution calculation) on the figure data, transfer it to the activation function module to perform ReLU activation function calculation, and perform the second feature diagram data in the two modes. It is acquired and transferred to the memory control module, and after receiving the second feature diagram data transferred from the activation function, the memory control module stores the second feature diagram data in the two modes in the second feature diagram memory. ,
First all connection layer calculation:
When the current convolutional neural network hardware accelerator is in first-class mode, the storage control module reads the second feature diagram data from the second feature diagram memory, transfers it to the restructurable calculation module, and reconstructable computation. The passage selection unit of the module receives the second feature diagram data, connects it to the multiplication logic unit, performs multiplication logic calculation and cumulative logic calculation (that is, complete convolution calculation) on the second feature diagram data, and obtains the convolution result. Transfer to the output processing module, the output processing module performs packaging processing of the convolution result, packages the convolution result and transfer target node information and transfers it to the communication interface module, and the communication interface module transfers it to the multicast network on-chip. , Transfer to the communication interface module of the corresponding destination node through the multicast network on-chip,
When the current convolutional neural network hardware accelerator is in the second mode, the memory control module reads the second feature diagram data from the second feature diagram memory and transfers it to the restructurable calculation module for reconstructable calculation. The passage selection unit of the module receives the second feature diagram data, connects it to the multiplication logic unit, performs multiplication logic calculation and cumulative logic calculation (that is, complete convolution calculation) on the second feature diagram data, and obtains the convolution result. Transfer to the output processing module, the output processing module performs packaging processing of the convolution result, packages the convolution result and transfer target node information and transfers it to the communication interface module, and the communication interface module transfers it to the multicast network on-chip. Transfer to the communication interface module of the target node with the convolutional neural network hardware accelerator mounted in the fifth mode through the multicast network on-chip.
When the current convolutional neural network hardware accelerator is in the third mode, the storage control module reads the second feature diagram data from the second feature diagram memory and transfers it to the restructurable calculation module for reconstructable calculation. The passage selection unit of the module receives the second feature diagram data, connects it to the multiplication logic unit, performs multiplication logic calculation and cumulative logic calculation (that is, complete convolution calculation) on the second feature diagram data, and obtains the convolution result. Transfer to the output processing module, the output processing module performs packaging processing of the convolution result, packages the convolution result and transfer target node information and transfers it to the communication interface module, and the communication interface module transfers it to the multicast network on-chip. Transfer to the communication interface module of the target node with the convolutional neural network hardware accelerator mounted in the fifth mode through the multicast network on-chip.
When the current convolutional neural network hardware accelerator is in type 4 mode, the memory control module reads the second feature diagram data from the second feature diagram memory and transfers it to the restructurable calculation module for reconstructable computation. The passage selection unit of the module receives the second feature diagram data, connects it to the multiplication logic unit, performs multiplication logic calculation and cumulative logic calculation (that is, complete convolution calculation) on the second feature diagram data, and obtains the convolution result. Transfer to the output processing module, the output processing module performs packaging processing of the convolution result, packages the convolution result and transfer target node information and transfers it to the communication interface module, and the communication interface module transfers it to the multicast network on-chip. Transfer to the communication interface module of the target node with the convolutional neural network hardware accelerator mounted in the fifth mode through the multicast network on-chip.
If the current convolutional neural network hardware accelerator is in type 5 mode, the input processing module will sequentially receive and depackage the package data in type 2 mode, type 3 mode, and type 4 mode. Performs processing, extracts the convolutional result in the third type mode of the data package, acquires the third type convolutional result and transfers it to the reconstructable calculation module, and there are three types of passage selection units in the reconstructable calculation module. Receives the convolutional result in the mode of, connects to the cumulative logic calculation unit, cumulatively calculates the convolutional result in the three types of modes, then acquires the cumulative result and transfers it to the output processing module, and the output processing module transfers the cumulative result. Performs packaging processing, packages the cumulative result and transfer destination node information, and transfers it to the communication interface module.
When the current convolutional neural network hardware accelerator is in first-class mode, the memory control module can still read the second feature diagram data from the second feature diagram memory and transfer it to the restructurable calculation module for restructuring. The calculation module performs convolutional calculation on the second feature diagram data and then transfers the convolutional result to the reconstructable calculation module, and the input processing module sequentially receives the package data in the fifth type mode and depackages processing. Is performed, the cumulative result in the fifth type mode is acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module performs cumulative calculation on the cumulative result and the convolutional result at the same time and performs the third feature diagram data. It is acquired and transferred to the storage control module, and the storage control module stores the third feature diagram data in the third feature diagram memory.
Calculation of the second whole connection layer:
When the current convolutional neural network hardware accelerator is in the first type mode, the memory control module reads the third feature diagram data from the third feature diagram memory and activates each to perform the ReLU activation function calculation. Transfer to the function module, get the activation result, then transfer to the reconstructable calculation module and output processing module, the path selection unit of the reconstructable calculation module receives the activation result, connect it to the multiplication logic unit, Multiply logic calculation and cumulative logic calculation (that is, complete convolution calculation) are performed on the activation result, the first rudimentary result is obtained, transferred to the activation function module, and ReLU activation function calculation is performed. One output result is acquired and transferred to the storage control module, the storage control module receives the first output result, then writes the first output result to the result memory, and the output processing module performs packaging processing on the activation result. Perform, package the activation result and transfer destination node information, then transfer to the communication interface module, transfer to the multicast network on-chip, and through the multicast network-on-chip in the second and fourth modes. Transfer to the communication interface module of the target node with the convolutional neural network hardware accelerator mounted,
When the current convolution neural network hardware accelerator is in the second type mode, the input processing module sequentially receives the package data in the first type mode, performs depackaging processing, and extracts the activation result of the data package. Then, the activation result is acquired and transferred to the reconstructable calculation module, and the path selection unit of the reconstructable calculation module receives the activation result, connects to the multiplication logic unit, and performs the multiplication logic calculation on the activation result. Perform cumulative logic calculation (ie complete convolution calculation), obtain the second rudimentary result and transfer it to the activation function module to perform ReLU activation function calculation, obtain the second output result and transfer it to the output processing module. Then, the output processing module performs packaging processing on the second output result, packages the second output result and the transfer target node information and transfers it to the communication interface module, and the communication interface module transfers it to the multicast network on-chip. Then, through the multicast network on-chip, transfer the convolutional neural network hardware accelerator in the first-class mode to the communication interface module of the target node on which it is mounted.
When the current convolution neural network hardware accelerator is in the 4th mode, the input processing module sequentially receives the package data in the 1st mode, performs the depackaging process, and extracts the activation result of the data package. Then, the activation result is acquired and transferred to the reconstructable calculation module, and the path selection unit of the reconstructable calculation module receives the activation result, connects to the multiplication logic unit, and performs the multiplication logic calculation on the activation result. Perform cumulative logic calculation (ie complete convolution calculation), obtain the third rudimentary result and transfer it to the activation function module to perform ReLU activation function calculation, obtain the third output result and transfer it to the output processing module. Then, the output processing module performs packaging processing on the third output result, packages the third output result and the transfer target node information and transfers it to the communication interface module, and the communication interface module transfers it to the multicast network on-chip. Then, through the multicast network on-chip, transfer the convolutional neural network hardware accelerator in the first-class mode to the communication interface module of the target node on which it is mounted.
In addition, when the current convolutional neural network hardware accelerator is in the first type mode, the input processing module further sequentially receives the package data in the second type mode and the fourth type mode and performs the depackaging process. , The second output result and the third output result of the data package are extracted, the second output result and the third output result are acquired and transferred to the memory control module, and the memory control module sequentially outputs the second output result and the third output result. Write the result to the result memory,
Each of the three output results shows the probability of belonging to the type corresponding to the target diagram data. Alternatively, the three output results can be inputs to the next convolutional network or the next convolutional layer calculation.

In this embodiment, the multicast network on-chip and the convolutional neural network hardware accelerator are interconnected to realize the calculation of the convolutional neural network. In this embodiment, the structure of the multicast network on-chip is shown in FIG. 2, in which the circular icon represents the node router and the square icon is the convolutional neural network hardware accelerator mounted on the node router, 6x6. It consists of a lower two-dimensional network and a 4x4 upper layer network, and includes a total of 40 node routers. The middle and lower layer two-dimensional network has a total of 36 node routers and has a 3x3 child network. divided into four multicast area to the unit, would be four area surrounded by a dotted line in FIG. 2, within each region comprises one intermediate node router R ₃ and a plurality of ordinary nodes router R ₂ , Ordinary node router R ₂ has five horizontal transfer directions, intermediate node router R ₃ has five horizontal transfer directions and one uplink transfer direction, and the upper network is transfer node router R ₁ and transfer node. Router R ₁ has five horizontal forwarding directions and one downward forwarding direction, intermediate node router R ₃ in each region interconnects with the corresponding forwarding node router R _1, and the multicast network on-chip structure is forwarding. It is possible to transfer the information to be transferred from one node router to many node routers at the same time, thus greatly increasing the efficiency of information transfer and increasing the speed of transfer of reusable information in the convolutional neural network calculation process. , Increases the parallelism of convolutional neural network calculations.

The multicast network on-chip scale adopted in this example is set according to the size of the convolutional neural network used, and since the type of convolutional neural network used in the calculation differs in the actual situation, the multicast network depends on the actual situation. It is a dual-layer network structure that sets the scale of on-chip and includes, but is not limited to, a 6 × 6 lower layer two-dimensional network and a 4 × 4 upper layer network.

Each convolutional neural network hardware accelerator is connected to 36 node routers in the lower 2D network.
The operation method is as follows:
Step 1. Place the convolutional neural network hardware accelerator and divide it into 5 modes at the initialization stage.
Step 2, Convolutional Neural Network in First and Second Modes The hardware accelerator performs the first convolutional layer calculation:
Step 2.1, If the current convolutional neural network hardware accelerator is in first-class mode, the convolutional neural network hardware accelerator acquires the target diagram data, performs the convolutional calculation, and then calculates the activation function. Acquire one feature diagram data,
Step 2.2, If the current convolutional neural network hardware accelerator is in the second mode, the convolutional neural network hardware accelerator performs the convolutional calculation on the target diagram data and then the activation function calculation. Acquire one feature diagram data,
Step 3, Perform a second convolutional layer calculation using a convolutional neural network hardware accelerator in the first, second, third and fourth modes:
Step 3.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator convolves the first feature diagram data, performs the activation function calculation, and then performs the second. Acquire feature diagram data and
Step 3.2. When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves the first feature diagram data and then performs the activation function calculation. (2) Acquire feature diagram data and
Step 3.3, If the current convolutional neural network hardware accelerator is in the third mode, the convolutional neural network hardware accelerator is the first feature diagram data in the first mode and the second mode. The first feature diagram data is convolutionally calculated, and the activation function calculation is further performed to acquire the second feature diagram data in the two modes.
Step 3.4, If the current convolutional neural network hardware accelerator is in the 4th mode, the convolutional neural network hardware accelerator is the 1st feature diagram data in the 1st mode and the 2nd mode. After convolutional calculation of the first feature diagram data, the activation function calculation is performed to acquire the second feature diagram data of the two modes.
Step 4, perform the first full connection layer calculation using the convolutional neural network hardware accelerator in the first, second, third, fourth and fifth modes.
Step 4.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data and then obtains the convolution result.
Step 4.2, If the current convolutional neural network hardware accelerator is in the second mode, the convolutional neural network hardware accelerator performs the convolutional calculation on the second feature diagram data and then obtains the convolutional result.
Step 4.3, If the current convolutional neural network hardware accelerator is in the third mode, the convolutional neural network hardware accelerator convolutionally calculates the second feature diagram data and then obtains the convolutional result.
Step 4.4, if the current convolutional neural network hardware accelerator is in type 4 mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data and then obtains the convolution result.
Step 4.5, If the current convolutional neural network hardware accelerator is in mode 5, the convolutional neural network hardware accelerator convolves in mode 2, mode 3 and mode 4. After accumulating the results, obtain the integrated results and
Step 4.6, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator further integrates the integration results in the fifth type mode, and then the third feature diagram. Acquire data,
Step 5, perform the second full connection layer calculation using the convolutional neural network hardware accelerator in the first, second and fourth modes.
Step 5.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator calculates the activation function for the third feature diagram data and then calculates the activation result. Obtain, convolve the activation result, perform the calculation, obtain the first rudimentary result, and finally perform the activation function calculation on the first rudimentary result to obtain the first activation result. Store and
Step 5.2, If the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves the activation result in the first type mode and then calculates the second type. Obtain the rudimentary result, activate the second rudimentary result, perform the function calculation, obtain the second output result,
Step 5.3, If the current convolutional neural network hardware accelerator is in the 4th mode, the convolutional neural network hardware accelerator convolves the activation result in the 1st mode and then calculates the third. Obtain a rudimentary result, perform an activation function calculation on the third rudimentary result, and obtain a third output result.
Step 5.4, If the current Convolutional Neural Network Hardware Accelerator is in Type 1 mode, then the Convolutional Neural Network Hardware Accelerator is in Type 2 mode and Type 4 mode with a second output result and a second mode. Store the output result of the third.

For example, each of the three output results indicates the probability of belonging to the type corresponding to the target diagram data.

The corresponding operation method greatly reduces the identification time of numeric figures, the reconstructable design enhances scalability, data is transferred through the multicast network on-chip, calculation and communication are completely separated, and the reuse range is calculated as a unit. The level of reusability has been greatly increased by expanding the reusability of all calculation and communication units to the reusable level.

表２で示すように、本発明は１枚の図形の識別の所要時間が０．１６７３５ｍｓであり、その中の第一層の畳み込み層の所要時間が０．０３４８５ｍｓであり、第二層の畳み込み層の所要時間が０．０９５４３ｍｓであり、第一層の全接続層の所要時間が０．０３５２２ｍｓであり、第二層の全接続層の所要時間が０．００１８５ｍｓである。本発明は１０枚の図形の識別の所要時間が１．６７３５ｍｓであり、その中第一層の畳み込み層の所要時間が０．３４８５ｍｓであり、第二層の畳み込み層の所要時間が０．９５４３ｍｓであり、第一層の全接続層の所要時間が０．３５２２ｍｓであり、第二層の全接続層の所要時間が０．０１８５ｍｓである。本発明は１００枚の図形の識別の所要時間が１６．７３５ｍｓであり、その中第一層の畳み込み層の所要時間が３．４８５ｍｓであり、第二層の畳み込み層の所要時間が９．５４３ｍｓであり、第一層の全接続層の所要時間が３．５２２ｍｓであり、第二層の全接続層の所要時間が０．１８５ｍｓである。

As shown in Table 1, the simulated time for identifying one figure is 0.16735 ms, and the time required for the CNN convolutional neural network based on FPGA in the experiment to identify one figure is 0. Compared with .272 ms, the present invention increased the speed of identifying one figure by 38% compared to the comparative test, and in the example of the present invention, the simulated time of identifying 10000 figures was 1673.5 ms. Compared with the time required for the CNN convolutional neural network based on FPGA in the experiment to identify 10,000 figures to 2912 ms, the present invention is 43% faster than the comparative test to identify 10,000 figures. Higher

As shown in Table 2, the present invention takes 0.16735 ms for identifying one figure, 0.03485 ms for the first layer convolution layer, and 0.03485 ms for the second layer convolution. The time required for the layers is 0.09543 ms, the time required for all the connecting layers of the first layer is 0.03522 ms, and the time required for all the connecting layers of the second layer is 0.00185 ms. In the present invention, the time required for identifying 10 figures is 1.6735 ms, the time required for the first layer convolution layer is 0.3485 ms, and the time required for the second layer convolution layer is 0.9543 ms. The time required for all the connecting layers of the first layer is 0.3522 ms, and the time required for all the connecting layers of the second layer is 0.0185 ms. In the present invention, the time required for identifying 100 figures is 16.735 ms, the time required for the first layer convolution layer is 3.485 ms, and the time required for the second layer convolution layer is 9.543 ms. The time required for all the connecting layers of the first layer is 3.522 ms, and the time required for all the connecting layers of the second layer is 0.185 ms.

Claims (2)

It is a convolutional neural network hardware accelerator based on a multicast network on-chip, and the multicast network-on-chip is composed of an N × M lower layer two-dimensional network and an L × S upper layer network (M and N are integers of 2 or more). Yes, L and S are integers greater than or equal to 1, and 1 ≦ L <N, 1 ≦ S <M).
The lower two-dimensional network has a total of N × M node routers and is divided into L × S blocks, and each block includes an intermediate node router and a plurality of ordinary node routers. Includes five horizontal forwarding directions, the intermediate node router contains five horizontal forwarding directions and one uplink forwarding direction, the upper network is each forwarding node router, and each of the forwarding node routers is five. Data is transferred in one horizontal transfer direction and one downward transfer direction, and the intermediate node router in each block and the corresponding transfer node router are interconnected, and five horizontal transfer directions are four in front, back, left, and right. One direction and one direction to transfer to the corresponding convolutional neural network hardware accelerator, the features of which are:
Each of the convolutional neural network hardware accelerators is connected to one node router in the lower two-dimensional network, and each of the convolutional neural network hardware accelerators includes a communication interface module, an input processing module, an output processing module, and the like. Includes data memory group, reconstructable compute module, memory control module, activation function module, weighted memory,
The convolutional neural network hardware accelerator is arranged so as to be divided into five modes at the initialization stage, and each mode has at least one convolutional neural network hardware accelerator.
In the first type mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n feature diagram memories (1 ≦ n ≦ N), and a result memory, and the target diagram. The data is written to the target diagram memory, the storage control module writes the first weighted data to the weighted memory, and a kind of activation function is set in the activation function module.
In the second mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n-1 feature diagram memory and a result memory, and the storage control module has a second weighting. Data is written to the weighted memory, the activation function is set in the activation function module, and the activation function is set.
In the third mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n-2 feature diagram memories and a result memory, and the storage control module has a third weighting. Data is written to the weighted memory, the activation function is set in the activation function module, and the activation function is set.
In the fourth mode, the storage control module of the convolutional neural network hardware accelerator divides the data memory group into a target diagram memory, n-2 feature diagram memories and a result memory, and the storage control module has a fourth weighting. Data is written to the weighted memory, the activation function is set in the activation function module, and the activation function is set.
In the fifth mode, the storage control module of the convolutional neural network hardware accelerator does not use the data memory group, the weighted memory, and the activation function module.
Individual convolutional neural network hardware accelerators are operated by the following method:
(1) Calculation of the first convolution layer:
When the current convolutional neural network hardware accelerator is in the first type mode, the storage control module reads the target diagram data from the target diagram memory, transfers the target diagram data to the output module and the reconstructable calculation module, and outputs the output. The processing module performs packaging processing of the target diagram data and transfers it to the communication interface module, and the reconstructable calculation module convolves and calculates the target diagram data and then transfers it to the activation function module to activate the activation function. The calculation is performed, the first feature diagram data calculated by the corresponding convolutional neural network hardware accelerator is acquired, and the storage control module stores the first feature diagram data in the first feature diagram memory.
When the current convolutional neural network hardware accelerator is in the second type mode, the input processing module receives the package data from the communication interface module in the first type mode, performs depackaging processing, and obtains the target diagram data. It is acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module convolves and calculates the target diagram data and then transfers it to the activation function module to perform the activation function calculation, and the convolutional neural network corresponds to the convolutional neural network. The first feature diagram data calculated by the hardware accelerator is obtained, and the storage control module stores the first feature diagram data in the first feature diagram memory.
(2) Calculation of the second convolution layer:
When the current convolutional neural network hardware accelerator is in first-class mode, the storage control module reads the first feature diagram data from the first feature diagram memory, and the reconstructable calculation module and output processing module. The reconstructable calculation module convolves and calculates the first feature diagram data, then transfers it to the activation function module to perform the activation function calculation, and the convolution neural network hardware accelerator calculates the first. The second feature diagram data is acquired and transferred to the storage control module, the storage control module stores the second feature diagram data in the second feature diagram memory, and the output module is the first feature diagram data. Is packaged and then transferred to the communication interface module.
When the current convolutional neural network hardware accelerator is in the second type mode, the storage control module reads the first feature diagram data from the first feature diagram memory and turns it into the reconstructable calculation module and output module. After transferring, the output processing module sequentially receives the package data in the first type mode and the second type mode, performs the depackaging process, and performs the first feature diagram data and the first feature diagram data in the first type mode. The first feature diagram data in the two modes is acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module convolves and calculates the first feature diagram data and then activates the activation. Transfer to the function module, perform activation function calculation, acquire the second feature diagram data calculated by the convolutional neural network hardware accelerator, transfer to the memory control module, and the memory control module has the second feature. The data is stored in the second feature diagram memory, and the output processing module packages the first feature diagram data and then transfers the data to the communication interface module.
When the current convolutional neural network hardware accelerator is in the third type mode, the input processing module sequentially receives the package data in the first type mode and the second type mode, performs the depackaging process, and performs the first type. The first feature diagram data of the type mode and the first feature diagram data of the second type mode are acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module is the first of the two modes. After each of the feature diagram data of the above is convolutionally calculated, it is transferred to the activation function module to perform the activation function calculation, and the second feature diagram data in the two modes is acquired and transferred to the storage control module. The storage control module stores the second feature diagram data in the two modes in the second feature diagram memory, and stores the second feature diagram data in the second feature diagram memory.
When the current convolutional neural network hardware accelerator is in the fourth type mode, the input processing module sequentially receives the package data in the first type mode and the second type mode, performs the depackaging process, and performs the first type. The first feature diagram data in the seed mode and the first feature diagram data in the second type mode are acquired and transferred to the reconstructable calculation module, and the reconstructable calculation module is in two modes. After each of the first feature diagram data of the above is convolutionally calculated, it is transferred to the activation function module to perform the activation function calculation, the second feature diagram data in the two modes is acquired, and the data is transferred to the storage control module. Then, the storage control module stores the second feature diagram data in the two modes in the second feature diagram memory.
(3) Calculation of the first all connection layers:
When the current convolutional neural network hardware accelerator is in the second type mode, the storage control module reads the second feature diagram data from the second feature diagram memory and transfers it to the reconstructable calculation module. , The reconstructable calculation module transfers the convolutional result of the convolutional calculation of the second feature diagram data to the output processing module, and the output processing module packages the convolutional result and then transfers it to the communication interface module. death,
When the current convolutional neural network hardware accelerator is in the third type mode, the storage control module reads the second feature diagram data from the second feature diagram memory and transfers it to the reconstructable calculation module. , The reconstructable calculation module transfers the convolutional result of the convolutional calculation of the second feature diagram data to the output processing module, and the output processing module packages the convolutional result and then transfers it to the communication interface module. ,
When the current convolutional neural network hardware accelerator is in the fourth type mode, the storage control module reads the second feature diagram data from the second feature diagram memory and transfers it to the reconstructable calculation module. The reconstructable calculation module transfers the convolutional result of the convolutional calculation of the second feature diagram data to the output processing module, and the output processing module packages the convolutional result and then transfers it to the communication interface module.
When the current convolutional neural network hardware accelerator is in the 5th mode, the input processing module sequentially receives the package data in the 2nd mode, the 3rd mode and the 4th mode. The packaging process is performed, the convolution results in the three modes are acquired, and the data is transferred to the reconstructable calculation module. The reconstructable calculation module acquires the cumulative calculated results of the convolution results in the three modes. Then, the data is transferred to the output processing module, and the output processing module processes the cumulative result and then transfers the cumulative result to the communication interface module.
Otherwise, when the current convolutional neural network hardware accelerator is in first-class mode, the storage control module can read the second feature diagram data from the second feature diagram memory and reconstruct it. Transferred to the calculation module, the reconstructable calculation module convolves the second feature diagram data and transfers the convolutional calculation result to the reconstructable calculation module, and the input processing module sequentially performs in the fifth type mode. Receives the package data of, performs depackaging processing, acquires the cumulative result in the fifth type mode, transfers it to the reconstructable calculation module, and the reconstructable calculation module combines the cumulative result and the convolutional result. At the same time, the third feature diagram data cumulatively calculated is acquired and transferred to the storage control module, and the storage control module stores the third feature diagram data in the third feature diagram memory.
(4) Calculation of the second whole connection layer:
When the current convolution neural network hardware accelerator is in the mode of the first kind, the storage control module reads the third feature diagram data from the third feature diagram memory and performs the activation function calculation, respectively. It is transferred to the activation function module, and after obtaining the activation result, it is transferred to the reconstructable calculation module and the output processing module, and the reconstructable calculation module performs a convolution calculation on the activation result, and the first The first rudimentary result is acquired and transferred to the activation function module, the activation function calculation is performed, the first output result is acquired and transferred to the storage control module, and the storage control module is the first. One output result is written to the result memory, the output processing module packages the activation result, and then transfers the activation result to the communication interface module.
When the current convolution neural network hardware accelerator is in the second type mode, the input processing module sequentially receives the package data in the first type mode, performs the depackaging process, and obtains the activation result. The reconstructable calculation module performs a convolution calculation on the activation result, obtains a second elementary result, transfers the result to the activation function module, and performs the activation calculation. The second output result is acquired and transferred to the output processing module, and the output processing module processes the second output result and then transfers it to the communication interface module.
When the current convolution neural network hardware accelerator is in the 4th type mode, the input processing module sequentially receives the package data in the 1st type mode, performs the depackaging process, and obtains the activation result. The reconstructable calculation module performs a convolution calculation on the activation result, obtains a third rudimentary result, transfers the result to the activation function module, and activates the result. The conversion function calculation is performed, the third output result is acquired and transferred to the output processing module, and the output processing module processes the third output result and then transfers it to the communication interface module.
When the current convolutional neural network hardware accelerator is in the first type mode, the input processing module further sequentially receives the package data in the second type mode and the fourth type mode and performs the depackaging process. , The second output result and the third output result are acquired and transferred to the storage control module, and the storage control module sequentially transfers the second output result and the third output result to the result memory. The three output results indicate the probability of belonging to the type corresponding to the target diagram data. It is an operation method of a convolutional neural network hardware accelerator based on a multicast network on-chip, and the multicast network on-chip is composed of an N × M lower layer two-dimensional network and an L × S upper layer network (of which M and N are). It is an integer of 2 or more, L and S are integers of 1 or more, and 1 ≦ L <N, 1 ≦ S <M).
The lower two-dimensional network has a total of N × M node routers and is divided into L × S blocks, and each block includes an intermediate node router and a plurality of ordinary node routers. The router includes five horizontal forwarding directions, the intermediate node router contains five horizontal forwarding directions and one uplink forwarding direction, the upper network is a forwarding node router, and the forwarding node router has five horizontal forwarding directions. Including direction and one downward transfer direction, the intermediate node router in each block and the corresponding transfer node router are interconnected, of which five horizontal transfer directions are front, back, left and right, four directions and the corresponding convolutional neural network hardware. It is one direction to transfer to the wear accelerator, and its features are as follows:
Each of the convolutional neural network hardware accelerators is connected to one node router in the lower two-dimensional network, and the process of the convolutional operation is as follows:
Step 1. Each of the convolutional neural network hardware accelerators is placed and divided into five modes at the initialization stage, and each mode has at least one convolutional neural network hardware accelerator.
Step 2, Convolutional Neural Network in First and Second Modes Perform the first convolutional calculation using the hardware accelerator.
Step 2.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator acquires the target diagram data, performs the convolution calculation, and then performs the activation function calculation. Obtained the first feature diagram data
Step 2.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator performs the activation function calculation after convolving the target diagram data. Acquire feature diagram data and
Step 3, perform the second convolutional layer calculation using the convolutional neural network hardware accelerator in the first, second, third and fourth modes.
Step 3.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator convolves and calculates the first feature diagram data, and further performs an activation function calculation. Acquired the second feature diagram data,
Step 3.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves the first feature diagram data and then performs the activation function calculation. Acquired the second feature diagram data,
Step 3.3, When the current convolutional neural network hardware accelerator is in the third type mode, the convolutional neural network hardware accelerator is in the first feature diagram data in the first type mode and the second type mode. Convolutional calculation of the first feature diagram data of each, and further activation function calculation to acquire the second feature diagram data in the two modes.
Step 3.4, When the current convolutional neural network hardware accelerator is in the 4th type mode, the convolutional neural network hardware accelerator is in the 1st feature diagram data and the 2nd type mode in the 1st type mode. After convolutional calculation of the first feature diagram data of, the activation function calculation is performed, and the second feature diagram data in the two modes is acquired.
Step 4, the first full connection layer calculation is performed using the convolutional neural network hardware accelerator in the first, second, third, fourth and fifth modes.
Step 4.1 When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.3, When the current convolutional neural network hardware accelerator is in the third type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.4, When the current convolutional neural network hardware accelerator is in the fourth type mode, the convolutional neural network hardware accelerator convolves and calculates the second feature diagram data, and then obtains the convolution result.
Step 4.5, If the current convolutional neural network hardware accelerator is in mode 5, the convolutional neural network hardware accelerator is in mode 2, mode 3 and mode 4. After accumulating the convolutional results of
Step 4.6, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator further integrates the integration results in the fifth type mode, and then the third feature. Acquire figure data,
Step 5, perform the second full connection layer calculation using the convolutional neural network hardware accelerator in the first, second and fourth modes.
Step 5.1, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator calculates the activation function in the third feature diagram data and then calculates the activation result. Obtain and then perform a convolutional calculation of the activation result to obtain the first rudimentary result, and finally calculate the activation function of the first rudimentary result to obtain the first activation result. And store
Step 5.2, When the current convolutional neural network hardware accelerator is in the second type mode, the convolutional neural network hardware accelerator convolves and calculates the activation result in the first type mode, and then the second type. Obtain the rudimentary result of, activate the second rudimentary result, perform the function calculation, and obtain the second output result.
Step 5.3, When the current convolutional neural network hardware accelerator is in the fourth type mode, the convolutional neural network hardware accelerator convolves and calculates the activation result in the first type mode, and then the third type. Obtain the rudimentary result of, calculate the activation function of the third rudimentary result, and obtain the third output result.
Step 5.4, When the current convolutional neural network hardware accelerator is in the first type mode, the convolutional neural network hardware accelerator is the second output result in the second type mode and the fourth type mode. And the third output result are stored, and each of the three output results shows the probability of belonging to the type corresponding to the target diagram data.

Images