CN115860082A - Neural form data processing system based on SPI bus - Google Patents

Abstract

The invention relates to the technical field of integrated circuits and neural networks, in particular to a neural morphology data processing system based on an SPI bus. The invention provides an SPI (serial peripheral interface) bus-based neuromorphic data processing system, which realizes cooperative work of a plurality of internal architectures through an SPI (serial peripheral interface) protocol with adjustable data bit width, adopts compact design, stores neuron parameters outside a chip, effectively reduces the overhead and power consumption of on-chip storage, designs abundant instruction packets and high-flexible and configurable neurons for the architectures, can easily complete the configuration of an on-chip network, effectively reduces the overhead of a large number of registers in the on-chip network by adopting a register multiplexing technology, and ensures the reliability of data by using a router to transmit information between cores.

Description

Neural form data processing system based on SPI bus

Technical Field

The invention relates to the technical field of integrated circuits and neural networks, in particular to a neural morphology data processing system based on an SPI bus.

Background

The human brain has a surprising ability to recognize complex patterns, and these calculations are performed by hundreds of millions of neurons and synapses in the human brain. The neuromorphic calculation is just to reproduce the calculation function of the brain by simulating the connection mode of neurons and synapses of the human brain, and a corresponding theory, algorithm and system architecture are constructed according to the plasticity of the neurons and synapses.

The pulse neural network model is composed of pulse neurons and synapses, each neuron is connected with other neurons, and information transmission is carried out through pulses with extremely short duration after the neurons reach an excitation threshold. Unlike typical multilayer perceptrons, impulse neural networks have spatiotemporal characteristics and sparsity, with neurons triggering over multiple operating cycles. On a biomimetic level, there are also various neuron models, such as the common Leaky Integrated-And-Fire (LIF) And Hodgkin-Hexley (Hodgkin-Huxley) types.

Neural networks are widely applied to pattern recognition, automatic driving, data processing and the like, and as the scale of the network is increased and the recognition task is more complex, the hardware implementation is also more difficult. The pulse neural network encodes input data into a form of 0 or 1 in a pulse encoding mode, and the encoded pulse data conforms to the digital circuit design, so that a highly connected neuron array can be realized, and hardware resource overhead is reduced by times. Compared with software simulation, the hardware integrated pulse neural network can realize the accelerated processing of the network by utilizing the parallel computing capability of a digital circuit, and a plurality of different networks can be simultaneously calculated by adopting a hardware architecture with a plurality of neural cores.

The SPI protocol is a high-speed full-duplex four-wire communication bus, has the characteristics of good transmission reliability, good expansibility and the like, and is widely applied to chips such as FLASH, SRAM and the like. In order to meet different equipment requirements, similar buses such as Dual SPI and Qual SPI are provided, data transmission bandwidth is improved based on the traditional SPI, multiple modes can be selected, and the application range of the SPI protocol is wider. In the technical field of neural networks, the weight data volume in input data is huge, the requirement on the reliability of activation data is high, and the output data volume is generally small. In order to solve the above data transmission problem, a new data transmission protocol based on the conventional SPI is required.

Disclosure of Invention

Because the communication mode between the cores of the existing neuromorphic data processing system is complex, and a set of complete instructions and a universal protocol are not available for controlling the work of the whole processing system. Therefore, the invention provides a neural morphology data processing system based on an SPI bus, which is used for solving the problems of low router use efficiency, poor configuration of a neuron register and parameters, high power consumption and the like of the current data processing system.

In order to achieve the purpose, the technical scheme of the invention is as follows:

an SPI bus based neuromorphic data processing system, the system comprising: the system comprises an SPI host, an SPI controller, a top controller, a storage controller (controlling an off-chip SRAM), a buffer (on-chip memory) and a core array; the SPI controller is used for bridging the SPI host and the top layer controller, communication between the SPI host and the top layer controller is established in a mode that the data bit width is adjustable, transmission of data and instructions is completed through an MOSI (host output and slave input) channel with 8-bit data bit width and an MISO (host input and slave output) channel with 1-bit data bit width, wherein the 8-bit data bit width of the MOSI channel is flexibly adjustable, and the specific data width can be set by the SPI host; the storage controller is respectively connected with the top layer controller and the off-chip SRAM and is used for generating an SRAM storage protocol and reading and storing data from the off-chip SRAM; the core array and the top controller perform data interaction, the data processing mode of the core array is based on a pulse neural network algorithm, and each core in the array comprises five processing modules of a scheduler, a buffer, a neuron, a router and a neuron controller; the scheduler is used for storing activation data, the type of the activation data is pulse data, the pulse data comprises two parts, the first part is original input data, is coded into pulse data and acts on a first layer of operation core; the second part of data is from pulse data output by the router after the previous layer of operation core is calculated, and acts on other operation cores, wherein the other operation cores are a hidden layer and an output layer in the neural network; the buffer is used for storing a configuration file read from an external SRAM, and the configuration file comprises parameters such as weight data, leakage voltage, a router, a reset mode, reset voltage, threshold voltage, initial membrane voltage and leakage inversion flag; the neuron is used for executing the calculation of the pulse neural network, input data required by the calculation come from a scheduler and a buffer, an execution command comes from a core controller, and a final calculation result is transmitted to the router; the router is used for sending and receiving pulse data, and the sent and received data come from cores in five directions, namely east, south, west, north and local; the neuron controller is used for controlling the conversion of neurons, the conversion of activation data, recording the activation times of the neurons and controlling all modules under each core to work cooperatively;

the data processing mode of the neuromorphic data processing system is as follows: the SPI host sends an instruction packet to a top controller module through an SPI controller, the top controller module receives and analyzes the instruction packet, the instruction packet comprises a control packet and a data packet, the control packet comprises a global initialization signal, a set core parameter, a configuration file, a write activation data command, a start calculation signal and a read data signal, and the data packet comprises core set data, pulse data and single neuron data; the top layer controller module controls data flow and working state of the core array through instructions of the control package, the data flow comprises configuration files, activation data, neuron parameters, time steps and calculation results, and the working state comprises initialization, calculation starting, axon switching, neuron switching and calculation stopping; the top controller module transmits core setting data and pulse data in a data packet to the core array, transmits single neuron data in the data packet to the storage controller, obtains a required configuration file data storage address from two variables of the neuron and the core position, reads data from an external memory according to the address and transmits the data to the core array.

The invention has the beneficial effects that the SPI bus-based data processing system is provided, the cooperative work of a plurality of internal frameworks is realized through the SPI protocol with adjustable data bit width, the compact design is adopted, the neuron parameters are stored outside the chip, the overhead and the power consumption of the on-chip storage are effectively reduced, in addition, rich instruction packets and the high-flexibility and configurable neurons are designed for the frameworks, the configuration of the on-chip network can be easily completed, the overhead of a large number of registers in the on-chip network is effectively reduced by adopting the register multiplexing technology, and the information is transmitted by using the router between the cores, so that the reliability of the data is ensured.

Drawings

FIG. 1 is a schematic diagram of the overall architecture of a neuromorphic data processing system according to the present invention;

FIG. 2 is a schematic diagram of an SPI protocol with adjustable bit width according to the present invention;

FIG. 3 is a schematic diagram of the internal structure of a single core proposed by the present invention;

FIG. 4 is a schematic diagram of a neuron register model proposed by the present invention;

FIG. 5 is a diagram of the steps performed by a prior art spiking neural network;

FIG. 6 is a block diagram of an architecture control command packet according to the present invention;

FIG. 7 is a diagram illustrating a command packet transmission format according to the present invention;

fig. 8 is a schematic structural diagram of a router module proposed by the present invention;

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The main mode of the invention is to adopt an extensible SPI protocol to control the parameter configuration and data transmission of the neuromorphic data processing system, store the neuron configuration parameters in an external SRAM, store activation data in a scheduler, and adopt routers to communicate among different cores.

The neuromorphic data processing system is generally used for executing a convolutional neural network or an impulse neural network, and compared with the impulse neural network, the neuromorphic data processing system has lower power consumption and is more suitable for practical application requirements. In the verification aspect, an FPGA or an ASIC is generally used for verification. The overall architecture of a neuromorphic data processing system provided by the invention is shown in fig. 1, and the implementation method of the data processing system comprises the following modules: the SPI controller module receives and sends data and packages the data; the memory controller module is used for generating an SRAM memory protocol, reading and storing data; a core array module for executing brain-like computation, and being configurable and reconfigurable; and the top controller module is used for connecting the core array, the SPI controller and the storage controller to work cooperatively. The whole system can be applied to execute the impulse neural network, each core in the core array can be used as an input layer, a hidden layer and an output layer in the impulse neural network, and 16 layers of networks can be executed maximally. In an embodiment, 3 kernels are adopted to complete the handwritten digit picture recognition of the 3-layer impulse neural network, wherein the image size is 16x16=256 pixels (obtained by 28 × 28 pixel compression), and the image content includes 0 to 9 handwritten digits.

The SPI controller in the neuromorphic data processing system is used for generating and analyzing an SPI protocol, the interior of the SPI controller is connected with the top layer controller, and the external SPI host is connected with the SPI controller and used for data interaction of the two parts. As shown in fig. 2, the SPI protocol only needs 8 independent signals to complete data transmission from the master to the slave, which are respectively the system clock, the chip select bit, the SPI clock (data enable signal), the data bit SPI _ MOSI (master and slave), the data bit SPI _ MISO (master and slave), and the read valid signal. Before data transmission, the SPI clock frequency is ensured to be less than or equal to 1/4 of the system clock, and the SPI clock can be obtained by system clock frequency division or other clock signals. And 8-bit data channels can be selected in the MOSI data transmission channel, so that the transmission requirements of different upper computers are met. Different data processing modes can be selected in the neuromorphic data processing system according to the instruction of the SPI host, and input data can be converted into 8-bit data to be transmitted to the top layer controller in any data processing mode. Fig. 2 lists the data transmission cases of 8bit and 1bit, the data transmission from the master and the data reception from the slave are separated by only one system clock cycle, the instruction represented by 0x02d in the figure is the core setting, the instruction represented by 0x01 is the core selection (see fig. 7), and 0x is the 16-bit notation method. Under the condition that the SPI clock is 1/4 system clock frequency, 4 system clock cycles are consumed for transmitting one byte of data of the 8-bit channel, and 32 system clock cycles are consumed for transmitting one byte of data of the 1-bit channel. In the MISO data transmission channel, since only a small part of the neural network operation process parameters and the final calculation result need to be output, a 1-bit output data channel is adopted in the embodiment. In order to ensure the reliability of data, a data effective signal is introduced, and the signal is kept at a high level all the time in the process of transmitting data from the slave computer to the host computer, and is kept at a low level in other situations. The data sent by the slave and the data received by the master are separated by only two system clock cycles. Whether the MOSI channel or the MISO channel, a set of data may be transmitted only when the three conditions of the chip select bit being low, the system clock rising edge, and the SPI clock falling edge are simultaneously satisfied.

The SPI controller transmits the received data to the top controller for further parsing, and the top controller executes different instructions according to the received control packet (see fig. 6). In the initial parameter configuration stage, the top layer controller writes the parsed register configuration file (see fig. 4) into an external SRAM, the memory space occupied by a single core is 21x32x256 bits, and the activation data (poisson-encoded image data) is written into the scheduler under the selected first layer core. In the calculation stage of the neural network, the top layer controller circularly reads the configuration parameters of the neuron register from the external SRAM through the storage controller, single circulation reads single neurons under all activated cores, and the sequence number of each neuron is kept consistent (such as the neuron 5 of the core 0-0 and the neuron 5 of the core 0-1; 8230; and the like), and the top layer controller transfers the neuron data of each core to a buffer to prepare for next calculation. The whole data processing system is also controlled by the top layer controller to analyze external instructions to start computation, end computation and axon switching, convert neurons and output results.

As shown in FIG. 3, a single core under the core array is composed of five parts, namely a neuron controller, a buffer, a scheduler, a neuron and a router. The core array is arranged in a 4X4 matrix structure (see fig. 1), and the serial number of each core corresponds to the X-axis and Y-axis coordinates. All cores are under the same time step in the operation process, the time step can reach 32 times at most, and the time step is a special concept in the impulse neural network and is also called as iteration times.

The scheduler is used for storing activation data (pulse data), the data of the first layer network is different from the data of other networks, the activation data of the first layer (input layer) network is the pulse data of an input image transmitted by the top layer controller, the pulse data is encoded into pulses under 32 time steps by a picture with 256 pixels, and each time step comprises 256 pieces of 1-bit data. The data of the hidden layer and the output layer are pulses sent by routers of the network of the previous layer. The scheduler uses two 32x128bit internal SRAM memory cells, where 32 corresponds to 32 time steps and 2x128bit corresponds to 256bit pulses. Different from the common SRAM, the SRAM can be written in simultaneously in a single-bit or multi-bit mode, when the upper layer network does not release pulses, the SRAM can not carry out any operation, and the access of the memory is effectively reduced. During the calculation of the data processing system, the scheduler sends a 256-bit pulse to the neuron every time the scheduler switches a time step.

The buffer is used for temporarily storing the register parameters of the neurons, and the total buffer space is 664 bits. And initiating a reading signal by the top layer controller at the same time of the start of the nth calculation, reading the configuration parameters required for the (n + 1) th time from the external SRAM, and storing the configuration parameters in the buffer. Each buffer is provided with an internal storage unit of 256x20 bits, 20 bits are membrane voltages calculated by the neuron module at each time step, and 256 represent 256 neurons under each core. The membrane voltage parameters at the first time step are obtained from the external SRAM (included in the neuron register parameters), and the membrane voltage parameters at the other 31 time steps are obtained from the internal memory cells in the register by reading. Before the calculation of a single neuron starts, the buffer transmits the parameter file of 664 bits to the neuron module under the action of a neuron controller.

Each core has 256 neurons, and 256 synapse units are under each neuron for implementing a spiking neural network. The single neuron in the neuron module is configured with 15 registers, which are respectively membrane voltage, axon type, synapse weight, leakage reversal flag bit, positive threshold, negative threshold, reset voltage, reset mode, router parameter and leakage weight, the total occupation size of the registers is 664 bits (refer to fig. 4), the registers under the single neuron are multiplexed to 256 neurons by adopting a register multiplexing technology, and other cores also adopt the same processing method. The 16 cores originally required 61440 registers, and only 240 registers were actually used through register multiplexing. In the calculation process of the neuron module, two clock cycles are required to complete the calculation of an axon once, the first clock cycle is used for acquiring weight data, and the second clock cycle is used for completing summation calculation, and the specific calculation process is shown in fig. 5, which is a process that a buffer, a scheduler, a neuron controller and a router module work cooperatively and execute a pulse neural network in the neuron module, and the process comprises the following steps:

step 1, after all parameter configuration is finished, initiating a calculation starting signal by an SPI host, and transmitting the signal to a neuron controller through a top layer controller

And 2, the neuron controller initiates an activation data reading request, and reads 256-bit input image pulse data at a time step from the scheduler.

And 3, the neuron controller initiates an initialization neuron module, and at the moment, the buffer sends 664bit register parameters required by the first neuron to the neuron calculation module.

And 4, resolving 256x 2bit weight indexes, 4x 9bit weights and other parameters from the register parameters by the neuron module, wherein the index of 2 bits of one axon just comprises 4 weights, determining the effective of the 256 axons by the image pulse of the 256 bits (the image pulse is invalid and 1 effective), and finally adding the weights under all the effective pulses to obtain a calculation result.

And 5, comparing the calculation result with the threshold voltage to determine whether to send the pulse, if so, transmitting pulse data by the local router according to the target core position, and resetting and storing the membrane voltage into the buffer. If no pulse is issued, the membrane voltage at the current time step is stored by the buffer (calculation result). And (3) after the calculation of one neuron is finished, starting the conversion neuron by the neuron controller, and skipping to the step 3 until the calculation of 256 neurons is finished, namely finishing the calculation of one time step.

Step 6, in this embodiment, when the 10 th neuron at the 32 th time step is calculated, the whole calculation process is finished, one image recognition is completed, the core parameters are reset, and the scheduler is waited to update and start the next recognition.

As shown in fig. 6, the diagram is a dedicated command packet of the neuromorphic data processing system, which can be divided into a control packet and a data packet. Wherein the control packet includes: global initialization, core setting, configuration file writing, activation data writing, starting of a start signal and starting of reading data, wherein each process corresponds to a state machine in the top layer controller module. The data packet includes: configuration data, pulse data and core data required for a single neuron. When the control command is 0x01 (initialization), 0x05 (start signal), 0x06 (read data), only a single command is transmitted; when the control command is 0x02 (core setting), the data packet can be sent only after the sub-command needs to be sent once again, and the types of the sub-command include 0x01 (core selection), 0x02 (activation position) and 0x03 (reading position); when the control command is 0x03 (write configuration file) or 0x04 (write active data), a packet is transmitted immediately after the command is transmitted (see fig. 7).

As shown in fig. 8, which is a router architecture, each router module has 5 FIFOs (First Input First Output) for storing packets from east, south, west, north and local cores. Where each packet contains 4 registers, respectively the horizontal location (X), vertical location (Y), axon type and delivery time of the target core. In the transmission direction, after receiving a pulse issued by the neuron module, the router firstly transmits along the horizontal (X) direction according to the destination coordinate, and transmits along the vertical (Y) direction after the transmission along the horizontal direction is finished, if the router at east or north of the local core sends a data packet, then the router at west or south of the next core receives the data packet, and after a plurality of transmissions, the local routing FIFO sends the data to the scheduler for storage. For a core located at the boundary of 4 edges, routers with the direction of propagation towards the outside of the boundary are not enabled.

The neuron controller is used for controlling the start and the end of the calculation of the neurons, two signals are sent by the SPI host, the conversion of the neurons and the conversion of the activation data are controlled each time, and the updating operation of the buffer is completed. The neuron controller is also used for storing the number of activation pulses of each neuron under 32 time steps, in the embodiment, 10 neuron activation pulses of an output layer are transmitted to a top layer controller module and transmitted to an SPI (serial peripheral interface) host through an SPI (serial peripheral interface) protocol, 10 neurons correspond to 0-9 handwritten numbers, the SPI host judges the maximum value of the 10 neuron activation pulses, the serial number corresponding to the maximum value is the result of a picture number, and the whole nerve form data processing system completes one-time identification.

Claims (1)

1. An SPI bus based neuromorphic data processing system, the system comprising: the system comprises an SPI host, an SPI controller, a top controller, a storage controller, a buffer and a core array; the SPI controller is used for bridging the SPI host and the top layer controller, communication between the SPI host and the top layer controller is established in a mode that the data bit width is adjustable, transmission of data and instructions is completed through an MOSI channel with 8-bit data bit width and a MISO channel with 1-bit data bit width, wherein the 8-bit data bit width of the MOSI channel is flexibly adjustable, and the specific data width is set by the SPI host; the storage controller is respectively connected with the top layer controller and the off-chip SRAM and is used for generating an SRAM storage protocol and reading and storing data from the off-chip SRAM; the core array and the top controller perform data interaction, the data processing mode of the core array is based on a pulse neural network algorithm, and each core in the array comprises five processing modules of a scheduler, a buffer, a neuron, a router and a neuron controller; the scheduler is used for storing activation data, the type of the activation data is pulse data, the pulse data comprises two parts, the first part is original input data, is coded into pulse data and acts on a first layer of operation core; the second part of data is from pulse data output by the router after the previous layer of operation core is calculated, and acts on other operation cores, wherein the other operation cores are a hidden layer and an output layer in the neural network; the buffer is used for storing a configuration file read from an external SRAM, and parameters of the configuration file comprise weight data, leakage voltage, a router, a reset mode, reset voltage, threshold voltage, initial membrane voltage and leakage reversal marks; the neuron is used for executing the calculation of the pulse neural network, input data required by the calculation come from a scheduler and a buffer, an execution command comes from a core controller, and a final calculation result is transmitted to the router; the router is used for sending and receiving pulse data, and the sent and received data come from cores in five directions, namely east, south, west, north and local; the neuron controller is used for controlling the conversion of neurons, activating data conversion, recording the number of times of neuron activation and controlling all modules under each core to work cooperatively;

the data processing mode of the neuromorphic data processing system is as follows: the SPI host sends an instruction packet to a top controller module through an SPI controller, the top controller module receives and analyzes the instruction packet, the instruction packet comprises a control packet and a data packet, the control packet comprises a global initialization signal, a set core parameter, a configuration file, a write activation data command, a start calculation signal and a read data signal, and the data packet comprises core set data, pulse data and single neuron data; the top layer controller module controls data flow and working state of the core array through instructions of the control package, the data flow comprises configuration files, activation data, neuron parameters, time steps and calculation results, and the working state comprises initialization, calculation starting, axon switching, neuron switching and calculation stopping; the top controller module transmits core setting data and pulse data in a data packet to the core array, transmits single neuron data in the data packet to the storage controller, obtains a required configuration file data storage address from two variables of the neuron and the core position, reads data from an external memory according to the address and transmits the data to the core array.

Images