WO2018107476A1

WO2018107476A1 - Memory access device, computing device and device applied to convolutional neural network computation

Info

Publication number: WO2018107476A1
Application number: PCT/CN2016/110436
Authority: WO
Inventors: 汪涛; 宋风龙; 刘武龙
Original assignee: 华为技术有限公司
Priority date: 2016-12-16
Filing date: 2016-12-16
Publication date: 2018-06-21
Also published as: CN110073329A; CN110073329B

Abstract

A memory access device, a computing device and a device applied to convolutional neural network computation, wherein same are used for improving the memory access efficiency and calculating the computation throughput, while reducing the calculation power consumption. The memory access device comprises: an input cache unit for caching data blocks to be calculated; and a cascading unit connected to the input cache unit, wherein the cascading unit reads the data blocks to be calculated from the input cache unit, and the data blocks to be calculated comprise a first data block and a second data block; the first data block and the second data block are connected end-to-end so as to obtain a cascaded data block; and a third data block is captured from the cascaded data block, the third data block containing a section of continuous data in the cascaded data block, and the length of the third data block being equal to the length of the data block in the input cache unit.

Description

Memory access devices, computing devices, and devices for convolutional neural network operations

Technical field

The present application relates to the field of computers, and in particular, to a memory access device, a computing device, and a device applied to a convolutional neural network operation in the computer field.

Background technique

Convolutional neural network (CNN) is the most widely used algorithm in deep learning. It is widely used in image classification, speech recognition, video understanding, face detection and many other applications. For the computational characteristics of neural network data intensiveness, convolutional neural network operations usually use a custom neural network processor. In recent years, neural network processors have become a research hotspot in academia and industry.

For the current neural network processor, in general, its research direction mainly includes both computation and storage. Among them, in terms of computation, the core of the convolution operation is the multiply-accumulate operation. Convolution operations usually contain a large amount of special data such as -1, ⁰ , 2 ⁿ , which takes up a large part of the computing resources. However, special data such as -1, ⁰ , 2 ⁿ is generated at runtime, and the compiler can only perform static optimization and cannot optimize the data in operation. Resulting in lower calculation rates and throughput.

In terms of storage, since the data of the convolution algorithm is locally strong, there are frequent address non-aligned memory accesses. In the case of address non-aligned access to the cache, the cache needs to access two consecutive access blocks at the same time, and after complicated operations such as address decoding, data strobing, and rotation shift, the power consumption is high and difficult. The required access data is generated in one clock cycle.

Summary of the invention

The present application provides a memory access device, a computing device, and a device applied to a convolutional neural network operation to improve memory access efficiency and computational throughput while reducing computational power consumption.

In a first aspect, a memory access device is provided, including: an input buffer unit for buffering a data block to be calculated; a cascading unit connected to the input buffer unit, the cascading unit for using the input The data block to be calculated is read in a buffer unit, where the data block to be calculated includes a first data block and a second data block; and the first data block and the second data block are connected end to end to obtain a level And a third data block is intercepted from the concatenated data block, the third data block includes a piece of continuous data in the concatenated data block, and a length of the third data block is Input slow The data blocks in the storage unit are of equal length.

The cascading unit may connect the first data block read from the input buffer unit and the second data block end to end to obtain a cascading data block. And a third data block of one data block length at any starting position is intercepted from the concatenated data. Therefore, a fast address non-aligned access can be realized by arbitrarily intercepting data in the cascading data block, thereby improving the efficiency of address non-aligned access.

In a possible implementation, the memory access device further includes: a control unit, the control unit is connected to the cascading unit, and configured to send a first control instruction to the cascading unit, the first The control instruction is configured to indicate a truncation manner of the concatenated data block; the concatenation unit intercepts the third data block from the concatenated data block according to the first control instruction.

In the embodiment of the present application, a vector length data can be quickly obtained from any two data blocks of the input buffer unit according to the first control instruction, that is, an arbitrary address non-aligned access is supported by one instruction, and the address can be reduced. Align access instructions to improve access efficiency.

In a possible implementation manner, the first control instruction includes first indication information, where the first indication information is used to indicate a starting position of the third data block in the concatenated data block.

In a possible implementation manner, the first indication information includes a data sequence number of a start position of the third data block, and the first control instruction further includes second indication information, where the second indication information is used by And indicating a data format of the data block to be calculated; the cascading device determines a starting position of the third data block in the cascading data block according to the data sequence number and the data format.

In a possible implementation manner, the input buffer unit includes a read port, and the read port is connected to a first control register, where the first control register stores first configuration information, where the first configuration information is used. And indicating a range of addresses of the data block to be read in the input buffer unit, a start address and a step size in the address range, the read port starts from the start address, and the step size is The address of the adjacent two read operations grows in steps, and the data blocks in the address range are cyclically read.

In a possible implementation manner, the input buffer unit includes a write port, the write port is connected to a second control register, the second control register stores second configuration information, and the second configuration information is used for Instructing an address range of the new data block in the input buffer unit, a start address and a step size in the address range, the write port starting from the start address, adjacent to the step size The address of the two write operations grows in steps, and a new block of data is cyclically written into the address range.

In the embodiment of the present application, the control register corresponding to the read port or the write port only needs to be stored for reading. The corresponding data can be accessed by the address range of the data block, the starting address and the step size of the address range. This makes it possible to streamline the instructions of the write port or the read port. Further, in the cyclic self-index access mode, the address range and the step size of the access data can be configured to improve the flexibility of accessing the data of the input buffer unit.

In a second aspect, a computing device is provided, the computing device comprising a multiplication buffer unit, a multiplication scheduling unit, and an addition unit, the multiplication buffer unit for buffering a multiply and accumulate instruction to be processed; the multiplication scheduling unit is configured to Obtaining, by the multiplication buffer unit, a first multiply-accumulate instruction, when a source operand of the multiplication operation in the first multiply-accumulate instruction includes an optimizable operand, determining an operation result of the multiplication operation by an optimization operation, and The operation result of the multiplication operation in the first multiply-accumulate instruction is directly sent to the addition unit, n is an integer greater than or equal to 0, the optimizable operand includes -1 or 2 ⁿ , and the optimization operation includes symbol extraction a reverse operation or a shift operation; the addition unit performs an addition operation in the first multiply-accumulate instruction according to an operation result of the multiplication operation in the first multiply-accumulate instruction, to obtain a correspondence corresponding to the first multiply-accumulate instruction The result of the operation of the multiply-accumulate operation.

In the embodiment of the present application, when the source operand of the multiplication operation in the first multiply-accumulate instruction includes -1 or 2 ⁿ , the calculation result of the multiplication operation is determined by a symbol negation operation or a shift operation, It is sent directly to the addition unit without multiplication by the multiplier, thereby increasing the rate and throughput of the multiply-accumulate operation and reducing the power consumption of the multiply-accumulate operation.

In a possible implementation manner, the multiplication scheduling unit is configured to schedule, in one clock cycle, a plurality of multiply and accumulate instructions acquired from the multiply buffer unit, the plurality of multiply accumulate instructions including a first type of multiply and accumulate And the at least one second type multiply-accumulate instruction, the source operand of the multiplication operation in the first type multiply-accumulate instruction does not include any one of -1, 0 and ²ⁿ , the second type multiply accumulate instruction The source operand of the multiplication operation in -1, 0 or 2 ⁿ .

The computing device in the embodiment of the present application can process multiple multiply and accumulate instructions in one clock cycle, thereby improving the speed and throughput of the multiply and accumulate operations.

In a possible implementation, the adding unit further includes an adding buffer unit, an adding scheduling unit, an adder, and at least one accumulating register, where the adding buffer unit is configured to buffer a source operand for the adding operation, The source operand includes an operation result of the multiplication operation in the multiply-accumulate instruction to be processed; the addition scheduling unit determines a first source operand and a second source operand of the addition of the first multiply-accumulate instruction, wherein And the first source operand corresponds to the same target accumulation register as the second source operand, the second source operand is from the addition buffer unit or the target accumulation register; First source operand and second source operand A summation is performed to obtain a summation result; the addition scheduling unit writes the summation result to the addition buffer unit or the target accumulation register.

In the embodiment of the present application, the addition scheduling unit uses the adder to sum the operations of the multiplication operation of the multiply-accumulate instruction corresponding to the same accumulation register in the addition buffer unit, thereby reducing the number of accesses to the accumulation register and reducing the generation of the access accumulation register. The pipeline stalls, increasing the rate and throughput of processing the multiply and accumulate operations.

In a possible implementation manner, when the addition buffer unit stores target data corresponding to the target accumulation register, the addition scheduling unit determines the target data as the second source operand, and Writing the summation result to the addition buffer unit; when the addition buffer unit does not store the target data, the addition scheduling unit uses the multiply and accumulate result stored in the target accumulation register as the second source Operands are written to the target accumulation register.

In the embodiment of the present application, the addition scheduling unit first uses the adder to first sum the operations of the multiplication and accumulation instructions corresponding to the same accumulation register in the addition buffer unit, thereby reducing the number of access accumulation registers and reducing the access accumulation register generation. The pipeline stalls, increasing the rate and throughput of processing the multiply and accumulate operations.

In a possible implementation, when the first multiply-accumulate instruction is the first multiply accumulative instruction in the first group of multiply-accumulate instructions, the multiply scheduling unit is configured to accumulate instructions for the first group of multiply A new target accumulation register is identified, and the operation result of the multiplication operation in the multiply-accumulate instruction in the first group multiply-accumulate instruction corresponds to the same accumulation register.

In a third aspect, there is provided a device for use in a convolutional neural network operation, comprising the memory access device of the first aspect or any of the possible implementations of the first aspect, and comprising the second or second aspect A computing device in any of the possible implementations.

DRAWINGS

FIG. 1 is a schematic diagram of a process of convolutional neural network operation according to an embodiment of the present application.

FIG. 2 is a schematic structural diagram of a memory access device according to an embodiment of the present application.

FIG. 3 is a schematic diagram of a process in which a cascading unit performs a cascading operation according to an embodiment of the present application.

4 is a schematic diagram of a concatenation unit applied to a convolution operation according to an embodiment of the present application.

FIG. 5 is a schematic structural diagram of an input buffer unit according to an embodiment of the present application.

FIG. 6 is a schematic diagram of a method for accessing an input buffer unit according to an embodiment of the present application.

FIG. 7 is a schematic diagram of an input buffer unit adjacent to two convolution operations write data according to an embodiment of the present application.

FIG. 8 is a schematic structural diagram of a computing device according to an embodiment of the present application.

FIG. 9 is a schematic structural diagram of a computing device according to still another embodiment of the present application.

FIG. 10 is a schematic flow chart of the multiply and accumulate operation in the embodiment of the present application.

FIG. 11 is a schematic flow chart of a multiply and accumulate operation according to another embodiment of the present application.

FIG. 12 is a schematic diagram of an apparatus applied to a convolutional neural network according to an embodiment of the present application.

FIG. 13 is a schematic structural diagram of an apparatus applied to a convolutional neural network according to still another embodiment of the present application.

FIG. 14 is a schematic diagram of the operation of the weight buffer in the embodiment of the present application.

FIG. 15 is a schematic diagram of a broadcast unit applied to a convolution operation in the embodiment of the present application.

16 is a structural diagram showing the relationship between a multiply-accumulate array, a cascading unit, and a broadcast unit in the embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described in the following with reference to the accompanying drawings in the embodiments.

In view of the above-mentioned problem that the neural network device access address non-aligned data operation is complicated and low in efficiency, the embodiment of the present application proposes a memory access device, and the central idea is that a fast address can be realized by setting an input buffer unit and a cascading unit. Unaligned block access.

The embodiment of the present application further provides a computing device, and the central idea is to introduce a multiplication scheduling unit in a computing device, which is used for performing fast multiply and accumulate operations on special data such as -1, ⁰ , 2 ⁿ generated during operation. It can increase the rate and throughput of the multiply and accumulate operations. Further, by introducing an addition scheduling unit in the computing device, the computing device simultaneously operates the multiple multiply and accumulate instructions, and processes the data correlation between the multiple multiply and accumulate instructions, where the data correlation refers to data between the instructions. There is a dependency, for example, the need for the B instruction to depend on the result of the operation of the A instruction. The multiply-accumulate operation is applied to a convolution algorithm, a two-dimensional filtering algorithm, or a finite impulse response (FIR) algorithm.

The embodiment of the present application further provides a device applied to a convolutional neural network operation, where the device includes the above-mentioned memory access device and computing device. The device can optimize the process of convolution operation from both computation and storage. In terms of calculation, it can improve the fast multiply-accumulate operation of special data such as -1, ⁰ , 2 ⁿ generated during operation by setting the computing device. The rate and throughput of the multiply-accumulate operation. In terms of storage, it sets the input buffer unit and the cascading unit for the convolution operation data overlap, that is, the data locality is strong, and realizes the function of data caching and fast address non-aligned access. This reduces the number of access caches and improves the efficiency of address unaligned access.

Optionally, each device in the embodiment of the present application may be applied to a convolutional neural network operation. For ease of understanding, the convolutional neural network and the equipment applied to the convolutional neural network operation are first introduced below. Convolutional neural network is a kind of artificial neural network, which has become a research hotspot in the field of speech analysis and image recognition. Its weight-sharing network structure makes it more similar to biological neural networks, reducing the complexity of the network model and reducing the number of weights. This advantage is more obvious when the input of the network is a multi-dimensional image, so that the image can be directly used as the input of the network, avoiding the complicated feature extraction and data reconstruction process in the traditional recognition algorithm. A convolutional network is a multi-layer perceptron specifically designed to recognize two-dimensional shapes that is highly invariant to translation, scaling, tilting, or other forms of deformation.

For ease of understanding, the process of convolution calculation is briefly introduced in conjunction with FIG. The convolution operation is actually a process of weighted summation. For example, each element in the used image region is multiplied by each element in the convolution kernel, and the sum of all products is used as the new value of the center pixel of the region. A convolution kernel is a matrix of fixed size and consisting of numerical parameters. The reference point of the matrix is usually at the center of the matrix, and the size of the matrix is the size of the kernel. As shown in FIG. 1, the convolution kernel matrix G performs a dot product operation on the same size data block in the input matrix R to obtain a calculation result of one of the output matrices O; then the convolution kernel is in the input matrix with the specified moving step size. Moving constantly, traversing all the data, and getting the output matrix O. Wherein, the convolution kernel size is 3*3, and the convolution kernel movement step is 1,

G5=R1*G1+R2*G2+R3*G3+R4*G4+R5*G5+R6*G6+R7*G7+R8*G8+R9*G9, if convolution operation is performed on an image, available A 3*3 convolution kernel with a reference point of the array as the core. The reference point of the kernel is first positioned at the first pixel of the image, and the remaining elements of the kernel cover its corresponding partial pixel in the image. For each kernel point, we can get the value of this point and the value of the corresponding image point in the image. Multiply and sum these values and place the result at the position corresponding to the input image reference point. This operation is repeated for each point of the image by scanning the convolution kernel over the entire image. Finally, a convolved image of the image can be obtained.

In the CNN network, the convolution calculation accounts for more than 90% of the entire operation, and is the main component of the entire CNN operation.

For the neural network processor currently applied to convolutional neural network operations, in general, its research direction mainly includes both computation and storage. Among them, in computing, it is mainly for application algorithms. Computationally intensive and containing a large amount of sparse data, design dedicated parallel computing paths (such as fast multiply-accumulate circuits) to increase the rate of convolution operations and computational throughput. In terms of storage, the application algorithm has strong locality and frequent address non-aligned memory access, and designs a dedicated storage path to reduce data transmission and increase data transmission bandwidth.

The memory access device of the embodiment of the present application is first introduced in conjunction with FIG. 2 . FIG. 2 is a schematic structural diagram of a memory access device 100 according to an embodiment of the present application. As shown in FIG. 2, the memory access device 100 includes:

The input buffer unit 110 is configured to cache a data block to be calculated.

For example, when applied to a convolutional neural network operation, an input buffer unit can be used to buffer input data for a convolution operation.

The cascading unit 120 is connected to the input buffer unit 110, and the cascading unit 120 reads the data block to be calculated from the input buffer unit 110, where the data block to be calculated includes the first data block. And a second data block; connecting the first data block and the second data block end to end to obtain a concatenated data block; intercepting a third data block from the concatenated data block, the third data block includes A piece of continuous data in the concatenated data block, and the length of the third data block is equal to the length of any one of the input buffer units 110.

Optionally, the first data block and the second data block may be data blocks belonging to different storage lines in the input buffer unit, or the first data block and the second data block may be the same storage line in the input buffer unit. Data block. The length of the first data block and the second data block may be the length of one vector data. That is, the cascading unit can quickly obtain a vector length data from the cascading data block according to any starting address. Or the cascading unit can support arbitrary address unaligned access according to an instruction.

The first data block and the second data block are connected end to end, and may refer to the first data block as a high bit and the second data block as a low bit, which are connected together to obtain a cascading data block.

In the embodiment of the present application, the cascading unit may connect the first data block and the second data block read from the input buffer unit end to end to obtain a cascading data block. And a third data block of one data block length at any starting position is intercepted from the concatenated data. Therefore, a fast address non-aligned access can be realized by arbitrarily intercepting data in the cascading data block, thereby improving the efficiency of address non-aligned access.

In the embodiment of the present application, when it is required to access different data in the same two data blocks of the input buffer unit multiple times, two data blocks may be first acquired, and then the two data blocks are cascaded to obtain a cascading data block. And directly intercepting the required third data block from the cascaded cascading data block, without multiple access to the input buffer unit, reducing the number of access to the data cache unit, thereby reducing the visit The power consumption of the data cache unit is reduced, and the time for address unaligned access is reduced, which improves the efficiency of address unaligned access.

For example, in the embodiment of the present application, data required for address non-aligned access can be generated in one clock cycle.

Optionally, the memory access device 100 further includes a control unit 130, and the control unit 130 is connected to the cascading unit 120, and configured to send a first control instruction to the cascading unit 120, where the first control instruction is used by And indicating the intercepting manner of the cascading data block; the cascading unit 120 intercepts the third data block from the cascading data block according to the first control instruction.

The control unit 130 may be configured to receive a decoding circuit signal, and generate corresponding control logic to control a unit in the memory access device according to the decoding circuit signal.

The first control instruction described above may be used to indicate the manner in which the cascading data block is intercepted. For example, the first control instruction may include first indication information, which may be used to indicate a starting position of the third data block in the concatenated data block. The cascading unit may intercept the third data block from the starting position according to the first indication information.

For another example, the first indication information may include a data sequence number of a start position of the third data block, and the first control instruction may further include second indication information, where the second indication information is used to determine the a data format of the data block to be calculated, the cascading device determining, according to the data sequence number of the start location and the data format of the data block to be calculated, the third data block in the cascading data block The starting position. Wherein, the above data format may indicate the width of each element in the data block. The above data sequence number is used to indicate the sequence number of the element in the data block.

As an example, the instruction can be expressed as "VRt=CAS.TYPE(VRm, VRn), Rs". Wherein, CAS is represented as an instruction opcode, and TYPE is a data format of the concatenation operation. For example, TYPE may be 8, 16, 32, 64, or bits (English: bits). Alternatively, TYPE can represent the width of an element in the vector. VRm and VRn respectively represent two vector registers before cascading. Rs represents the initial position of the block after the cascaded data block, and Rs can cooperate with the TYPE to determine the starting position and the intercept length of the intercepted data block.

FIG. 3 shows a process in which a cascade unit performs a cascade operation. The cascading unit can read data from the input buffer unit and read the data into a vector register. In the embodiment of the present application, the unit length of one data block may be equal to the length of the vector register. Data blocks can also be referred to as vector data. According to the instruction, the cascading unit needs to cascade the two vector registers VRm and VRn to obtain a vector data of twice the vector length. The vector data stored by the VRm and the VRn respectively correspond to the first data block and the second data block. Next, the cascading unit may determine to intercept the starting position and length of the third data block according to the first indication information. For example, when R _S = 4, it means that a third data block of a vector length is intercepted from the element of the 4th unit length of the concatenated data block. Specifically, assume that the vector length is 256 bits and the TYPE is 32 bits. Define the processor parallel granularity PS (Parallelism Size), indicating the number of homogeneous arithmetic units sharing local storage units. For example, it may be a single instruction multiple data (SIMD) or a number of arithmetic and logic unit (ALU) arithmetic units in a vector processor, or in a graphics processing unit (GPU). The number of streaming processors (SPs) in a streaming multiprocessor (SM). The size of the PS can be determined according to TYPE, which is expressed by the formula: PS=256/32=8. That is, a vector data includes 8 unit length elements. Or a vector register contains 8 unit lengths. Then the concatenated data block includes 16 unit lengths. When Rs=4, it means that the cascading unit intercepts the 16-unit vector generated by the concatenation from the address 4 to extract 8 units of elements to obtain a new vector, that is, obtains the third data block.

As an example, FIG. 4 shows a schematic diagram of a cascade unit applied to a convolution operation. For a 3*3 convolution kernel, when you need to continuously access the vector data of the cross-store row in the column direction, you only need to set Rs to 0, 1, and 2 respectively. Where Rs=0 is address-aligned access and Rs=1 or 2 is address-unaligned access.

The access device in the embodiment of the present application can be applied to a convolution operation. In particular, when multiple storage lines are accessed multiple times across multiple rows, the foregoing functions can be completed by setting multiple Rs by multiple cascade instructions. Access to the input cache unit.

Optionally, the input buffer unit can configure the read/write active area through a control register (Control Register, CR). The input buffer unit can include a read port and a write port. The write port is used to write data to the input buffer unit according to the instruction. The read port is used to read data from the input buffer unit according to the instruction. The above read port and write port may be one or more ports, respectively. Each The read port or write port can correspond to a control register for storing instructions. The read port or write port performs a read or write operation according to the configuration of the corresponding control register.

The input cache unit can support multiple read and write modes. For example, an input cache unit can support a loop self-index or an immediate index. Among them, the loop self-index can automatically maintain the pointer I through hardware to determine the location of accessing the input buffer unit. For example, the loop self-indexing can determine the specific address of the access input buffer unit based on the address range, the start address and the step size of the address range.

For example, taking a read port as an example, the input buffer unit includes a read port, and the read port is connected to a first control register, where the first control register stores first configuration information, and the first configuration information is used to indicate a data block address range to be read in the input buffer unit, a start address and a step size in the address range, the read port starts from the start address, and the step size is adjacent to the two The address of the secondary read operation grows in steps, and the data blocks in the address range are cyclically read.

The above starting address may also be referred to as a loop start address (for example, represented by start), and the address range may refer to a partial address range in the input buffer unit. The address range can also be referred to as the loop window length (for example, represented by Winlen). The above step size can refer to the address growth step size of the read port each time. For example, the above step size may also be referred to as a cyclic address growth step size (eg, expressed in steps).

For example, FIG. 5 is a schematic structural diagram of an input buffer unit according to an embodiment of the present application. As shown in FIG. 5, it is assumed that the input buffer unit includes two read ports, read port 0 and read port 1. The loop window is 6 cache lines in length. The loop self-index start address is 1, read port 0 needs to read data d0/d2/d4, read port 1 needs to read data d1/d3/d5, then read

port

0 and 1 are respectively configured as "start=1, WinLen= 6, Step = 2", "start = 2, WinLen = 6, Step = 2". Thus, in a continuous clock cycle, the address of the read port 0 accessing the input buffer unit is "IB[1]-IB[3]-IB[5]-IB[1]-IB[3]...", corresponding to the accessed data. It is "d0-d2-d4-d0-d2..."; the address of access port 1 access buffer unit is "IB[2]-IB[4]-IB[6]-IB[2]-IB[4]... ", the corresponding access data is "d1-d3-d5-d1-d3...".

FIG. 6 is a schematic diagram showing a method of accessing an input buffer unit in an embodiment of the present application. As shown in FIG. 6, the manner of accessing the input buffer unit can be expressed as Addr=Start+I%WinLen, where I=I+step, and % means the remainder. Addr indicates the specific address of the read port access input buffer, Start indicates the start address, I indicates the internal pointer, Winlen indicates the length of the loop window, and step indicates the step size. The loop window can be any contiguous range of addresses within the input buffer unit. The read port cyclically reads the data in the loop window according to the step size.

For another example, taking the write port as an example, the input buffer unit includes a write port, the write port is connected to a second control register, the second control register stores second configuration information, and the second configuration information is used for Instructing, in the input buffer unit, an address range in which a new data block is stored, a start address in the address range, and a step size, the write port is from the start address, and the step size is adjacent to two The address of the secondary write operation is incremented by a step size, and a new data block is cyclically written into the address range. The specific manner of writing data to the write port is similar to the manner in which the read port reads data, and details are not described herein again.

As an example, the instruction format of the cyclic self-indexed read operation can be expressed as: MOV IB[I++], Dest, which means that data is read from the internal self-indexing address of the input buffer unit to the destination register (which can be called Dest) and updated. The self-indexing pointer I=I+step, where MOV represents data handling, and the data direction is data buffer unit→register. IB represents the input buffer unit.

The instruction format of the cyclic self-index write operation can be expressed as LD Addr, IB[I++], which means that the data is loaded from the Addr address of the memory, written to the internal self-index address of the input buffer unit, and the self-indexed pointer is updated at the same time I=I+Step Where LD represents data handling, data direction is memory → input buffer unit, and IB represents input buffer unit.

As an example, the instruction format of an immediate index read operation may be represented as MOV IB[imm], Dest, indicating that data is read from IB[imm] to dest.

The instruction format of the immediate index write operation can be expressed as LD Addr, IB[imm], indicating that data is loaded from the memory Addr address and written to IB[imm].

In the embodiment of the present application, the input buffer unit supports a cyclic self-index access mode. In the access mode, the control register corresponding to the read port or the write port only needs to store the address range of the data block to be read, and in the address range. The starting address and step size, you can access the corresponding data. This makes it possible to streamline the instructions of the write port or the read port. Further, in the cyclic self-index access mode, the address range and the step size of the access data can be configured to improve the flexibility of accessing the data of the input buffer unit.

Optionally, as shown in FIG. 7, the upper and lower parts in FIG. 7 respectively show schematic diagrams of the convolution operation write data of the input buffer unit in the row direction. As can be seen from Fig. 7, when the convolution kernel moves in the direction of the input line data row, the adjacent two convolution operation input data has an overlapping portion. The write operation to the input buffer unit can be divided into two phases of initialization and update. For the k*k convolution kernel, 2*k vectors are loaded during the initialization phase; due to the overlapping nature of the data, only two vectors need to be loaded during the update phase, and the previous 2k-2 vectors are reused, thereby reducing the input. Buffer access The number of system memory. The system memory can be, for example, a Dynamic Random Access Memory (DRAM).

For example, in Fig. 7, at the time of the first convolution operation, the input data is d0 to d5. In the second convolution operation, as the convolution kernel moves in the row direction, the input data is d2 to d7. The input data d3 to d5 of the two adjacent convolution operations are overlapped. Therefore, in the second convolution operation, only data d6 and d7 need to be written, and data d0 and d1 are overwritten.

The access device of the embodiment of the present application is described above with reference to FIG. 1 to FIG. 7. The computing device of the embodiment of the present application will be described below with reference to FIG. 8 to FIG.

FIG. 8 shows a schematic structural diagram of a computing device 300 of an embodiment of the present application. Computing device 300 can be used for multiply and accumulate operations. For example, computing device 300 can be a multiply-accumulate array or a device that includes a multiply-accumulate array. As shown in FIG. 8, computing device 300 includes a multiplication buffer unit 310, a multiplication scheduling unit 320, and an addition unit 330,

The multiplication buffer unit 310 is configured to buffer a multiply and accumulate instruction to be processed.

Alternatively, the multiplication buffer unit 310 may also be referred to as a multiplication buffer unit (English: Mul_Buffer). The multiply buffer unit can buffer the multiply and accumulate instructions prepared by the operand after decoding. Optionally, each entry in the multiply cache unit may include 3 fields. The three fields are "instruction type (English: opcode)", "source operand 0 value (English: src0)", "source operand 1 value (src1)". The depth of the multiplication buffer unit can be set according to the width of the instruction transmission.

The multiplication scheduling unit 320 is configured to obtain a first multiply and accumulate instruction from the multiplication buffer unit 310, and determine, when the source operand of the multiplication operation in the first multiply accumulate instruction includes an optimizable operand, Determining an operation result of the multiplication operation, and directly transmitting the operation result of the multiplication operation in the first multiply-accumulate instruction to the addition unit, where n is an integer greater than or equal to 0, and the optimizable operand includes -1, 0 Or 2 ⁿ , the optimization operation includes a symbol inversion operation, a shift operation, or a cancel operation.

The adding unit 330 performs an addition operation in the first multiply-accumulate instruction according to an operation result of the multiplication operation in the first multiply-accumulate instruction, and obtains an operation result of the multiply-accumulate operation corresponding to the first multiply-accumulate instruction .

The multiplication scheduling unit 320 may also be referred to as a multiplication scheduler (English: Mul_Scheduler), and the multiplication scheduling unit 320 may multiply -1, 0, 2 in the multiply-accumulate instruction according to the instruction type and the source operand in the multiplication buffer unit 310. Special data generated during operation such as ⁿ (n≥0) is scheduled. For example, in a specific scheduling process, for a case where the source operand contains 0, the multiplication result can be canceled. For the case where the source operand contains -1, the multiplication result can be obtained by modifying the sign bit inversion. In the case where the source operand contains 2 ⁿ , the result of the multiplication operation can be obtained by the shift operation, and the operation result of the above multiplication operation is directly sent to the addition unit 330 so that the addition unit performs the addition operation.

Optionally, the multiplication scheduling unit is configured to schedule, in one clock cycle, a plurality of multiply and accumulate instructions acquired from the multiplication buffer unit, the plurality of multiply and accumulate instructions including a first type multiply and accumulate instruction and at least one a two-type multiply-accumulate instruction, the source operand of the multiplication operation in the first type multiply-accumulate instruction does not include any one of -1, 0, and 2 ⁿ , and the multiplication operation in the second type multiply-accumulate instruction Source operands include -1, 0 or 2 ⁿ .

Optionally, the computing device may further include a multiplier connected to the multiplication scheduling unit 320 and the adding unit 330. When the source operand of the multiplication operation in the first multiply-accumulate instruction does not include -1, 0, or ²ⁿ , the multiply scheduling unit 320 transmits the source operand to the multiplier, and the multiplier obtains the multiplication operation according to the source operand. The result of the operation is sent to the addition unit 330.

It should be understood that the multiplier can process a first type of multiply-accumulate instruction in one clock cycle, and the multiply scheduling unit can process a plurality of second type multiply-accumulate instructions without using a multiplier in one clock cycle.

Optionally, when the source operand of the multiply-accumulate scheduling instruction includes -1, 0 or 2 ⁿ (n ≥ 0), the multiply scheduling unit 320 performs the above scheduling processing on -1, 0 or 2 ⁿ (n ≥ 0) At the same time, the next instruction can be read from the multiplication buffer unit 310 and the above scheduling can be continued. The source operand until the multiply accumulate instruction does not contain -1, 0 or 2 ⁿ (n ≥ 0), or until the multiply accumulate instruction to be processed is not included in the multiplication buffer unit 310. When the source operand of the multiply accumulate instruction does not include -1, 0 or 2 ⁿ (n ≥ 0), the multiply scheduling unit 320 may send the multiply accumulate instruction to the multiplier, after the multiplier performs the multiplication operation, The result of the obtained multiplication operation is sent to the addition unit. Therefore, in the embodiment of the present application, the computing device 300 can process multiple multiply and accumulate instructions simultaneously, which improves the rate and throughput of the multiply and accumulate operations.

Optionally, in one clock cycle, the multiplication scheduling unit may sequentially acquire the multiply and accumulate instructions from the multiplication buffer unit, and schedule the multiply and accumulate instructions according to the scheduling method. For example, in one clock cycle, after obtaining the first type multiply and accumulate instruction, the multiply scheduling unit may send the first type multiply and accumulate instruction to the multiplier, and the next instruction obtained by the multiply scheduling unit is the second type of multiply accumulate When the instruction is executed, the second type multiply-accumulate instruction may be shifted, inverted, or canceled, and then the operation result is directly sent to the addition unit. If the next instruction acquired by the multiplication scheduling unit is still the first type multiply-accumulate instruction, since the multiplier is already processing the first type multiply-accumulate instruction, the multiply scheduling unit may stop acquiring the multiply and accumulate instruction from the multiplication buffer unit. The multiply and accumulate instructions are processed until the next clock cycle.

Optionally, the adding unit 330 further includes an adding buffer unit, an adding scheduling unit, an adder, and at least one accumulating register. The addition buffer unit is configured to buffer a source operand for an add operation, the source operand including an operation result of the multiplication operation in the multiply-accumulate instruction to be processed; the addition scheduling unit determines the first multiplication a first source operand and a second source operand of the addition operation of the accumulation instruction, wherein the first source operand corresponds to the same target accumulation register as the second source operand, and the second source operand is from The addition buffer unit or the target accumulation register; the addition scheduling unit uses the adder to sum the first source operand and the second source operand to obtain a summation result; the addition scheduling unit The summation result is written to the addition buffer unit or the target accumulation register.

Optionally, the multiply scheduling unit may allocate a corresponding accumulating register tag (English; tag) to the multiply and accumulate instructions. A set of multiply-accumulate instructions corresponds to the same accumulator register. The result of the multiplication operation in the group multiply-accumulate instruction needs to be summed, and the summation result is written into the accumulation register corresponding to the group multiply-accumulate instruction.

Optionally, the first source operand may be the bottommost data of the addition buffer unit (ie, the earliest data entering the addition buffer unit). The choice of the second source operand includes two ways. In the first mode, if the addition buffer unit further stores the target data of the same accumulation register corresponding to the first source operand, the target data may be used as the second source operand of the addition operation, and the operation of the addition operation may be performed. The result is written to the addition buffer unit. In the second mode, if the addition buffer unit does not include the data of the same accumulation register corresponding to the first source operand, the data stored in the accumulation register corresponding to the first source operand is used as the second source operand of the addition operation, and Write the result of the addition operation to the accumulation register.

Optionally, in the first mode, when the operation result of the addition operation is written back to the addition buffer unit When the operation result is used as the source operand of the addition operation, the corresponding accumulation register is determined, that is, the operation result of the addition operation and the first source operand, The second source operand corresponds to the same accumulation register. Further, the second source operand may be an operation result of the multiplication operation in the multiply-accumulate instruction, or may be a summation result obtained by summing the operation results of the multiplication operation in the multiply-accumulate instruction.

In the second mode, the data stored in the accumulation register may be a multiply-accumulate operation result of the multiply-accumulate instruction in the same group multiply-accumulate instruction as the first multiply-accumulate instruction.

In the embodiment of the present application, since the addition buffer unit can simultaneously buffer the result of the multiplication operation of the plurality of multiply-accumulate instructions, and the addition scheduling unit first uses the adder to first multiply the multiply-accumulate instruction corresponding to the same accumulation register in the addition buffer unit. The summation of the operations can reduce the number of accesses to the accumulator register, reduce the pipeline stall caused by accessing the accumulator register, and improve the rate and throughput of the processing multiply and accumulate operations.

Optionally, when the first multiply-accumulate instruction is the first multiply-accumulate instruction in the first group of multiply-accumulate instructions, the multiply scheduling unit is configured to identify a new target accumulating register for the first multiply-accumulate instruction The operation result of the multiplication operation in the multiply-accumulate instruction in the first group multiply-accumulate instruction corresponds to the same accumulation register. Wherein, the foregoing identifying a new target accumulation register may be to allocate a new accumulated register label for the first group of multiply-accumulate instructions.

As an example, the above-mentioned addition buffer unit may also be referred to as an addition buffer (English: Arithmetic Logic Unit Buffer, ALU_Buffer). The addition buffer unit can be used to cache the multiplication result of the multiply-accumulate instruction. The data of the addition buffer unit may be derived from a multiplication scheduling unit or a multiplier. The depth of the addition buffer unit can be determined according to the width of the instruction transmission.

As an example, the above-described addition scheduling unit may be referred to as an addition scheduler (English: ALU_scheduler) to schedule the addition of the multiply-accumulate instruction. The addition scheduling unit schedules multiple multiply-accumulate instructions to avoid pipeline stalls caused by data correlation between multiple consecutive multiply-accumulate instructions.

As an example, the at least one accumulation register may be a plurality of accumulation registers, and the plurality of multiply-accumulate registers may ensure that multiple sets of multiply-accumulate instructions are simultaneously run in the computing device. The number of accumulated registers can be set according to the instruction transmit width.

Optionally, the addition buffer unit caches an operation result of the multiplication operation of the multiply-accumulate instruction to be processed. The multiply-accumulate instruction to be processed may include a plurality of multiply-accumulate instructions. The plurality of multiply-accumulate instructions may include at least one set of multiply-accumulate instructions. Wherein, each of the at least one set of multiply and accumulate instructions is multiplied by The result of the multiplication operation is used for summation, and each set of multiply accumulate instructions corresponds to one of the at least one accumulator register. The summation result of each set of multiply accumulate instructions is used to write to the corresponding accumulator register.

Optionally, when the data stored in the accumulation register acts as the second source operand of the addition operation into the addition pipeline, the accumulation register no longer enters the addition operation as the second source operand before the accumulation register completes the writeback operation. Pipeline to ensure that no data correlation occurs between multiple multiply and accumulate instructions within the same group.

It should be understood that in the prior art, when the multiply-accumulate operation device has data correlation in the multiply accumulate instruction, a pipeline stall is required. In addition, in the summation unit in the prior art, each time a multiply-accumulate operation of processing a target multiply-accumulate instruction is performed, the multiply-accumulate operation result in the corresponding accumulating register needs to be read into the adder, and the target is The result of the multiplication operation of the multiply-accumulate instruction is summed, and the obtained summation result is stored in the multiply-accumulate register as the operation result of the updated multiply-accumulate operation. The multiply-accumulate register cannot again act as a second source operand into the pipeline of the adder until the write-back operation is completed, resulting in a lower rate and throughput of the multiply-accumulate operation.

In the embodiment of the present application, since the addition buffer unit can simultaneously buffer the result of the multiplication operation of the plurality of multiply-accumulate instructions, and the addition scheduling unit first uses the adder to first multiply the multiply-accumulate instruction corresponding to the same accumulation register in the addition buffer unit. The operation summation sums all the multiply-accumulate instructions corresponding to the same accumulating register in the addition buffer unit, and after obtaining the summation result, sums the multiply-accumulated result in the accumulating register, and writes the result back Accumulating registers, thereby reducing the number of accesses to the accumulator registers, increases the rate and throughput of processing multiply accumulate operations.

As a specific example, FIG. 9 shows a computing device 500 of an embodiment of the present application. Computing device 500 can also be referred to as a multiply accumulator. As shown in FIG. 9, computing device 500 is based on a multiplier (represented by Mul) and an adder (represented by ALU), and may also include the following elements:

Multiplication buffer unit (represented by Mul_Buffer): Can be the multiplication buffer unit in Figure 8. The multiply and accumulate instructions are prepared in the multiply buffer unit, and each entry of the multiply buffer unit includes {"instruction type opcode", "source operand 0 value src0", "source operand 1 value src1" ”}3 fields, the depth of the multiplication buffer unit can be set according to the width of the command transmission.

Multiplication scheduler (represented by Mul_Scheduler): Can be the multiplication scheduling unit in Figure 8. According to the instruction type and source operand data in Mul_Buffer, special values such as multiplication operation -1/0/2n (n>=0) in the multiply and accumulate instruction are scheduled, and the instruction of the scheduling completion can be sent to the addition after the Mul and Mul pipelines. Cache unit (represented by ALU_Buffer).

Addition cache unit (represented by ALU_Buffer): The multiplication result in the cache multiply accumulate instruction in ALU_Buffer, the data of ALU_Buffer can come from Mul_Scheduler and Mul. Similar to Mul_Buffer, the depth of ALU_Buffer can be set according to the width of the command transmission.

Addition scheduler (represented by ALU_Scheduler): ALU_Scheduler schedules the addition of the multiply-accumulate instruction to avoid pipeline stalls caused by data correlation between multiple consecutive multiply-accumulate instructions.

Accumulating register banks (represented by ACC): Multiple multiply accumulate registers ensure that multiple sets of multiply accumulate instructions are simultaneously run in the multiply accumulator. Among them, a set of multiply and accumulate instructions is defined as an instruction sequence containing one MACC instruction. For the MACC instruction, please refer to the following detailed description. The number of accumulating registers can be set according to the instruction transmission width.

By adding the above hardware logic, the computing device 500 can support two multiply-accumulate instructions, which are:

1) Normal multiply accumulate instruction, defined as "MAC X, Y". Its function is to multiply the input data X and Y, and add the multiplication result to the current accumulating register, and finally write the addition result back to the multiply accumulate register, ie ACC+=X*Y.

2) Set the multiply-accumulate instruction with the initial value of the multiply-accumulate register to be zero, defined as "MACC X, Y". Its function is to multiply the input data X and Y, and write the multiplication result back to the accumulation register (equivalent to setting the initial value of the multiply-accumulate register to 0 first, then multiply and accumulate), that is, ACC=X*Y.

The operation of computing device 500 can be divided into three steps:

1) Decoding and the multiply and accumulate instructions after the completion of the read operand enters Mul_Buffer;

2) Mul_Scheduler schedules the multiplication operation;

3) ALU_Scheduler schedules the addition operation.

FIG. 10 and FIG. 11 are schematic flowcharts showing the multiply and accumulate operations of the embodiment of the present application, respectively. FIG. 10 is a flowchart of the multiplication operation in the multiply and accumulate operation in the embodiment of the present application. Fig. 11 is a flow chart showing the addition operation of the multiply-accumulate operation of the embodiment of the present application. The specific flow of performing the multiply accumulate instruction by the computing device 500 will be described in detail below with reference to FIGS. 9 through 11.

a. Decode and multiply and accumulate instructions after the completion of the read operand into Mul_Buffer

As shown in FIG. 10, the multiply-accumulate instruction is decoded, and after reading the operand, the instruction buffer enters the Mul_Scheduler. When the instruction enters Mul_Scheduler, as long as the instruction buffer has a multiply and accumulate instruction, and these multiply accumulate instructions are ready for two source operands except the accumulating register, these The instructions are sent to Mul_Buffer. The condition that Mul_Bufffer stops accepting the multiply accumulate instruction is: 1) Mul_Buffer is full; 2) there is no multiply accumulate instruction currently decoded.

Optionally, when entering the Mul_Scheduler, the instruction may consider the data correlation between the multiply-accumulated instruction and other instructions, and perform corresponding scheduling.

After this step, the Mul_Buffer buffers a plurality of multiply and accumulate instructions that are ready for operation and decoded.

b. Mul_Scheduler schedules the multiplication operation.

As shown in Figure 10, Mul_Scheduler schedules the multiplication of the multiply and accumulate instructions in Mul_Buffer. It is used to judge the source operand of the multiply-accumulate instruction, and determines the scheduling mode according to the judgment result. It mainly includes two cases: the first case is that the source operand of the multiply accumulate instruction contains -1/0/2 ⁿ (n>=0); the second case is that the source operand of the multiply accumulate instruction does not contain - 1/0/2 ⁿ (n>=0). The scheduling of these two cases is as follows.

1. If the source operand of the multiply accumulate instruction contains -1/0/2 ⁿ (n>=0), the instruction is processed as follows, and the instruction is removed from the Mul_Buffer.

1) If the source operand contains 0, cancel the instruction directly.

2) If the source operand contains -1//2 ⁿ (n>=0), the multiplication result is directly obtained by modifying the sign bit or the shift operation, and the multiplication result is sent to the ALU_Buffer. At the same time, the accumulating register tag (English: Tag) to be written to, such as Tag(ACC0), indicates that the multiply accumulate instruction finally needs to write the result to the accumulating register ACC0.

2. If the instruction source operand does not contain -1/0/2 ⁿ (n>=0), the instruction is normally sent to Mul, marking the accumulation register Tag to be written to. Continue to fetch instructions from Mul_Buffer. If the source operand contains -1/0/2 ⁿ (n>=0), repeat step 1 until the multiply source operand of the multiply accumulate instruction does not contain -1/0/2 ⁿ (n>=0).

The normal MAC/MACC instruction entering the Mul pipeline is delayed by the pipeline of the multiplier, and finally the multiplication result is written to the ALU_Buffer, and the accumulation register Tag to be written is marked. For the MACC instruction, set the accumulator register value of the corresponding tag to 0.

After this step, the result of the multiplication in the multiply-accumulate instruction is buffered in the ALU_Buffer, and these multiplication results need to be added to the accumulating register.

c. ALU_Scheduler schedules the addition operation

As shown in FIG. 11, the ALU_Scheduler schedules the addition operation. The specific scheduling method is as follows:

The data of the bottom of ALU_Buffer (the earliest ALU_Buffer) is used as the first source operand of ALU addition, and the second source operand of ALU addition is selected as follows:

1. If the ALU_Buffer contains data with the same accumulated register tag as the first source operand, the data is used as the second source operand of the ALU, and the ALU addition result is written back to the ALU_Buffer, and the accumulating register Tag is reserved.

2. If the ALU_Buffer does not contain data with the same accumulated register tag as the first source operand, then the accumulating register with the same tag as the first source operand is used as the second source operand of the ALU, and the ALU addition result is written back. Corresponding accumulation register.

When the accumulator register acts as the ALU second source operand into the ALU pipeline, since the data correlation is generated, the accumulator register cannot enter the second source operand again and enters the ALU pipeline before the accumulating register completes the write back operation, which ensures that the ALU pipeline is entered. No data correlation occurs between multiple multiply and accumulate instructions within the same group.

In the embodiment of the present application, the computing device 500 may perform optimization processing on the numerical values of -1, ⁰ , 2 ^n, etc. in the multiplication operation, and in the operation in the addition operation, first, the addition buffer label has the same accumulated register label. The data is summed and then summed with the multiply-accumulated result in the accumulator register, thereby reducing the number of accesses to the accumulator register, thereby reducing pipeline stalls and improving the efficiency and throughput of the processing multiply-accumulate operation.

Optionally, the memory access device and the computing device of the embodiment of the present application are described above with reference to FIG. 1 to FIG. 11 . Hereinafter, the device applied to the convolutional neural network operation according to the embodiment of the present application will be described with reference to FIG. 12 to FIG. 15 . .

FIG. 12 is a schematic diagram of an apparatus 700 applied to a convolutional neural network in accordance with an embodiment of the present application. As shown in FIG. 12, the device 700 applied to the convolutional neural network operation includes the memory access device 710 and the computing device 720 in the embodiment of the present application. The access device 710 may be any of the memory devices in the embodiment of the present application. The computing device 720 may be any computing device in the embodiment of the present application.

In the embodiment of the present application, the device applied to the convolutional neural network operation includes a memory access unit, and the memory access unit may be. Therefore, fast address non-aligned access can be realized by intercepting data in the cascading data block, thereby improving the efficiency of address non-aligned access. And the computing device included in the convolutional neural network computing device determines the operation of the multiplication operation by a symbol inversion operation or a shift operation when the source operand of the multiplication operation in the first multiply-accumulate instruction includes -1 or 2 ⁿ As a result, it is directly sent to the addition unit without multiplication by the multiplier, thereby increasing the rate and throughput of the multiply-accumulate operation and reducing the power consumption of the multiply-accumulate operation.

As a specific embodiment, FIG. 13 is a schematic structural diagram of a device 800 according to an embodiment of the present application. Device 800 can be applied to convolutional neural network operations. The memory access device 710 in FIG. 7 may include an input buffer area 830 and a cascading unit 850 in FIG. Further, the memory access device 710 may further include a control unit 810, a weight buffer 840, and a broadcast unit 860. Computing device 720 in FIG. 7 can include fast multiply-accumulate array 870 in FIG. Specifically, as shown in FIG. 13, the device 800 includes:

Control Unit (CU) 810: Receives the decoding circuit signal and generates corresponding control logic to control the entire system.

Memory 820: stores input data, weight data, and final convolution results. The memory 820 can be system memory. For example, the memory 820 can be a DRAM.

Input Buffer (IB) 830: connected to the control unit 810, the memory 820, and the cascading unit 850, and buffers the input data of the convolution operation according to the parallel calculation direction of the convolution, and can support the loop self-index or immediate Number indexing two access methods. The input buffer may be the input buffer unit described in FIGS. 2 to 7. For example, the input buffer can be a cache.

Weight Buffer (WB) 840: connected to the control unit 810, the memory 820, and the broadcast unit 820, and buffers the weight data of the convolution operation according to the parallel calculation direction of the convolution. For example, the weight buffer can be a cache.

Cascading Unit (CaU) 850: Cascading two vectors and truncating the appropriate bit segments from the two vectors to generate new vector data for convolution (multiply-accumulate) operations to reduce The number of accesses to the input buffer when accessing the same cross-store row. Wherein, the cascading unit may be the cascading unit described in FIGS. 2 to 7.

Broadcasting Unit (BU) 860: Broadcasts a single weight data of a convolution kernel to generate vector data.

A multiply-accumulate array (MAC Array, MACA) 870: multiply and accumulate input data and convolution weights using an algorithm and a control scheduling method. The multiply-accumulate array 870 can be the computing device described above. For example, the multiply-accumulate array can be the computing device 500 or computing device 700 described above.

Partial-Sum Buffer (PB,) 880: multiply accumulate result generated by the cache fast multiply-accumulate array 870, and output the data in the portion and the buffer 880 to the fast according to the decoding control signal generated by the control unit 810. The accumulation array is used for accumulating operations with new multiplication results; or output to memory 820 as a final convolution result.

Alternatively, the input buffer 830 can be used to read input data for each volume operation. Lose For a schematic diagram of the structure of the buffer 830, reference may be made to the structure of the input buffer unit shown in FIG. 5. As shown in FIG. 5, it is assumed that the convolution kernel size is 3*3, and the processor parallelism is PS=4. The shaded portion in Fig. 5 indicates all the related data in a column direction convolution parallel operation (four convolution operations are performed simultaneously in the column direction), wherein the position of the first convolutional convolution kernel indicated in the dotted line frame . It can be seen that a convolution operation is related to 6 vectors, and is represented as d0 to d5 in the column direction, and these 6 vectors are respectively stored in the input buffer unit (that is, equivalent to the input buffer area 830).

FIG. 14 is a schematic diagram showing the operation of the weight buffer in the embodiment of the present application. As shown in FIG. 14, optionally, the weight buffer buffers the convolution kernel weight according to the parallel direction of the convolution operation. If they are parallel in the row direction, they are stored in a row sequence; if they are parallel in the row direction, they are stored in a column sequence. The weight buffer is set to 1 write port and 1 read port, and the depth of the buffer can be flexibly set.

Optionally, the cascading unit 850 can splicing the two vector registers end to end and intercepting consecutive vector values of one unit in the spliced 2x length vector register. For a schematic diagram of the concatenation unit 850 applied to the convolution operation, reference may be made to the description of the related content in FIG. 4, and details are not described herein again.

Figure 15 shows a schematic diagram of a broadcast unit applied to a convolution operation. As shown in Figure 15, the broadcast unit broadcasts a single element in the vector register in the form of a vector, and for convolution operations, broadcasts each weight element in the convolution kernel into a vector form. The instruction format can be “VRt=BRO.TYPE VRm, Rs”, where BRO is represented as an opcode and TYPE is a data format of a concatenation operation. For example, TYPE can be 8, 16, 32, 64, bits (English: bits ). Alternatively, TYPE can represent the width of an element in the vector. VRm represents a vector register. In Fig. 15, as an example, the broadcast unit broadcasts element 4 in the vector register to form vector data.

Alternatively, the multiply-accumulate array 870 can include PS multiply accumulators. PS represents processor parallel granularity. FIG. 16 is a diagram showing the relationship of the multiply-accumulate array 870 with the cascading unit and the broadcasting unit. As shown in FIG. 16, the multiply-accumulate array 870 receives the input data generated by the cascade unit 850, and the convolution kernel weight data generated by the broadcast unit 860 performs a multiply-accumulate operation. It optimizes the special values such as -1/0/2 ⁿ in the multiplication operation. Since the convolution operation usually contains a large number of -1/0/2 ⁿ special values, the speed of the multiply-accumulate operation can be improved. At the same time, it can handle the data correlation by hardware itself, and read out the value of the accumulation register through a dedicated instruction. For the specific structure of the multiply accumulators included in the multiply-accumulate array 870, reference may be made to the related content of the computing device in FIG. 7 to FIG. 11 , and details are not described herein again.

The device 800 provided by the embodiment of the present application can improve the convolution operation speed and throughput, and the input buffer can buffer the reused input data and reduce the number of accesses to the slow memory; the cascading unit can generate the vector data across the storage line. To avoid frequent access to the input buffer; the multiply accumulator performs fast multiplication of special values such as -1, ⁰ , 2 ^{n in the} multiplier, and can automatically process data correlation.

Additionally, the terms "system" and "network" are used interchangeably herein. The term "and/or" in this context is merely an association describing the associated object, indicating that there may be three relationships, for example, A and / or B, which may indicate that A exists separately, and both A and B exist, respectively. B these three situations. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

It should be understood that in the embodiment of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B from A does not mean that B is only determined based on A, and that B can also be determined based on A and/or other information.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.

The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, may be located in one place, or It can also be distributed to multiple network elements. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present application.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, can be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be in essence or part of the contribution to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. There are a number of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method of various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The technical features and descriptions of the above embodiments are used in other embodiments for the sake of brevity and clarity of the application, and will not be further described in other embodiments.

The foregoing is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any equivalents can be easily conceived by those skilled in the art within the technical scope disclosed in the present application. Modifications or substitutions are intended to be included within the scope of the present application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

A memory access device, comprising:

An input buffer unit for buffering data blocks to be calculated;

a cascading unit, configured to be connected to the input buffer unit, the cascading unit is configured to read the data block to be calculated from the input buffer unit, where the data block to be calculated includes a first data block and a a second data block; the first data block and the second data block are connected end to end to obtain a concatenated data block; a third data block is intercepted from the concatenated data block, the third data block includes the A contiguous piece of data in the cascading data block, and the length of the third data block is equal to the length of the data block in the input buffer unit.
The memory access device of claim 1 , wherein the memory access device further comprises:

a control unit, the control unit is connected to the cascading unit, and configured to send a first control instruction to the cascading unit, where the first control instruction is used to indicate an interception manner of the cascading data block;

The cascading unit intercepts the third data block from the concatenated data block according to the first control instruction.
The memory access device of claim 2, wherein the first control instruction comprises first indication information, the first indication information is used to indicate that the third data block is in the cascading data block The starting position.
The memory access device of claim 3, wherein the first indication information includes a data sequence number of a start position of the third data block, and the first control instruction further includes second indication information, The second indication information is used to indicate a data format of the data block to be calculated;

The cascading device determines a starting position of the third data block in the cascading data block according to the data sequence number and the data format.
The memory access device according to any one of claims 1 to 4, wherein the input buffer unit comprises a read port, the read port is connected to a first control register, and the first control register stores a a configuration information, the first configuration information is used to indicate an address range of a data block to be read in the input buffer unit, a start address and a step size in the address range, and the read port is from the The start address begins with the step size being the address growth step of two adjacent read operations, and the data blocks in the address range are cyclically read.
The memory access device according to any one of claims 1 to 4, wherein the input buffer unit comprises a write port, the write port is connected to a second control register, and the second control register stores a Two configuration information, where the second configuration information is used to indicate the input buffer unit Storing an address range of a new data block, a start address and a step size in the address range, the write port starting from the start address, and increasing the address of the adjacent write operation by the step size The step size, the new data block is cyclically written into the address range.
A computing device, comprising: a multiplication buffer unit, a multiplication scheduling unit, and an addition unit,

The multiplication buffer unit is configured to buffer a multiply and accumulate instruction to be processed;

The multiplication scheduling unit is configured to obtain a first multiply accumulating instruction from the multiplication buffer unit, and determine the multiplication by an optimization operation when a source operand of the multiplication operation in the first multiply accumulative instruction includes an optimizable operand And an operation result of the operation, and directly transmitting the operation result of the multiplication operation in the first multiply and accumulate instruction to the addition unit, where n is an integer greater than or equal to 0, and the optimizable operand includes -1 or 2 n , The optimization operation includes a symbol inversion operation or a shift operation;

The adding unit performs an addition operation in the first multiply-accumulate instruction according to an operation result of the multiplication operation in the first multiply-accumulate instruction, and obtains an operation result of the multiply-accumulate operation corresponding to the first multiply-accumulate instruction.
The computing device of claim 7 wherein said multiply scheduling unit is operative to schedule a plurality of multiply-accumulate instructions fetched from said multiply buffer unit in one clock cycle, said plurality of multiply accumulate instructions comprising one a first type multiply-accumulate instruction and at least one second type multiply-accumulate instruction, the source operand of the multiplication operation in the first type multiply-accumulate instruction not including any one of -1, 0 and 2n , The source operand of a multiplication operation in a two-type multiply-accumulate instruction includes -1, 0 or 2 n .
A computing device according to claim 7 or 8, wherein said adding unit further comprises an addition buffer unit, an addition scheduling unit, an adder and at least one accumulation register,

The addition buffer unit is configured to buffer a source operand for an add operation, where the source operand includes an operation result of the multiplication operation in the multiply-accumulate instruction to be processed;

The addition scheduling unit determines a first source operand and a second source operand of the addition of the first multiply accumulate instruction, wherein the first source operand corresponds to the second source operand An accumulation register, the second source operand being from the addition buffer unit or the target accumulation register;

The addition scheduling unit sums the first source operand and the second source operand to obtain a summation result;

The addition scheduling unit writes the summation result to the addition buffer unit or the target Accumulate registers.
The computing device according to claim 9, wherein said addition scheduling unit determines said target data as said second when said addition buffer unit stores target data corresponding to said target accumulation register a source operand, and writing the summation result to the addition buffer unit; when the addition buffer unit does not store the target data, the addition scheduling unit uses the multiply and accumulate result stored in the target accumulation register as The second source operand and writing the summation result to the target accumulation register.
The computing device of claim 9 or 10, wherein when said first multiply-accumulate instruction is the first multiply accumulate instruction of the first set of multiply-accumulate instructions, said multiply scheduling unit is The first set of multiply-accumulate instructions identifies a new target accumulator register, and the result of the multiplication of the multiply accumulate instructions in the first set of multiply-accumulate instructions corresponds to the same accumulator register.
An apparatus for use in a convolutional neural network operation, comprising the memory access device of any one of claims 1 to 6, and the computing device of any one of claims 7 to 11. .