CN114330669B - A vector processor-oriented half-precision vectorized conv1×1 convolution method and system - Google Patents

Abstract

The invention discloses a vector processor-oriented half-precision vectorized conv1×1 convolution method and system. The method includes: storing half-precision weight data and half-precision input data in a double-rate synchronous dynamic random access memory; calling The direct memory access operation loads half-precision weight data and half-precision input data from the double-rate synchronous dynamic random access memory into the on-chip scalar memory SM space and the on-chip array memory AM space respectively; in the SM space, the pair is loaded into the on-chip SM space. Perform vectorization processing on the weight data of , in AM space, perform convolution operation conv1×1 between the vectorized weight data and the input data in AM space to obtain the feature map data after convolution. The invention can combine the architectural features of the vector processor to vectorize the convolution calculation (conv1×1) to the vector processor architecture, and realize the improvement of FLOPs on the premise of ensuring the accuracy.

Description

A vector processor-oriented half-precision vectorized conv1×1 convolution method and system

technical field

The invention relates to the technical field of vector processors, in particular to a vector processor-oriented half-precision vectorized conv1×1 convolution method and system.

Background technique

The architecture of the vector processor is a new type of architecture, as shown in Figure 1, which includes a scalar processing unit (SPU) that performs scalar operations, a vector processing unit (VPU) that performs vector operations, and direct memory responsible for data transfer. Access (Direct Memory Access, DMA) components and so on. The SPU consists of a scalar processing element SPE and a scalar memory SM. The VPU consists of L vector processing components VPE and array memory AM. The L vector processing components VPE operate cooperatively in a single-instruction, multiple-data (SIMD) manner. One VPE integrates 3 vector computing components for simultaneous support of vector Fixed-point and floating-point operations.

A single VPE can process 1 piece of 8-byte data (such as FP64, Int64), 2 pieces of 4-byte data (such as FP32, Int32), or 4 pieces of 2-byte data (such as FP16) at a time. The DMA part is responsible for data transfer between SM and DDR (Double Rate Synchronous Dynamic Random Access Memory), AM and DDR, and the minimum granularity of its operation is also 8 bytes.

Convolution (Convolution) is one of the core calculations of neural networks, of which conv1×1 is the most common specification in convolution operations, so its efficiency has a great impact on the performance of neural networks, and optimizing convolution calculations appears to be especially important.

SUMMARY OF THE INVENTION

In view of this, the present invention provides a vector processor-oriented half-precision vectorized conv1 × 1 convolution method, which combines the architectural features of the vector processor to orient the convolution calculation (conv1 × 1) to the vector processor architecture. Vectorization achieves the improvement of FLOPs under the premise of ensuring accuracy.

The present invention provides a vector processor-oriented half-precision vectorized conv1×1 convolution method, including:

Store half-precision weight data and half-precision input data in double-rate synchronous dynamic random access memory;

Invoke a direct memory access operation to load the half-precision weight data and the half-precision input data from the double-rate synchronous dynamic random access memory into the on-chip scalar memory SM space and the on-chip array memory AM space, respectively;

In the SM space, vectorize the weight data loaded into the on-chip SM space, and in the AM space, perform the convolution operation conv1×1 between the vectorized weight data and the input data in the AM space to obtain The feature map data after convolution;

Wherein, the data format of the half-precision weight data Weight _ddr is [Co, Cin, ks, ks], Co is the number of output channels, Cin is the number of input channels, ks is the size of the convolution kernel, when the size of the convolution kernel is 1, the data format can be regarded as [Co, Cin], so the weight data can be expressed as a matrix Weight _ddr = M × K, the data format of the half-precision input data Input _ddr is [Cin, Hi, Wi, n], Hi and Wi are the height and width of the image respectively, n is the number of batch processing in the convolution operation, [Hi, Wi, n] can be regarded as one dimension, let N=Hi×Wi×n, so The input data can be represented as a matrix Input _ddr =K×N, where M represents Co, K represents Cin, and N represents the size of the image dimension.

Preferably, the direct memory access operation is invoked to load the half-precision weight data and the half-precision input data from the double-rate synchronous dynamic random access memory into the on-chip scalar memory SM space and the on-chip array memory AM space, respectively, including :

其中m的大小由SM的空间大小和AM空间的大小综合决定；Call the direct memory access operation, load the half-precision weight matrix W _ddr into the on-chip SM space, divide the original data from M dimension into x ₁ Wb _sm matrix, become W _sm = x ₁ ×Wb _sm , Wb _sm = m×K,

The size of m is determined comprehensively by the size of the SM space and the size of the AM space;

p表示向量处理器的体系结构中向量功能运算单元部件的数量，L表示向量处理部件的数量。Invoke the direct memory access operation, load the half-precision input matrix I _ddr into the on-chip AM space, divide the original data from N dimensions into x ₂ Ib _am matrices, become I _am =x ₂ ×Ib _am , where Ib _am = K×n, that is, N=x ₂ ×n, where n=P×L×4,

p represents the number of vector functional operation unit components in the architecture of the vector processor, and L represents the number of vector processing components.

Preferably, in the SM space, vectorization processing is performed on the weight data loaded into the on-chip SM space, and in the AM space, a convolution operation is performed on the vectorized weight data and the input data in the AM space. conv1×1 to obtain the feature map data after convolution, including the following steps:

Step 1, initialize i=0, wherein, i represents the block index of the weight sub-block matrix Wb _sm(i) in the M dimension;

Step 2, initialize j=0, wherein, j represents the block index of the input sub-block matrix Ib _am(j) on the N dimension;

Step 3. Initialize k=0, where k represents the column index of the weight sub-block Wb _sm and the row index of the input sub-block Ib _am , m1 represents the row index of the weight sub-block, n1 represents the column index of the input sub-block, That is, the weight sub-block is represented as Wb _sm(i,m1,k) , and the input sub-block is represented as Ib _am(j,k,n1) ;

Step 4. Initialize the vector register to 0, so that the vector register can accumulate and store the calculation result;

Step 5. The minimum granularity of the scalar load instruction is 4 bytes, and the half-precision data is 2 bytes. Two half-precision data will be loaded into R[0:15] and R[16:31] of the specified scalar register at a time. Load the kth column data Wb _{sm(i, 0, k)} ... Wb _{sm(i, m-1, k)} of the weight sub-block Wb _sm(i) in the SM space into the scalar register R ₃₀ , R ₃₁ ...R _30+m-1 in R[0:15], the k+1th column data Wb _sm(i,0,k+1) of the weight sub-block Wb _sm(i) at the same time ...Wb _{sm(i,m-1,k+1)} are sequentially loaded into R[16:31] of scalar registers R ₃₀ , R ₃₁ ... R _30+m-1 ;

Step 6. Based on the _{half -} precision weight data _{stored in the scalar registers R 30 , R 31} . Copy and expand the lower 16-bit data R[0:15] in the lower 32 bits of the register into d-bit data and store them in the scalar registers R ₄₀ , R ₄₁ ... R _40+m-1 , where d is a scalar register bit length;

Step 7. Based on the replicated and expanded data stored in the scalar registers R ₄₀ , R ₄₁ . . . R _{40+m-1 ,} the scalar registers R ₄₀ , R ₄₁ . The data is stored in the vector registers VR ₅₀ , VR ₅₁ . . . VR _50+m-1 , the L vector processing components store the same data, and the data in the kth column of Wb _sm(i) is vectorized;

个字节，故单次最少可加载

个半精度数据；Step 8. Load the k-th row data Ib _{am(j, k, 0)} ... Ib _{am(j, k, n-1)} of the input sub-block matrix Ib _am(j) in the AM space to p In the vector registers VR ₀ , VR ₁ ... VR _p-1 , p represents the number of functional vector arithmetic unit components in the architecture of the super-long data instruction word, and the minimum granularity of a single load is

bytes, so at least one can be loaded at a time

half-precision data;

Step 9. Perform multiplication and addition operations on the vectorized data VR ₅₀ of Wb _{sm(i, 0, k) and} the k _- th row data VR ₀ , VR ₁ . The L vector processing components operate in parallel, and store the calculation results in the vector registers VR ₁₀ , VR ₁₁ . . . VR _10+p-1 ;

Step 10. Based on the vector registers VR ₅₁ ...VR _50+m-1 store the vectorized data of the weight sub-blocks Wb _{sm(i, 1, k) ...} , the vector registers VR ₀ , VR ₁ . . . VR _p-1 store the data of the kth row of the input sub-block Ib _am(j) , repeat step 9, and compare each group of vectorized data of the weight with Ib _am Multiply the data in the kth row of _(j) , and accumulate the multiplied results to the vector registers VR _10+p , VR _10+p+1 ... VR _10+m×p-1 , in this process L vector processing The components operate in parallel at the same time, traverse the data of the kth column of Wb sm( _i ), until the multiplication and addition calculation of the kth column of Wb _sm(i) and the k row of Ib _am(j) is completed;

Step 11, judge whether k+1 is less than K, if yes, then jump to step 19, if not, continue to execute step 12;

Step 12. Based on the Wb _{sm (i, 1, k+1) stored} in R[16:31] of the scalar registers R ₃₀ , R ₃₁ . _1,k+1) data, perform high-order expansion operation on scalar registers R ₃₀ , R ₃₁ ... R _30+m-1 , and copy and expand the lower 32-bit, middle-high 16-bit data R[16:31] in the register as d-bit data is stored in scalar registers R ₄₀ , R ₄₁ . . . R _40+m-1 , where d is the bit length of a scalar register;

Step 13: Based on the replicated and expanded data _stored in the _scalar registers R ₄₀ , _{R 41 .} Store the broadcasted data in the vector registers VR ₅₀ , VR ₅₁ . . . VR _50+m-1 , the L vector processing units store the same data, and the vectorization of the data in the k+1th column of Wb _sm(i) is completed ;

个字节，故单次最少可加载

个半精度数据；Step 14: Convert the k+1 row data Ib _am(j,k+1,0) of the input sub-block matrix Ib _am(j) in the AM space to Ib _{am(j,k+1,n- 1)} Load into p vector registers VR ₀ , VR ₁ ... VR _p-1 , p represents the number of functional vector arithmetic unit components in the architecture of the super-long data instruction word, and the minimum granularity of a single load is

bytes, so at least one can be loaded at a time

half-precision data;

Step 15: Multiply the vectorized data VR ₅₀ of Wb _{sm(i, 0, k+1)} by the data VR ₀ , VR ₁ . . . VR _p-1 of the k+1 row of Ib _am(j) respectively Add operation, while L vector processing components operate in parallel, and store the calculation results in vector registers VR ₁₀ , VR ₁₁ . . . VR _10+p-1 ;

Step 16. Based on the vector registers VR ₅₁ ... VR _50+m-1 , the weight sub-blocks Wb _{sm(i, 1, k+1)} ... Wb _{sm(i, m-1, k+1)} are stored The vectorized data of , the vector registers VR ₀ , VR ₁ . . . VR _p-1 store the data of the k+1th row of the input sub-block Ib _am(j) . Multiply the data with the k+1th row data of Ib _am(j) respectively, and accumulate the multiplication results to the vector registers VR _10+p , VR _10+p+1 ... VR _10+m×p-1 In this process, L vector processing components operate in parallel at the same time, traversing the data of the k+1th column of Wb sm( _i ) until the k+1th column of Wb _sm(i) and the k+1 row of Ib _am(j) The multiplication and addition calculation is completed;

Step 17, let k=k+2;

Step 18, determine whether k is less than K, if so, go back to step 5, if not, go to step 19;

Step 19, temporarily store the data results in the vector registers VR ₁₀ , VR ₁₁ . . . VR _10+m×p-1 to the AM space position AM _temp ;

Step 20, call direct memory access operation, the characteristic map data result that described AM space position AM _temp is stored is stored to double rate synchronous dynamic random access memory designated position;

Step 21, let j=j+1;

Step 22, determine whether j is less than x ₂ , if so, call the direct memory access operation, load the input sub-block matrix Ib _am(j) into the on-chip AM space, and return to step 3 after loading, if not, execute step 23 ;

Step 23, let i=i+1;

Step 24, judge whether i is less than x ₁ , if so, call the direct memory access operation, load the weight sub-block matrix Wb _sm(i) into the on-chip SM space, and return to step 2 after loading, if not, go to this point The conv1×1 calculation of the weight data W _ddr and the input data I _ddr is completed.

A half-precision vectorized conv1×1 convolutional system for vector processors, including:

The storage module is used to store the half-precision weight data and the half-precision input data in the double-rate synchronous dynamic random access memory;

a loading module, configured to invoke a direct memory access operation, and load the half-precision weight data and half-precision input data from the double-rate synchronous dynamic random access memory into the on-chip scalar memory SM space and the on-chip array memory AM space, respectively;

The processing module is used to vectorize the weight data loaded into the on-chip SM space in the SM space, and in the AM space, convolve the vectorized weight data with the input data in the AM space. conv1×1, get the feature map data after convolution;

Wherein, the data format of the half-precision weight data Weight _ddr is [Co, Cin, ks, ks], Co is the number of output channels, Cin is the number of input channels, ks is the size of the convolution kernel, when the size of the convolution kernel is 1, the data format can be regarded as [Co, Cin], so the weight data can be expressed as a matrix Weight _ddr = M × K, the data format of the half-precision input data Input _ddr is [Cin, Hi, Wi, n], Hi and Wi are the height and width of the image respectively, n is the number of batch processing in the convolution operation, [Hi, Wi, n] can be regarded as one dimension, let N=Hi×Wi×n, so The input data can be represented as a matrix Input _ddr =K×N, where M represents Co, K represents Cin, and N represents the size of the image dimension.

Preferably, the loading module is specifically used for:

其中m的大小由SM的空间大小和AM空间的大小综合决定；Call the direct memory access operation, load the half-precision weight matrix W _ddr into the on-chip SM space, divide the original data from M dimension into x ₁ Wb _sm matrix, become W _sm = x ₁ ×Wb _sm , Wb _sm = m×K,

The size of m is determined comprehensively by the size of the SM space and the size of the AM space;

p表示向量处理器的体系结构中向量功能运算单元部件的数量，L表示向量处理部件的数量。Invoke the direct memory access operation, load the half-precision input matrix I _ddr into the on-chip AM space, divide the original data from N dimensions into x ₂ Ib _am matrices, become I _am =x ₂ ×Ib _am , where Ib _am = K×n, that is, N=x ₂ ×n, where n=P×L×4,

p represents the number of vector functional operation unit components in the architecture of the vector processor, and L represents the number of vector processing components.

Preferably, the processing module is specifically configured to perform the following steps:

Step 1, initialize i=0, wherein, i represents the block index of the weight sub-block matrix Wb _sm(i) in the M dimension;

Step 2, initialize j=0, wherein, j represents the block index of the input sub-block matrix Ib _{am (j0} on the N dimension;

Step 3. Initialize k=0, where k represents the column index of the weight sub-block Wb _sm and the row index of the input sub-block Ib _am , m1 represents the row index of the weight sub-block, n1 represents the column index of the input sub-block, That is, the weight sub-block is represented as Wb _sm(i,m1,k) , and the input sub-block is represented as Ib _am(j,k,n1) ;

Step 4. Initialize the vector register to 0, so that the vector register can accumulate and store the calculation result;

Step 5. The minimum granularity of the scalar load instruction is 4 bytes, and the half-precision data is 2 bytes. Two half-precision data will be loaded into R[0:15] and R[16:31] of the specified scalar register at a time. Load the kth column data Wb _{sm(i, 0, k)} ... Wb _{sm(i, m-1, k)} of the weight sub-block Wb _sm(i) in the SM space into the scalar register R ₃₀ , R ₃₁ ...R _30+m-1 in R[0:15], the k+1th column data Wb _sm(i,0,k+1) of the weight sub-block Wb _sm(i) at the same time ...Wb _{sm(i,m-1,k+1)} are sequentially loaded into R[16:31] of scalar registers R ₃₀ , R ₃₁ ... R _30+m-1 ;

Step 6. Based on the _{half -} precision weight data _{stored in the scalar registers R 30 , R 31} . Copy and expand the lower 16-bit data R[0:15] in the lower 32 bits of the register into d-bit data and store them in the scalar registers R ₄₀ , R ₄₁ ... R _40+m-1 , where d is a scalar register bit length;

Step 7. Based on the replicated and expanded data stored in the scalar registers R ₄₀ , R ₄₁ . . . R _{40+m-1 ,} the scalar registers R ₄₀ , R ₄₁ . The data is stored in the vector registers vr ₅₀ , vr ₅₁ . . . VR _50+m-1 , the L vector processing components store the same data, and the data in the kth column of Wb _sm(i) is vectorized;

个字节，故单次最少可加载

个半精度数据；Step 8. Load the k-th row data Ib _{am(j, k, 0)} ... Ib _{am(j, k, n-1)} of the input sub-block matrix Ib _am(j) in the AM space to p In the vector registers VR ₀ , VR ₁ ... VR _p-1 , p represents the number of functional vector arithmetic unit components in the architecture of the super-long data instruction word, and the minimum granularity of a single load is

bytes, so at least one can be loaded at a time

half-precision data;

Step 9. Perform multiplication and addition operations on the vectorized data VR ₅₀ of Wb _{sm(i, 0, k) and} the k _- th row data VR ₀ , VR ₁ . The L vector processing components operate in parallel, and store the calculation results in the vector registers VR ₁₀ , VR ₁₁ . . . VR _10+p-1 ;

Step 10. Based on the vector registers VR ₅₁ ...VR _50+m-1 store the vectorized data of the weight sub-blocks Wb _{sm(i, 1, k) ...} , the vector registers VR ₀ , VR ₁ . . . VR _p-1 store the data of the kth row of the input sub-block Ib _am(j) , repeat step 9, and compare each group of vectorized data of the weight with Ib _am Multiply the data in the kth row of _(j) , and accumulate the multiplied results to the vector registers VR _10+p , VR _10+p+1 ... VR _10+m×p-1 , in this process L vector processing The components operate in parallel at the same time, traverse the data of the kth column of Wb sm( _i ), until the multiplication and addition calculation of the kth column of Wb _sm(i) and the k row of Ib _am(j) is completed;

Step 11, judge whether k+1 is less than K, if yes, then jump to step 19, if not, continue to execute step 12;

Step 12. Based on the Wb _{sm (i, 1, k+1) stored} in R[16:31] of the scalar registers R ₃₀ , R ₃₁ . _1,k+1) data, perform high-order expansion operation on scalar registers R ₃₀ , R ₃₁ ... R _30+m-1 , and copy and expand the lower 32-bit, middle-high 16-bit data R[16:31] in the register as d-bit data is stored in scalar registers R ₄₀ , R ₄₁ . . . R _40+m-1 , where d is the bit length of a scalar register;

Step 13: Based on the replicated and expanded data _stored in the _scalar registers R ₄₀ , _{R 41 .} Store the broadcasted data in the vector registers VR ₅₀ , VR ₅₁ . . . VR _50+m-1 , the L vector processing units store the same data, and the vectorization of the data in the k+1th column of Wb _sm(i) is completed ;

个字节，故单次最少可加载

个半精度数据；Step 14: Convert the k+1 row data Ib _am(j,k+1,0) of the input sub-block matrix Ib _am(j) in the AM space to Ib _{am(j,k+1,n- 1)} Load into p vector registers VR ₀ , VR ₁ ... VR _p-1 , p represents the number of functional vector arithmetic unit components in the architecture of the super-long data instruction word, and the minimum granularity of a single load is

bytes, so at least one can be loaded at a time

half-precision data;

Step 15. Multiply the data VR ₅₀ after vectorization of Wb _{sm(i, 0, k+1)} by the data VR ₀ , VR ₁ . . . VR _p-1 of the k+1th row of Ib _am(j) respectively Add operation, while L vector processing components operate in parallel, and store the calculation results in vector registers VR ₁₀ , VR ₁₁ . . . VR _10+p-1 ;

Step 16. Based on the vector registers VR ₅₁ ... VR _50+m-1 , the weight sub-blocks Wb _{sm(i, 1, k+1)} ... Wb _{sm(i, m-1, k+1)} are stored The vectorized data of , the vector registers VR ₀ , VR ₁ . . . VR _p-1 store the data of the k+1th row of the input sub-block Ib _am(j) . Multiply the data with the k+1th row data of Ib _am(j) respectively, and accumulate the multiplication results to the vector registers VR _10+p , VR _10+p+1 ... VR _10+m×p-1 In this process, L vector processing components operate in parallel at the same time, traversing the data of the k+1th column of Wb sm( _i ) until the k+1th column of Wb _sm(i) and the k+1 row of Ib _am(j) The multiplication and addition calculation is completed;

Step 17, let k=k+2;

Step 18, determine whether k is less than K, if so, go back to step 5, if not, go to step 19;

Step 19, temporarily store the data results in the vector registers VR ₁₀ , VR ₁₁ . . . VR _10+m×p-1 to the AM space position AM _temp ;

Step 20, call direct memory access operation, the characteristic map data result that described AM space position AM _temp is stored is stored to double rate synchronous dynamic random access memory designated position;

Step 21, let j=j+1;

Step 22, determine whether j is less than x ₂ , if so, call the direct memory access operation, load the input sub-block matrix Ib _am(j) into the on-chip AM space, and return to step 3 after loading, if not, execute step 23 ;

Step 23, let i=i+1;

Step 24, judge whether i is less than x ₁ , if so, call the direct memory access operation, load the weight sub-block matrix Wb _sm(i) into the on-chip SM space, and return to step 2 after loading, if not, go to this point The conv1×1 calculation of the weight data W _ddr and the input data I _ddr is completed.

To sum up, the present invention discloses a vector processor-oriented half-precision vectorized conv1×1 convolution method. First, half-precision weight data and half-precision input data are stored in a double-rate synchronous dynamic random access memory, Then the direct memory access operation is called to load the half-precision weight data and half-precision input data from the double-rate synchronous dynamic random access memory into the on-chip scalar memory SM space and the on-chip array memory AM space respectively; in the SM space, the pair is loaded into the on-chip The weight data in the SM space is vectorized. In the AM space, the vectorized weight data and the input data in the AM space are convolved with the convolution operation conv1×1 to obtain the feature map data after convolution; , the data format of the half-precision weight data Weight _ddr is [Co, Cin, ks, ks], Co is the number of output channels, Cin is the number of input channels, ks is the size of the convolution kernel, when the size of the convolution kernel is 1, The data format can be regarded as [Co, Cin], so the weight data can be expressed as a matrix Weight _ddr = M×K, the data format of the half-precision input data Input _ddr is [Cin, Hi, Wi, n], Hi and Wi are the height and width of the image, respectively, and n is the number of batch processing in the convolution operation. [Hi, Wi, n] can be regarded as one dimension, and N=Hi×Wi×n, so the input data can be It is expressed as a matrix Input _ddr =K×N, where M represents Co, K represents Cin, and N represents the size of the image dimension. The invention can combine the architectural features of the vector processor to vectorize the convolution calculation (conv1×1) to the vector processor architecture, and realize the improvement of FLOPs on the premise of ensuring the accuracy.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

Fig. 1 is a general architecture schematic diagram of a vector processor;

2 is a method flow chart of an embodiment of a vector processor-oriented half-precision vectorized conv1×1 convolution method provided by the present invention;

3 is a schematic diagram of scalar loading of Wb _{sm(0, m1, k)} disclosed in the present invention;

4 is a schematic diagram of the low 16-bit extension of the scalar register disclosed in the present invention;

5 is a schematic diagram of a broadcast implementation of a scalar register disclosed in the present invention;

Fig. 6 is a kind of vector loading schematic diagram of Ib _am(0,0,n1) disclosed by the present invention;

7 is a schematic diagram of vector multiplication and addition of Wb _{sm(i, 0, k)} and the kth row of input disclosed in the present invention;

8 is a schematic diagram of vector multiplication and addition of the kth column of weight and the kth row of input disclosed by the present invention;

FIG. 9 is a schematic diagram of the high-order 16-bit extension of the scalar register disclosed in the present invention;

10 is a schematic diagram of a broadcast implementation of a scalar register disclosed in the present invention;

11 is a schematic diagram of a vector loading of Ib _am(0,1,n1) disclosed in the present invention;

12 is a schematic diagram of vector multiplication and addition of Wb _{sm(i, 0, k+1)} and input row k+1 disclosed in the present invention;

Figure 13 is a schematic diagram of vector multiplication and addition of the k+1 column of weight and the k+1 row of input;

Figure 14 is a schematic diagram of vector multiplication and addition of the last column of weight and the last row of input;

FIG. 15 is a schematic structural diagram of an embodiment of a vector processor-oriented half-precision vectorized conv1×1 convolution system disclosed in the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

As shown in FIG. 2, it is a flowchart of an embodiment of a vector processor-oriented half-precision vectorized conv1×1 convolution method disclosed in the present invention, and the method may include the following steps:

S201. Store half-precision weight data and half-precision input data in a double-rate synchronous dynamic random access memory;

When vectorized convolution of half-precision data for vector processors is required, firstly, half-precision weight data and half-precision input data are stored in DDR (Double Rate Synchronous Dynamic Random Access Memory). Among them, the data format of the half-precision weight data Weight _ddr is [Co, Cin, ks, ks], Co is the number of output channels, Cin is the number of input channels, ks is the size of the convolution kernel, when the size of the convolution kernel is 1 , the data format can also be regarded as [Co, Cin], so the weight data can be expressed as a matrix Weight _ddr =M×K. The data format of the half-precision input data Input _ddr is [Cin, Hi, Wi, n], Hi and Wi are the height and width of the image respectively, and n is the number of batch processing in the convolution operation. Wi,n] is regarded as one dimension, let N=Hi×Wi×n, so the input data can be represented as a matrix Input _ddr =K×N, where M represents Co, K represents Cin, and N represents the size of the image dimension.

S202, calling the direct memory access operation, and loading the half-precision weight data and the half-precision input data from the double-rate synchronous dynamic random access memory into the on-chip scalar memory SM space and the on-chip array memory AM space respectively;

其中m的大小由SM的空间大小和AM空间的大小综合决定。如，与m相关的权值数据块Wb_sm大小不能大于SM空间；权值块与输入块卷积后的输出结果与输入数据块大小之和需小于AM空间。Specifically, the direct memory access operation is invoked, the half-precision weight matrix W _ddr is loaded into the on-chip SM space, and the original data is divided from M dimension (output channel dimension) into x ₁ Wb _sm matrix, which becomes W _sm =x ₁ ×Wb _sm , Wb _sm =m×K,

The size of m is determined comprehensively by the size of the SM space and the size of the AM space. For example, the size of the weight data block Wb _sm related to m cannot be larger than the SM space; the sum of the output result after the convolution of the weight block and the input block and the size of the input data block must be smaller than the AM space.

p表示向量处理器的体系结构中向量功能运算单元部件的数量，L表示向量处理部件的数量。The direct memory access operation is invoked, the half-precision input matrix I _ddr is loaded into the on-chip AM space, and the original data is divided from N dimensions (image layer dimension) into x ₂ Ib _am matrices, becoming I _am =x ₂ × Ib _am , where Ib _am =K×n. That is, N=x ₂ ×n, where n=P×L×4,

p represents the number of vector functional operation unit components in the architecture of the vector processor, and L represents the number of vector processing components.

S203. In the SM space, perform vectorization processing on the weight data loaded into the on-chip SM space, and in the AM space, perform a convolution operation conv1×1 between the vectorized weight data and the input data in the AM space , to obtain the feature map data after convolution.

Specifically, the following steps may be included:

Step 1, initialize i=0, wherein, i represents the block index of the weight sub-block matrix Wb _sm(i) in the M dimension;

Step 2, initialize j=0, wherein, j represents the block index of the input sub-block matrix Ib _am(j) on the N dimension;

Step 3. Initialize k=0, where k represents the column index of the weight sub-block Wb _sm and the row index of the input sub-block Ib _am , m1 represents the row index of the weight sub-block, n1 represents the column index of the input sub-block, That is, the weight sub-block is represented as Wb _sm(i,m1,k) , and the input sub-block is represented as Ib _am(j,k,n1) ;

Step 4. Initialize the vector register to 0, so that the vector register can accumulate and store the calculation result;

Step 5. The minimum granularity of the scalar load instruction is 4 bytes, and the half-precision data is 2 bytes. Two half-precision data will be loaded into R[0:15] and R[16:31] of the specified scalar register at a time. Load the kth column data Wb _{sm(i, 0, k)} ... Wb _{sm(i, m-1, k)} of the weight sub-block Wb _sm(i) in the SM space into the scalar register R ₃₀ , R ₃₁ ...R _30+m-1 in R[0:15], the k+1th column data Wb _sm(i,0,k+1) of the weight sub-block Wb _sm(i) at the same time ...Wb _{sm(i,m-1,k+1)} are sequentially loaded into R[16:31] of scalar registers R ₃₀ , R ₃₁ ... R _30+m-1 ;

For example, take the first weight sub-block Wb _sm(0) = 6×4, m=6, K=4 as an example, when k=0, use the scalar load instruction to sequentially load the first weight of Wb _sm(0) The data of the column is loaded into R[0:15] of the scalar registers _R30 , _R31 ...R30 _+m-1 , and the data of the second column of Wb _sm(0) is loaded into the scalar register _R30 , In R[16:31] of R ₃₁ ...R _30+m-1 , as shown in Figure 3 below.

Step 6. Based on the _{half -} precision weight data _{stored in the scalar registers R 30 , R 31} . Copy and expand the lower 16-bit data R[0:15] in the lower 32 bits of the register into d-bit data and store them in the scalar registers R ₄₀ , R ₄₁ ... R _40+m-1 , where d is a scalar register bit length;

For example, taking d=64 as an example, the implementation of the extended instruction of the lower 32 bits in the lower 16 bits in step 6 is shown in FIG. 4 .

Step 7. Based on the replicated and expanded data stored in the scalar registers R ₄₀ , R ₄₁ . . . R _{40+m-1 ,} the scalar registers R ₄₀ , R ₄₁ . The data is stored in the vector registers VR ₅₀ , VR ₅₁ . . . VR _50+m-1 , the L vector processing components store the same data, and the k-th column data of Wb _sm5i) is vectorized and completed;

For example, taking L=8 as an example, the implementation of broadcasting the scalar register R ₄₀ to the vector register VR ₅₀ is shown in FIG. 5 .

个字节，故单次最少可加载

个半精度数据；Step 8. Load the k-th row data Ib _{am(j, k, 0)} ... Ib _{am(j, k, n-1)} of the input sub-block matrix Ib _am(j) in the AM space to p In the vector registers VR ₀ , VR ₁ ... VR _p-1 , p represents the number of functional vector arithmetic unit components in the architecture of the super-long data instruction word, and the minimum granularity of a single load is

bytes, so at least one can be loaded at a time

half-precision data;

For example, take the first input sub-block Ib _am(0) = 4×64, K=4, N=64 as an example, when k=0, use the vector load instruction to load the first line of Ib _am(0) The data is loaded into the p vector registers VR ₀ , VR ₁ . . . VR _p-1 . Taking L=8 and p=2 as an example, the specific implementation of vector loading is shown in FIG. 6 .

Step 9. Perform multiplication and addition operations on the vectorized data VR ₅₀ of Wb _{sm(i, 0, k)} and the data VR ₀ , VR ₁ ... VR _p-1 of the kth row of Ib _am(j) respectively, because The architecture integrates p functional vector arithmetic unit components, so the above multiplication and addition operations are supported in the same cycle, while L vector processing components operate in parallel, and store the calculation results in vector registers VR ₁₀ , VR ₁₁ . . . VR _{10 +p-1} ;

For example, VR ₅₀ performs multiplication and addition operations with VR ₀ and VR ₁ respectively. Taking L=8 and p=2 as an example, the results are stored in VR ₁₀ and VR _11. Since the initial values of VR ₁₀ and VR ₁₁ are 0, the multiplication The addition result is the multiplication itself, and the specific implementation is shown in Figure 7.

Step 10. Based on the vector registers VR ₅₁ ...VR _50+m-1 store the vectorized data of the weight sub-blocks Wb _{sm(i, 1, k) ...} , the vector registers VR ₀ , VR ₁ . . . VR _p-1 store the data of the kth row of the input sub-block Ib _am(j) , repeat step 9, and compare each group of vectorized data of the weight with Ib _am Multiply the data in the kth row of _(j) , and accumulate the multiplied results to the vector registers VR _10+p , VR _10+p+1 ... VR _10+m×p-1 , in this process L vector processing The components operate in parallel at the same time, traverse the data of the kth column of Wb sm( _i ), until the multiplication and addition calculation of the kth column of Wb _sm(i) and the k row of Ib _am(j) is completed, and the specific implementation is shown in Figure 8;

Step 11, judge whether k+1 is less than K, if yes, then jump to step 19, if not, continue to execute step 12;

Step 12. Based on the Wb _{sm (i, 1, k+1) stored} in R[16:31] of the scalar registers R ₃₀ , R ₃₁ . _1,k+1) data, perform high-order expansion operation on scalar registers R ₃₀ , R ₃₁ ... R _30+m-1 , and copy and expand the lower 32-bit, middle-high 16-bit data R[16:31] in the register as d-bit data is stored in scalar registers R ₄₀ , R ₄₁ . . . R _40+m-1 , where d is the bit length of a scalar register;

For example, taking d=64 as an example, the implementation of the extended instruction of the lower 32 bits in the upper 16 bits in step 12 is shown in FIG. 9 .

Step 13: Based on the replicated and expanded data _stored in the _scalar registers R ₄₀ , _{R 41 .} Store the broadcasted data in the vector registers VR ₅₀ , VR ₅₁ . . . VR _50+m-1 , the L vector processing units store the same data, and the vectorization of the data in the k+1th column of Wb _sm(i) is completed ;

For example, when k=0, the data vectorization of the k+1th column of Wb _sm(i) is as follows, and the specific broadcast implementation is shown in FIG. 10 .

个字节，故单次最少可加载

个半精度数据；Step 14: Convert the k+1 row data Ib _am(j,k+1,0) of the input sub-block matrix Ib _am(j) in the AM space to Ib _{am(j,k+1,n- 1)} Load into p vector registers VR ₀ , VR ₁ ... VR _p-1 , p represents the number of functional vector arithmetic unit components in the architecture of the super-long data instruction word, and the minimum granularity of a single load is

bytes, so at least one can be loaded at a time

half-precision data;

For example, taking the first input sub-block Ib _am(0) = 4×64, K=4, N=64 as an example, when k+1=1, use the vector load instruction to load the second sub-block of Ib am(0 ₎ . The data of the row is loaded into p vector registers VR ₀ , VR ₁ . . . VR _p-1 . Taking L=8 and p=2 as an example, the specific implementation of vector loading is shown in FIG. 11 .

Step 15. Multiply the data VR ₅₀ after vectorization of Wb _{sm(i, 0, k+1)} by the data VR ₀ , VR ₁ . . . VR _p-1 of the k+1th row of Ib _am(j) respectively Add operation, because the architecture integrates p functional vector arithmetic unit components, so the above multiplication and addition operations are supported in the same cycle, while L vector processing components operate in parallel, and the calculation results are stored in vector registers VR ₁₀ , VR ₁₁ . ..VR _10+p-1 ;

For example, when k+1=1, VR ₅₀ performs multiplication and addition operations with VR ₀ and VR ₁ respectively, and accumulates the multiplication and addition data of k rows in VR ₁₀ and VR ₁₁ , and continues to save the results in VR ₁₀ and VR ₁₁ , taking L=8 and p=2 as an example, the specific implementation is shown in FIG. 12 .

Step 16. Based on the vector registers VR ₅₁ ... VR _50+m-1 , the weight sub-blocks Wb _{sm(i, 1, k+1)} ... Wb _{sm(i, m-1, k+1)} are stored The vectorized data of , the vector registers VR ₀ , VR ₁ . . . VR _p-1 store the data of the k+1th row of the input sub-block Ib _am(j) . Multiply the data with the k+1th row data of Ib _am(j) respectively, and accumulate the multiplication results to the vector registers VR _10+p , VR _10+p+1 ... VR _10+m×p-1 In this process, L vector processing components operate in parallel at the same time, traversing the data of the k+1th column of Wb sm( _i ) until the k+1th column of Wb _sm(i) and the k+1 row of Ib _am(j) The multiplication and addition calculation is completed, and the specific implementation is shown in Figure 13;

Step 17, let k=k+2;

Step 18, determine whether k is less than K, if so, go back to step 5, if not, go to step 19;

Step 19. So far, the conv1×1 calculation of the weight sub-block matrix Wb _sm(i) and the input sub-block matrix Ib _am(j) has been completed. When Wb _sm(i) traverses to the last column, Ib _am(j) traverses When reaching the last line, the specific operation is as shown in Figure 14, and the data results stored in the vector registers VR ₁₀ , VR ₁₁ . . . VR _10+m×p-1 are temporarily stored in the AM space position AM _temp ;

Step 20, call direct memory access operation, the characteristic map data result that described AM space position AM _temp is stored is stored to double rate synchronous dynamic random access memory designated position;

Step 21, let j=j+1;

Step 22, determine whether j is less than x ₂ , if so, call the direct memory access operation, load the input sub-block matrix Ib _am(j) into the on-chip AM space, return to step 3 after loading, and repeat the above scalar data loading, Operations such as copy extension, broadcast, vector data loading, and vector multiply-add, if not, go to step 23;

Step 23, let i=i+1;

Step 24, judge whether i is less than x ₁ , if so, call the direct memory access operation, load the weight sub-block matrix Wb _sm(i) into the on-chip SM space, and return to step 2 after loading, and repeat the above scalar data loading , copy extension, broadcast, vector data loading, and vector multiply-add operations, if not, then all the weight data W _ddr and the input data I _ddr conv1×1 calculation is completed.

To sum up, the present invention discloses a vector processor-oriented half-precision vectorized conv1×1 convolution method, which can combine the architectural features of the vector processor to make the convolution calculation (conv1×1) oriented to the vector processor. The architecture is vectorized, and the improvement of FLOPs is achieved on the premise of ensuring accuracy.

As shown in FIG. 15, it is a schematic structural diagram of an embodiment of a vector processor-oriented half-precision data vectorization conv1×1 convolution system disclosed in the present invention. The system may include:

A storage module 1501, configured to store half-precision weight data and half-precision input data in a double-rate synchronous dynamic random access memory;

a loading module 1502, configured to invoke a direct memory access operation, and load the half-precision weight data and the half-precision input data from the double-rate synchronous dynamic random access memory into the on-chip scalar memory SM space and the on-chip array memory AM space, respectively;

The processing module 1503 is used to perform vectorization processing on the weight data loaded into the on-chip SM space in the SM space, and in the AM space, convolve the vectorized weight data with the input data in the AM space Operate conv1×1 to get the feature map data after convolution.

The present invention discloses the working principle of the vector processor-oriented half-precision vectorized conv1×1 convolution system, which is the same as the working principle of the vector processor-oriented half-precision vectorized conv1×1 convolution method, and will not be repeated here.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the possibilities of hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of functionality. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (6)

1. A vector processor-oriented semi-precision vectorized conv1 x 1 convolution method, comprising:

storing the half-precision weight data and the half-precision input data in a double-rate synchronous dynamic random access memory;

calling direct memory access operation, and respectively loading the semi-precision weight data and the semi-precision input data from the double-rate synchronous dynamic random access memory to an on-chip scalar memory SM space and an on-chip array memory AM space;

in an SM space, vectorizing the weight data loaded to the SM space on the chip, and in an AM space, performing convolution operation conv1 multiplied by 1 on the vectorized weight data and input data on the AM space to obtain feature map data after convolution;

wherein, the semi-precision Weight data Weight _ddr The data format of (C) is [ Co, Cin, ks]Co is the number of output channels, Cin is the number of input channels, ks is the size of convolution kernel, and when the size of convolution kernel is 1, the data format can be regarded as [ Co, Cin]Therefore, the weight data can be expressed asMatrix Weight _ddr M × K, the semi-precision Input data Input _ddr The data format of (1) is [ Cin, Hi, Wi, n]Where Hi and Wi are the height and width of the image, respectively, and n is the number of batch processes at a time in the convolution operation, the values [ Hi, Wi, n ] can be obtained]Considering one dimension, let N be Hi × Wi × N, so the Input data can be represented as a matrix Input _ddr Where M denotes Co, K denotes Cin, and N denotes the size of the image dimension.

2. The method of claim 1, wherein the invoking of the direct memory access operation loads the half-precision weight data and the half-precision input data from the double rate synchronous dynamic random access memory into an on-chip Scalar Memory (SM) space and an on-chip Array Memory (AM) space, respectively, comprising:

invoking a direct memory access operation to apply a semi-precision weight matrix W _ddr Loading into on-chip SM space, dividing original data into x from M dimension ₁ A Wb _sm Matrix, becomes W _sm ＝x ₁ ×Wb _sm ，Wb _sm ＝m×K，

Wherein the size of m is comprehensively determined by the space size of SM and the size of AM space;

invoking a direct memory access operation to input the semi-precision into matrix I _ddr Loading into AM space on chip, dividing original data into x from N dimension ₂ Ib _am Matrix, becomes I _am ＝x ₂ ×Ib _am Wherein Ib is _am K x N, i.e. N x ₂ X n, where n ═ P × L × 4,

p denotes the number of vector function arithmetic unit elements in the architecture of the vector processor, and L denotes the number of vector processing elements.

3. The method according to claim 2, wherein in the SM space, vectorization processing is performed on the weight data loaded into the on-chip SM space, and in the AM space, convolution operation conv1 × 1 is performed on the vectorized weight data and input data in the AM space to obtain convolved feature map data, including the following steps:

step 1, initializing i to 0, wherein i represents a weight subblock matrix Wb _sm(i) A block index in the M dimension;

step 2, initializing j to 0, wherein j represents an input sub-block matrix Ib _am(j) A block index in the N dimension;

step 3, initializing k to 0, wherein k represents the weight subblock Wb _sm Column index and input sub-block Ib _am M1 denotes a row index of the weight subblock, n1 denotes a column index of the input subblock, i.e., the weight subblock is denoted as Wb _{sm(i，m1，k)} Input sub-block denoted Ib _{am(j，k，n1)} ；

Step 4, initializing the vector register to 0 so as to accumulate the vector register and store the calculation result;

and 5, the minimum granularity of the scalar loading instruction is 4 bytes, the semi-precision data is 2 bytes, and two pieces of semi-precision data are loaded to the R [0:15]And R [16:31]The weight sub-block Wb in the SM space _sm(i) K-th column data Wb of _{sm(i，0，k)} ......Wb _{sm(i，m-1，k)} Loaded into scalar registers R in sequence ₃₀ 、R ₃₁ ...R _30+m-1 R [0:15]Middle and simultaneous weight sub-block Wb _sm(i) Column k +1 data Wb _{sm(i，0，k+1)} ......Wb _{sm(i，m-1，k+1)} Loaded into scalar registers R in sequence ₃₀ 、R ₃₁ ...R _30+m-1 R [16:31]Performing the following steps;

step 6, based on scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 Stored semi-precision weight data for scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 And performing low-order expansion operation, namely performing low-order expansion operation on the low-order 16-order data R [0:15]Replication extension to d-bit data storage in scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 Wherein d is a scalar registerThe bit length of (d);

step 7, based on scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 Stored replicated extended data, for scalar registers R ₄₀ 、R ₄₁ ...R _40+m-1 Broadcast operations are performed in sequence and data is stored in vector register VR ₅₀ 、VR ₅₁ ...VR _50+m-1 In which L vector processing elements store the same data, Wb _sm(i) Completing the k-th column data vectorization;

step 8, inputting the sub-block matrix Ib in the AM space _am(j) Of kth line data Ib _am(j，k，0 )......Ib _{am(j，k，n-1)} Loading into p vector registers VR ₀ 、VR ₁ ...VR _p-1 Where p represents the number of functional vector arithmetic unit elements in an architecture for very long data instruction words, with a minimum granularity of one load

One byte, so that it can be loaded at least once

Half precision data;

step 9, mixing Wb _{sm(i，0，k)} Vectorized data VR ₅₀ Respectively react with Ib _am(j) VR of the kth line ₀ 、VR ₁ ...VR _p-1 Performing multiply-add operation, simultaneously operating L vector processing units in parallel, and storing the calculation result in a vector register VR ₁₀ 、VR ₁₁ ...VR _10+p-1 Performing the following steps;

step 10, register VR based on vector ₅₁ ...VR _50+m-1 Stored is the weight sub-block Wb _{sm(i，1，k)} ......Wb _{sm(i，m-1，k)} Vectorized data, vector register VR ₀ 、VR ₁ ...VR _p-1 Stored in is an input sub-block Ib _am(j) Repeating the step 9 to respectively combine each group of quantized data of the weight with the Ib _am(j) And adds the multiplication result to the vector register VR _10+p 、VR _10+p+1 ....VR _10+m×p-1 In the process, L vector processing elements operate in parallel simultaneously, traversing Wb _sm(i) Up to Wb _sm(i) K column of (1) and Ib _am(j) The multiplication and addition calculation of the k rows is completed;

step 11, judging whether K +1 is smaller than K, if so, skipping to execute step 19, and if not, continuing to execute step 12;

step 12, based on scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 R [16:31]Wb stored in _{sm(i，1，k+1)} ......Wb _{sm(i，m-1，k+1)} Data, to scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 And performing high bit expansion operation, and enabling 16 high bits data R [16:31]Replication extension to d-bit data storage in scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 In (d) is the bit length of a scalar register;

step 13, based on scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 Stored replicated extended data, for scalar registers R ₄₀ 、R ₄₁ ...R _40+m-1 Broadcast operation is carried out in sequence, and the broadcasted data is stored in a vector register VR ₅₀ 、VR ₅₁ ...VR _50+m-1 In which L vector processing elements store the same data, Wb _sm(i) Completing the vectorization of the k +1 th column of data;

step 14, inputting the sub-block matrix Ib in the AM space _am(j) Data Ib of the k +1 th line _{am(j，k+1，0)} ......Ib _{am(j，k+1，n-1)} Loading into p vector registers VR ₀ 、VR ₁ ...VR _p-1 Where p represents the number of functional vector arithmetic unit elements in an architecture for very long data instruction words, with a minimum granularity of one load

One byte, so that it can be loaded at least once

One and a halfPrecision data;

step 15, mixing Wb _{sm(i，0，k+1)} Vectorized data VR ₅₀ Respectively react with Ib _am(j) The (k + 1) th row of data VR ₀ 、VR ₁ ...VR _p-1 Performing multiply-add operation, simultaneously operating L vector processing units in parallel, and storing the calculation result in a vector register VR ₁₀ 、VR ₁₁ ...VR _10+p-1 Performing the following steps;

step 16, vector register VR based ₅₁ ...VR _50+m-1 Stored is the weight sub-block Wb _{sm(i，1，k+1)} ......Wb _{sm(i，m-1，k+1)} Vectorized data, vector register VR ₀ 、VR ₁ ...VR _p-1 Stored in is an input sub-block Ib _am(j) Repeating the step 15 for the (k + 1) th row of data, and respectively comparing each group of quantized data of the weight with the Ib _am(j) And adds the multiplication result to the vector register VR _10+p 、VR _10+p+1 ...VR _10+m×p-1 In the process, L vector processing elements operate in parallel simultaneously, traversing Wb _sm(i) Up to Wb _sm(i) Column k +1 and Ib _am(j) The multiplication and addition calculation of the k +1 line is completed;

step 17, making k equal to k + 2;

step 18, judging whether K is smaller than K, if so, returning to the step 5, otherwise, executing the step 19;

step 19, store in vector register VR ₁₀ 、VR ₁₁ ...VR _10+m×p-1 Temporarily storing the data result in the AM space position AM _temp ；

Step 20, calling direct memory access operation, and AM the spatial position of AM _temp Storing the stored characteristic diagram data result to the appointed position of the double-rate synchronous dynamic random access memory;

step 21, making j equal to j + 1;

step 22, judging whether j is less than x ₂ If yes, calling direct memory access operation and inputting the subblock matrix Ib _am(j) Loading the data into an on-chip AM space, returning to the step 3 after the loading is finished, and if not, executing the step 23;

step 23, making i equal to i + 1;

step 24, judging whether i is less than x ₁ If yes, calling direct memory access operation and making weight value sub-block matrix Wb _sm(i) Loading into the SM space on the chip, returning to the step 2 after loading, and if not, obtaining all weight data W _ddr And input data I _ddr The conv1 × 1 calculation of (a) is completed.

4. A vector processor-oriented, half-precision vectorized conv1 x 1 convolution system comprising:

the storage module is used for storing the half-precision weight data and the half-precision input data in the double-rate synchronous dynamic random access memory;

the loading module is used for calling direct memory access operation and respectively loading the semi-precision weight data and the semi-precision input data from the double-rate synchronous dynamic random access memory to an on-chip scalar memory SM space and an on-chip array memory AM space;

the processing module is used for vectorizing the weight data loaded to the on-chip SM space in the SM space, and performing convolution operation conv1 multiplied by 1 on the vectorized weight data and input data on the AM space in the AM space to obtain feature map data after convolution;

wherein, the semi-precision Weight data Weight _ddr The data format of (C) is [ Co, Cin, ks]Co is the number of output channels, Cin is the number of input channels, ks is the size of convolution kernel, and when the size of convolution kernel is 1, the data format can be regarded as [ Co, Cin]Therefore, the Weight data can be expressed as a matrix Weight _ddr M × K, the semi-precision Input data Input _ddr The data format of (1) is [ Cin, Hi, Wi, n]Hi and Wi are the height and width of the image, respectively, and n is the number of one batch process in the convolution operation, which can be defined as [ Hi, Wi, n [ ]]Considering one-dimensional, let N be Hi × Wi × N, so the Input data can be expressed as a matrix Input _ddr Where M denotes Co, K denotes Cin, and N denotes the size of the image dimension.

5. The system of claim 4, wherein the loading module is specifically configured to:

invoking a direct memory access operation to apply a semi-precision weight matrix W _ddr Loading into on-chip SM space, dividing original data into x from M dimension ₁ A Wb _sm Matrix, becomes W _sm ＝x ₁ ×Wb _sm ，Wb _sm ＝m×K，

Wherein the size of m is comprehensively determined by the space size of SM and the size of AM space;

invoking a direct memory access operation to input the semi-precision into matrix I _ddr Loading into AM space on chip, dividing original data into x from N dimension ₂ Ib _am Matrix, becomes I _am ＝x ₂ ×Ib _am Wherein Ib is _am K × N, i.e. N ═ x ₂ X n, where n ═ P × L × 4,

p denotes the number of vector function arithmetic unit elements in the architecture of the vector processor, and L denotes the number of vector processing elements.

6. The system of claim 5, wherein the processing module is specifically configured to perform the steps of:

step 1, initializing i to 0, wherein i represents a weight subblock matrix Wb _sm(i) A block index in the M dimension;

step 2, initializing j to 0, wherein j represents an input sub-block matrix Ib _am(j) A block index in the N dimension;

step 3, initializing k to 0, wherein k represents the weight subblock Wb _sm And the input sub-block Ib _am M1, and n1, i.e., the weight subblocks are denoted as Wb _{sm(i，m1，k)} Input sub-block denoted Ib _{am(j，k，n1)} ；

Step 4, initializing the vector register to 0 so as to accumulate the vector register and store the calculation result;

and 5, the minimum granularity of the scalar loading instruction is 4 bytes, the semi-precision data is 2 bytes, and two pieces of semi-precision data are loaded to the R [0:15]And R [16:31]The weight sub-block Wb in the SM space _sm(i) K-th column data Wb of _{sm(i，0，k)} ......Wb _{sm(i，m-1，k)} Loaded into scalar registers R in sequence ₃₀ 、R ₃₁ ...R _30+m-1 R [0:15]Middle and weight sub-block Wb _sm(i) Column k +1 data Wb _{sm(i，0，k+1)} ......Wb _{sm(i，m-1，k+1)} Loaded into scalar registers R in sequence ₃₀ 、R ₃₁ ...R _30+m-1 R [16:31]Performing the following steps;

step 6, based on scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 Stored semi-precision weight data for scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 And performing low-order expansion operation, namely performing low-order expansion operation on the low-order 16-order data R [0:15]Replication extension to d-bit data storage in scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 Wherein d is the bit length of a scalar register;

step 7, based on scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 Stored replicated extended data, for scalar registers R ₄₀ 、R ₄₁ ...R _40+m-1 Broadcast operations are performed in sequence and data is stored in vector register VR ₅₀ 、VR ₅₁ ...VR _50+m-1 In which L vector processing elements store the same data, Wb _sm(i) Completing the k-th column data vectorization;

step 8, inputting the sub-block matrix Ib in the AM space _am(j) Of kth line data Ib _{am(j，k，0)} ......Ib _{am(j，k，n-1)} Loading into p vector registers VR ₀ 、VR ₁ ...VR _p-1 Where p represents the number of functional vector arithmetic unit elements in an architecture for very long data instruction words, with a minimum granularity of one load

A byte, so that it can be loaded at minimum at a time

Half precision data;

step 9, mixing Wb _{sm(i，0，k)} Vectorized data VR ₅₀ Respectively react with Ib _am(j) VR of the kth line ₀ 、VR ₁ ...VR _p-1 Performing multiply-add operation, simultaneously operating L vector processing units in parallel, and storing the calculation result in a vector register VR ₁₀ 、VR ₁₁ ...VR _10+p-1 Performing the following steps;

step 10, register VR based on vector ₅₁ ...VR _50+m-1 Stored is the weight sub-block Wb _{sm(i，1，k)} ......Wb _{sm(i，m-1，k)} Vectorized data, vector register VR ₀ 、VR ₁ ...VR _p-1 Stored in is an input sub-block Ib _am(j) Repeating the step 9 to respectively combine each group of quantized data of the weight with the Ib _am(j) And adds the multiplication result to the vector register VR _10+p 、VR _10+p+1 ...VR _10+m×p-1 In the process, L vector processing elements operate in parallel simultaneously, traversing Wb _sm(i) Up to Wb _sm(i) K column of (1) and Ib _am(j) The multiplication and addition calculation of the k rows is completed;

step 11, judging whether K +1 is smaller than K, if so, skipping to execute step 19, and if not, continuing to execute step 12;

step 12, based on scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 R [16:31]Wb stored in _{sm(i，1，k+1)} ......Wb _{sm(i，m-1，k+1)} Data, to scalar register R ₃₀ 、R ₃₁ ...R _30+m-1 And performing high bit expansion operation, and enabling 16 high bits data R [16:31]Replication extension to d-bit data storage in scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 In (d) is the bit length of a scalar register;

step 13, based on scalar register R ₄₀ 、R ₄₁ ...R _40+m-1 Stored replicated extended data, for scalar registers R ₄₀ 、R ₄₁ ...R _40+m-1 Broadcast operation is carried out in sequence, and the broadcasted data is stored in a vector register VR ₅₀ 、VR ₅₁ ...VR _50+m-1 In which L vector processing elements store the same data, Wb _sm(i) Completing the vectorization of the k +1 th column of data;

step 14, inputting the sub-block matrix Ib in the AM space _am(j) Data Ib of the k +1 th line _{am(j，k+1，0)} ......Ib _{am(j，k+1，n-1)} Loading into p vector registers VR ₀ 、VR ₁ ...VR _p-1 Where p represents the number of functional vector arithmetic unit elements in an architecture for very long data instruction words, with a minimum granularity of one load

One byte, so that it can be loaded at least once

Half precision data;

step 15, mixing Wb _{sm(i，0，k+1)} Vectorized data VR ₅₀ Respectively react with Ib _am(j) The (k + 1) th row of data VR ₀ 、VR ₁ ...VR _p-1 Performing multiply-add operation while L vector processing units operate in parallel, storing the calculation result in vector register VR ₁₀ 、VR ₁₁ ...VR _10+p-1 Performing the following steps;

step 16, vector register VR based ₅₁ ...VR _50+m-1 Stored is the weight sub-block Wb _{sm(i，1，k+1)} ......Wb _{sm(i，m-1，k+1)} Vectorized data, vector register VR ₀ 、VR ₁ ...VR _p-1 Stored in is an input sub-block Ib _am(j) Repeating the step 15 for the (k + 1) th line of data, and respectively connecting each group of quantized data of the weight values with Ib _am(j) And adds the multiplication result to the vector register VR _10+p 、VR _10+p+1 ...VR _10+m×p-1 In this process, L vector processing elements operate in parallel, traversing Wb, simultaneously _sm(i) Up to Wb _sm(i) Column k +1 and Ib _am(j) The multiplication and addition calculation of the k +1 line is completed;

step 17, making k equal to k + 2;

step 18, judging whether K is smaller than K, if so, returning to the step 5, and if not, executing the step 19;

step 19, store in vector register VR ₁₀ 、VR ₁₁ ...VR _10+m×p-1 Temporarily storing the data result in the AM space position AM _temp ；

Step 20, calling direct memory access operation, and AM the spatial position of AM _temp Storing the stored characteristic diagram data result to the appointed position of the double-rate synchronous dynamic random access memory;

step 21, making j equal to j + 1;

step 22, judging whether j is less than x ₂ If yes, calling direct memory access operation and inputting the subblock matrix Ib _am(j) Loading the data into an on-chip AM space, returning to the step 3 after the loading is finished, and if not, executing the step 23;

step 23, making i equal to i + 1;

step 24, judging whether i is less than x ₁ If yes, calling direct memory access operation and making weight value sub-block matrix Wb _sm(i) Loading into the SM space on the chip, returning to the step 2 after loading, and if not, obtaining all weight data W _ddr And input data I _ddr The conv1 × 1 calculation of (c) is completed.

Images