WO2020258568A1 - 基于卷积神经网络的数据处理方法和装置 - Google Patents

基于卷积神经网络的数据处理方法和装置 Download PDF

Info

Publication number
WO2020258568A1
WO2020258568A1 PCT/CN2019/108928 CN2019108928W WO2020258568A1 WO 2020258568 A1 WO2020258568 A1 WO 2020258568A1 CN 2019108928 W CN2019108928 W CN 2019108928W WO 2020258568 A1 WO2020258568 A1 WO 2020258568A1
Authority
WO
WIPO (PCT)
Prior art keywords
convolution
matrix
output
layer
convolutional layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/108928
Other languages
English (en)
French (fr)
Inventor
梅国强
郝锐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Wave Intelligent Technology Co Ltd
Original Assignee
Suzhou Wave Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Wave Intelligent Technology Co Ltd filed Critical Suzhou Wave Intelligent Technology Co Ltd
Priority to US17/281,267 priority Critical patent/US20220004840A1/en
Publication of WO2020258568A1 publication Critical patent/WO2020258568A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/15—Correlation function computation including computation of convolution operations
    • G06F17/153—Multidimensional correlation or convolution
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/15—Correlation function computation including computation of convolution operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]

Definitions

  • the present invention relates to deep learning technology, in particular to a data processing method and device based on convolutional neural network.
  • convolutional neural networks have been widely used in many areas of life. For example, using convolutional neural networks to process video data, audio data or image data can automatically detect similar videos , Similar audio or similar images.
  • Convolutional neural networks generally include multiple convolutional layers, and each convolutional layer includes multiple convolution kernels.
  • one convolutional layer is generally used first
  • the convolution kernel calculates multiple corresponding convolution values according to the input of the convolution layer.
  • the convolution value obtained each time is stored in the storage device. After calculating all the convolution values, it is calculated according to the stored convolution value Get the output of this convolutional layer. Therefore, the existing method requires frequent reading and writing operations on the storage device during operation, resulting in low processing efficiency.
  • the present invention proposes a data processing method and device based on a convolutional neural network to improve data processing efficiency.
  • the first aspect of the present invention discloses a data processing method based on a convolutional neural network, including:
  • any convolutional layer of the convolutional neural network convert the input data of the convolutional layer into a first square matrix; wherein, the first square matrix is an N-order square matrix, and the N is based on The parameter setting of the convolution layer is a positive integer; the input data includes a plurality of input matrices, the first square matrix is divided into a plurality of regions, and the elements included in each region have the same matrix position, so The matrix position of the element refers to the position of the element in its corresponding input matrix;
  • each convolution kernel of the convolution layer use the convolution kernel to calculate each element in the input data to obtain the convolution value of each element in the input data; where In the process of using the convolution kernel to calculate each element in the input data, each time the convolution value of an element is calculated, the convolution value of the element is calculated using the same convolution kernel, And the convolution values of elements belonging to the same area are accumulated to obtain an output element corresponding to one area of the convolution kernel; where the area is each area of the first square matrix;
  • each convolution kernel of the convolution layer For each convolution kernel of the convolution layer, combine the output elements of each region corresponding to the convolution kernel to obtain the calculation result of the convolution kernel; all the convolution kernels of the convolution layer The calculation result of is used as the output of the convolutional layer.
  • the manner of calculating each element in the input data by using all the convolution kernels of the convolution layer includes:
  • the first square matrix is input to multiple multipliers, so that the multiple multipliers simultaneously use their own corresponding convolution kernels to calculate each element of the input data; wherein, all convolution kernels of the convolution layer Are allocated to the multiple multipliers in advance.
  • the convolution value and the convolution value of the element that belong to the same area are calculated by the adder and the convolution kernel is calculated to obtain one of the convolution values.
  • the product core corresponds to an output element of a region, and the output element is stored in a preset register.
  • the calculation result of each convolution kernel of the convolution layer is an output matrix, and the output matrix of all the convolution kernels of the convolution layer is used as the output of the convolution layer;
  • the method further includes:
  • the second square matrix is an N-th order square matrix
  • the second square matrix is divided into a plurality of regions, and each region includes elements All have the same matrix position, and the matrix position of the element refers to the position of the element in its corresponding output matrix.
  • the output elements of each region corresponding to the convolution kernel are combined to obtain the calculation result of the convolution kernel; the convolution
  • the calculation results of all convolution kernels of the layer, after serving as the output of the convolution layer also include:
  • pooling layer to process the output of the convolutional layer to obtain the pooled output of the convolutional layer, and the pooled output of the convolutional layer is used as the next one of the convolutional layer The input data of the convolutional layer.
  • the second aspect of the present invention discloses a data processing device based on a convolutional neural network, including:
  • the conversion unit is used to convert the input data of the convolutional layer into a first square matrix for any one convolutional layer of the convolutional neural network; wherein, the first square matrix is an Nth-order square matrix, The N is a positive integer set according to the parameters of the convolutional layer; the input data includes a plurality of input matrices, the first square matrix is divided into a plurality of regions, and the elements included in each region have the same
  • the matrix position of, the matrix position of the element refers to the position of the element in its corresponding input matrix;
  • the calculation unit is configured to calculate each element in the input data by using the convolution kernel for each convolution kernel of the convolution layer to obtain the convolution of each element in the input data
  • the product kernel is calculated, and the convolution values of elements belonging to the same area are accumulated to obtain an output element corresponding to one area of the convolution kernel; wherein, the area is each area of the first square matrix;
  • the combination unit is configured to combine the output elements of each region corresponding to the convolution kernel for each convolution kernel of the convolution layer to obtain the calculation result of the convolution kernel; the convolution layer The calculation results of all convolution kernels are used as the output of the convolution layer.
  • the calculation unit includes multiple multipliers
  • the calculation unit uses all the convolution kernels of the convolution layer to calculate each element in the input data, including:
  • the multiple multipliers simultaneously use their corresponding convolution kernels to calculate each element of the input data; wherein, the convolution kernels of the convolution layer are pre-allocated to the multiple multipliers.
  • the calculation unit includes an adder and a register
  • the calculation unit is configured to calculate the convolution value of an element every time, and accumulate the convolution value of the element and the convolution value of the elements that are calculated by using the same convolution kernel and belong to the same region, Obtaining an output element corresponding to a region of the convolution kernel includes:
  • the adder Each time the convolution value of an element is calculated, the adder accumulates the convolution value and the convolution value of the elements that are calculated using the same convolution kernel and belong to the same region to obtain one of the convolution values.
  • the convolution kernel corresponds to the output element of a region
  • the register is used to store the output element.
  • the calculation result of each convolution kernel of the convolution layer is an output matrix, and the output matrix of all the convolution kernels of the convolution layer is used as the output of the convolution layer;
  • the conversion unit is also used for:
  • the second square matrix is an N-th order square matrix
  • the second square matrix is divided into a plurality of regions, and each region includes elements All have the same matrix position, and the matrix position of the element refers to the position of the element in its corresponding output matrix.
  • the data processing device further includes:
  • the pooling unit is configured to use the pooling layer to process the output of the convolutional layer to obtain the pooled output of the convolutional layer, and the pooled output of the convolutional layer is used as the The input data of the next convolution layer of the convolution layer.
  • the present invention provides a data processing method and device based on a convolutional neural network.
  • the convolution kernel of the convolutional layer is used to calculate the elements in the input data of the convolutional layer one by one , Get the convolution value of each element, each time a convolution value is calculated, the convolution value is calculated with the same convolution kernel, and the convolution value of the elements belonging to the same area is accumulated to obtain a convolution
  • the core corresponds to the output element of a region.
  • the present invention in the process of calculating the convolution value, each time a convolution value is calculated, the convolution value is added to the corresponding convolution sum, and finally the output of the convolution layer is directly obtained Therefore, the present invention can obtain the output of the convolutional layer after calculating all the convolution values. There is no need to read the convolution value in the storage device for calculation, which effectively reduces the process of calculating the output of the convolutional layer and the storage device. Interaction to improve data processing efficiency.
  • FIG. 1 is a schematic diagram of a model structure of a convolutional neural network provided by an embodiment of the present invention
  • Figure 2a is a schematic diagram of matrix convolution operation
  • Figure 2b is a schematic diagram of a pooling operation on a matrix
  • FIG. 3 is a flowchart of a data processing method based on convolutional neural network provided by an embodiment of the present invention
  • FIG. 4 is a schematic diagram of a data format for storing input data of a convolutional layer of a convolutional neural network and a data format for output of a convolutional layer according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of a data format of multiple convolutional layers of a convolutional neural network according to an embodiment of the present invention
  • FIG. 6 is a device configuration diagram for implementing a data processing method based on a convolutional neural network provided by another embodiment of the present invention.
  • FIG. 7 is a flowchart of a data processing method based on a convolutional neural network according to another embodiment of the present invention.
  • FIG. 8 is a schematic diagram of input information of a data processing method based on a convolutional neural network according to another embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of a data processing device based on a convolutional neural network provided by an embodiment of the present invention.
  • the key difference between the convolutional neural network-based data processing method provided by the present invention and the existing processing method is that the input data of each convolutional layer in the existing data processing method consists of multiple feature matrices ( It can also be considered as a feature map) composition, and the number and size of the feature maps between different convolutional layers are inconsistent; and the data processing method based on convolutional neural network provided by any embodiment of this application is aimed at composing the convolutional neural network
  • Each convolutional layer uniformly converts the input data into a feature matrix (feature map) of the same size, so that each convolutional layer performs data processing according to the same form of input data.
  • the same or similar data storage format can be used for each convolutional layer, and the same or similar processing timing can be used, so it is based on the entire convolutional layer.
  • the process of data processing of neural network there is no need to adjust the data storage format and processing sequence, thus effectively improving the efficiency of data processing.
  • Convolutional neural network is a model that is widely used in the field of deep learning.
  • the trained convolutional neural network is used to process input data, and a certain type of characteristics of the input data can be obtained to analyze the input data.
  • Convolutional neural network mainly includes several convolutional layers, several pooling layers, a fully connected layer, and a probability classification function.
  • the input of the convolutional neural network is first processed by the first convolutional layer, and then the output of the first convolutional layer is input to the first pooling layer corresponding to the first convolutional layer, the first pooling
  • the input data is pooled by the layer to obtain the output of the first pooling layer, and then the output of the first pooling layer is used as the input of the second convolutional layer, and then the second pooling layer.
  • the number of convolutional layers included in a convolutional neural network can be determined according to actual conditions. As described above, the number of pooling layers in a convolutional neural network is equal to the number of convolutional layers.
  • a common convolutional neural network includes 5 convolutional layers, 5 pooling layers, a fully connected layer and a probability classification function.
  • each convolutional layer includes multiple convolution kernels.
  • the first convolutional layer includes 64 convolutional kernels
  • the second convolutional layer includes There are 128 convolution kernels
  • the third convolution layer includes 256 convolution kernels
  • the fourth and fifth convolution layers include 512 convolution kernels respectively.
  • the convolutional layer processes the input data, which is to use its own convolution kernel to perform operations on the input data.
  • the input data and output data of the convolutional layer and the pooling layer can be considered to be composed of one or more matrices (the matrix constituting the input data can be recorded as the input matrix), for example, the volume shown in Figure 1
  • three square 224-order matrices can be used as the input data of this convolutional neural network, or in other words, as the input of the first convolutional layer of this convolutional neural network.
  • the processing of the input data by the first convolutional layer is to use its own 64 convolution kernels to calculate these 3 square matrices of 224 order respectively.
  • the 64 convolution kernels of the first convolution layer are different, and each of the convolution kernels needs to calculate the input data (that is, three square matrices of order 224) to obtain the corresponding convolution kernel Calculation results. That is to say, the convolution kernel 1 of the first convolution layer is used to calculate the input data to obtain the calculation result of the convolution kernel 1, and the convolution kernel 2 of the first convolution layer is also used to calculate the input data. The calculation results of the convolution kernel 2 are obtained, and so on, and finally 64 calculation results are obtained. These 64 calculation results constitute the output of the first convolution layer.
  • the data process of the convolutional layer in other convolutional neural networks is similar to the above process.
  • a convolution kernel includes several coefficient matrices, and the number of coefficient matrices is equal to the number of input matrices constituting the input data.
  • the convolution kernel used for calculation is Including 3 coefficient matrices; the coefficient matrix is generally a 3rd-order square matrix or a 5th-order square matrix.
  • the order of the coefficient matrix can be increased according to actual needs.
  • the coefficient matrix in a convolution kernel corresponds to the input matrix in the input data one-to-one, that is, in the above three coefficient matrices, each coefficient matrix corresponds to one input matrix.
  • the values of the elements in all coefficient matrices included in a convolution kernel are determined in the process of training the convolutional neural network with sample data.
  • a coefficient matrix performs a convolution operation on an input matrix, which means that the coefficient matrix is used to calculate the convolution value of each element of the input matrix.
  • These convolution values are arranged according to the position of the corresponding element in the input matrix, which constitutes this time The result of the convolution operation.
  • the convolution operation on the input matrix means that the element located in the middle of the coefficient matrix (in Figure 2a, it is the element in the second row and second column of the coefficient matrix), and an element of the input matrix Align (for example, align with the elements in the first row and the first column of the input matrix), so that part or all of the elements of the coefficient matrix correspond to some of the elements in the input matrix (as shown in Figure 2a, which represents 4 elements of the coefficient matrix)
  • each box includes a point representing the element of the input matrix), and then the element of the coefficient matrix is multiplied by the corresponding element to obtain multiple products.
  • the coefficient The element in the second row and second column of the matrix (denoted as X22) is multiplied by the element in the first row and first column of the input matrix (denoted as Y11), the elements X23 and Y12 are multiplied, the elements X32 and Y21 are multiplied, the element X33 and Y22 are multiplied to obtain four products, and these four products are added to obtain a convolution value corresponding to the coefficient matrix of the element in the first row and first column of the input matrix.
  • the obtained convolution value is arranged according to the position of the corresponding element in the input matrix, and the result of this convolution operation is obtained. That is to say, in the result of the above convolution operation (also a matrix), the convolution value calculated from the element in the first row and first column of the input matrix is also the first row and first column in the result of the convolution operation The elements of other convolution values and so on.
  • a convolutional layer includes multiple different convolution kernels, and each convolution kernel needs to use its own coefficient matrix to perform convolution operations on the input matrix. Moreover, for each element in the input data, only the coefficient matrix corresponding to the input matrix where the element is located in a convolution kernel can be used to calculate the element. To sum up, for each element in the input data, this element will calculate multiple convolution values, each of which corresponds to a convolution kernel, and the number of convolution values is equal to the number of convolutional layers. The number of convolution kernels.
  • each coefficient matrix of the convolution kernel and the corresponding input matrix complete the convolution operation, and the corresponding elements in the results of these convolution operations are added to obtain the calculation result of the convolution kernel .
  • a convolution kernel includes 3 coefficient matrices, then after these 3 coefficient matrices have completed the corresponding convolution operation, the results of 3 convolution operations are obtained, that is, 3 matrices, and these 3 matrices are added , The calculation result of this convolution kernel is obtained.
  • the calculation result of a convolution kernel is still a matrix.
  • the set of calculation results of all convolution kernels of this convolution layer is the output of this convolution layer.
  • the pooling operation of the input data by the pooling layer refers to the pooling operation for each input matrix in the input data.
  • the pooling operation of a matrix as shown in Figure 2b, means that a matrix is divided into multiple areas with 2 rows and 2 columns, each area does not overlap each other, and then the element with the largest value in the area is extracted as the The output of the area, and finally the output of each area is arranged according to the corresponding position, which constitutes the result of the pooling operation.
  • the result of the pooling operation of each input matrix in the input data constitutes the output of a pooling layer.
  • the input data of each convolutional layer is a plurality of matrices with different numbers (the matrices included in the input data can also be considered as feature maps), and the number of rows of the matrix It is also very different from the number of columns, which leads to the need to frequently modify the data storage format and corresponding processing timing according to the format of the input data in the data processing process based on the entire convolutional neural network, which further reduces the data processing efficiency.
  • an embodiment of the present application provides a data processing method based on a convolutional neural network. Please refer to FIG. 3.
  • the method includes the following steps:
  • the process of processing the data is basically the same.
  • the data processing method provided in the embodiment of the present application is mainly implemented by improving the processing process of each convolutional layer of the convolutional neural network.
  • the method provided by the present application improves any one convolutional layer, and can be directly applied to other convolutional layers only by adjusting related parameters. Therefore, in the process of introducing the data processing method based on the convolutional neural network provided by the embodiments of the present application, the introduction is based on only one of the convolutional layers.
  • the data processing method provided in this embodiment is mainly applied to a convolutional layer including 128 convolution kernels.
  • the input data of this convolutional layer includes 64 input matrices, and each input matrix is a 112-order square matrix corresponding to Yes, combined with the foregoing introduction to the convolutional neural network, each convolution kernel of this convolution layer includes 64 coefficient matrices, and each coefficient matrix is a 3rd-order square matrix.
  • the first square matrix is an N-order square matrix
  • N is a positive integer set according to the parameters of the convolutional layer.
  • the data processing method provided by the embodiment of the application uniformly converts the input data into a first square matrix with the number of rows and columns fixed to N for each convolutional layer, which is equivalent to the convolutional layer in the prior art.
  • the multiple feature maps of different formats are replaced with a first square matrix of the same size (the first square matrix can also be considered as a feature map with a size of N ⁇ N).
  • the same or similar data storage format can be used for each convolutional layer, and the same or similar processing timing can be used, so it is based on the entire convolutional layer.
  • there is no need to adjust the data storage format and processing sequence thus effectively improving the efficiency of data processing.
  • step S301 the conversion of the input data into the first square matrix in step S301 can be understood as storing each element in the input data in the form of a square matrix.
  • the order N of the first square matrix is mainly determined according to the number of rows of the input matrix of the convolutional layer, the number of columns of the input matrix, and the number of convolution kernels of the convolutional layer.
  • the first square matrix may be set to a square matrix of order 1792.
  • the first square matrix is a square matrix pre-divided into multiple regions, and the number of regions is equal to the number of elements contained in an input matrix, that is, in this embodiment, the first square matrix is divided It is 112 square, 12544 areas. Specifically, if the first square matrix is set as a 1792 square matrix, the first square matrix can be divided according to the method shown in FIG. 4.
  • each small box represents a 16 ⁇ 16 square area with 16 rows and 16 columns. It can be found that a row of square areas in the horizontal direction includes 1792 divided by 16, which is 112 square areas and one column in the vertical direction. The square area also includes 112 areas, that is, the entire first square matrix includes 12544 square areas.
  • the process of converting the input data into the first square matrix of order 1792 shown in Figure 4 includes:
  • the 64 input matrices included in the input data are numbered from 1 to 64. There is no limit to which number is assigned to which input matrix. Just make sure that the 64 numbers are allocated, and each input matrix corresponds to a number. OK.
  • the numbers in Table 1 indicate that the elements here belong to the input matrix with the corresponding number, that is, in the first square area, the first row from left to right is the first element of the two input matrices 1, two Input the first element of matrix 2, the first element of two input matrices 3, and so on, until the first element of the two input matrices 8, the second row is exactly the same as the first row, and the third row Then, starting from the input matrix 9, fill in the form of the previous row until the first element of the input matrix 16, and the fourth row is exactly the same as the third row. It is equivalent to the first square area including the first element of 64 input matrix, and each element is copied into 4 copies.
  • the foregoing Table 1 is only one filling method.
  • the four duplicate arrangements of the first element of an input matrix can be all arranged in a row.
  • first element refers to the element in the first row and first column of the matrix.
  • the elements in the input matrix are directly arranged in order On the right, the number is in the order of top and bottom. Therefore, for the above input matrix, the second element refers to the element in the first row and second column of the input matrix, and the 113th element refers to the second row and first column of the input matrix.
  • the definition of the position of the area in the first square matrix is similar.
  • the elements in the area are 64 input matrices, each of the i-th element of the input matrix, and each element is copied as 4 parts.
  • step S302 and step S303 are steps that need to be repeated multiple times, and during the execution of step S302, each time the convolution value of an element is calculated, step S303 needs to be executed once, and then step S303 is executed. S302.
  • step S302 the key to step S302 is to point out that each convolution kernel needs to perform the process of calculating each element of the input data as described in step S302, and it does not limit that only one convolution kernel can be used for calculation at a time. Specifically, in the method provided in the embodiment of the present application, according to the number of copies of the same element in the input data in the first square matrix and the number of multipliers used to calculate the convolution value, multiple A convolution kernel performs the calculation described in step S302.
  • four convolution kernels can be used to simultaneously calculate 4 copies of the first element of the input matrix 1 stored in the first square matrix to obtain four different convolution values, and then execute Step S303, after the execution of step S303, four convolution kernels are used to simultaneously calculate 4 copies of the first element of the input matrix 2 stored in the first square matrix to obtain four convolution values before performing step S303. And so on.
  • using the convolution kernel to calculate the convolution value of the elements in the input data is actually using the coefficient matrix in this convolution kernel corresponding to the input matrix to which the calculated element belongs to perform the convolution operation.
  • step S302 the element used for calculation is read from the first square matrix, but when calculating the convolution value of this element, the element in the input matrix to which this element belongs is still used instead of directly adding the coefficient matrix
  • the elements in and the elements in the first square correspond one-to-one.
  • step S302 to calculate the convolution value of an element, it is first necessary to determine the coefficient matrix corresponding to the input matrix to which the element belongs from the convolution kernel used for calculation, and then combine the center element of the coefficient matrix with the The calculated element corresponds, and then follow the convolution operation process described above, read the other elements of the input matrix to which the calculated element belongs from the first square matrix stored in the memory, and then proceed according to the aforementioned convolution operation process Calculation.
  • the convolution kernel A when calculating the convolution value of this element using the convolution kernel (assumed as the convolution kernel A), first find the convolution kernel The coefficient matrix corresponding to the input matrix 1 in A, and then use the center element of this coefficient matrix to correspond to the first element of the input matrix 1.
  • the second element of the coefficient matrix The element in the third row and the third column (element X23) should correspond to the element in the first row and second column of the input matrix 1 (element Y12), the element X32 corresponds to the element Y21, and the element X33 corresponds to the element Y22, so from the first square matrix Read the above three elements of input matrix 1, and then use the read three elements, the aforementioned calculated element (that is, the first element of input matrix 1) and the coefficient matrix corresponding to input matrix 1, to calculate input matrix 1
  • the convolution value of the first element of, and, as described above, this convolution value is the convolution value corresponding to the convolution kernel A.
  • step S303 is described below in conjunction with the foregoing example:
  • convolution kernel A Take a convolution kernel (denoted as convolution kernel A) as an example.
  • convolution kernel A reads the first element of input matrix 1 from the first square matrix, and then calculates the element At this time, the convolution value of other elements has not been calculated, so the accumulation in step S303 is to save the convolution value of the first element of the input matrix 1 calculated by the convolution kernel A;
  • step S302 is executed again, the elements of input matrix 2 are read from the first square area of the first square matrix, the convolution value of the elements of input matrix 2 is calculated, and then step S303 is executed.
  • step S303 confirm this The convolution value obtained in the second calculation, and the convolution value obtained in the previous calculation, belong to the same square area of the first square matrix (both are located in the first square area), and are calculated using the convolution kernel A Convolution value, so add this convolution value and the previous convolution value to get a convolution sum, and delete the two convolution values calculated;
  • step In S303 the convolution value that satisfies the above conditions is accumulated to the aforementioned convolution sum, the accumulated convolution sum is retained, and the original convolution sum and convolution value are deleted, and so on, until the first square matrix is the first one
  • the 64 different elements that belong to the 64 input matrices are stored in the square area.
  • the convolution sum obtained by the accumulation of the 64 convolution values is the convolution kernel.
  • A is the output element calculated for the first square area of the first square matrix.
  • step S303 the accumulation process described in step S303 can be implemented by using an adder, and the convolution kernel obtained by accumulation can be stored in a register.
  • the output element of the convolution kernel for each square region of the first square matrix can be obtained.
  • the convolution layer includes 128 convolution kernels, and there are 112 ⁇ 112 square regions in the first square matrix, so 112 ⁇ 112 ⁇ 128 output elements can be obtained in the end.
  • the calculation result of the convolution kernel is also a matrix.
  • 112 ⁇ 112 output elements can be calculated finally, and each output element corresponds to a square area of the first square matrix. These output elements are calculated according to their corresponding square areas.
  • the matrix formed by the positions in the first square matrix is the calculation result of this convolution kernel.
  • the first element is the output element of the convolution kernel A corresponding to the first square area of the first square matrix
  • the second element is the output element corresponding to the first square matrix.
  • the output elements of the two square areas, and so on, can be combined to obtain the calculation result of the convolution kernel A. It can be seen that as a matrix, the number and arrangement of the elements of the calculation result of the convolution kernel are consistent with the number and arrangement of the square regions in the first square matrix. Therefore, the calculation result of a convolution kernel is one 112 ⁇ 112 square matrix.
  • the convolutional layer includes 128 convolution kernels, so the corresponding calculation results of 128 convolution kernels, that is to say, 128 square matrices of 112 order calculated by 128 convolution kernels are this The output of the convolutional layer.
  • the method provided in this embodiment further includes the following steps:
  • the second square matrix is an N-order square matrix, the second square matrix is divided into multiple regions, each region includes the elements have the same matrix position, the matrix position of the element, refers to the element in its corresponding output matrix In the location.
  • the output of the convolutional layer in this embodiment is 128 square matrices of order 112, and the process of converting these square matrices into the second square matrix is similar to the aforementioned converting the input data of the convolutional layer into the first square matrix. I won't repeat them here.
  • the second square matrix obtained by converting 128 square matrices of order 112 is still divided into square areas with 16 rows and 16 columns.
  • the first square area of the second square matrix The first element of 128 output matrices is filled in, the second element of 128 output matrices is filled in the second square area, and so on.
  • a square area includes 128 output elements, so each output element in the second square matrix is copied only once.
  • the first row of the first square area is filled with the first element of the 8 output matrices, each element is copied into two copies, and the second row is filled with the first element of the other 8 output matrices.
  • Each element is copied into two copies. And so on for other regions.
  • the data processing method based on the convolutional neural network provided by the embodiment of this application, when calculating the output element of the convolution kernel, based on the process of step S302 and step S303, each calculation obtains a convolution Value, the convolution value is calculated with the same convolution kernel, and the convolution value of the elements belonging to the same area of the first square matrix is accumulated, and the accumulation result is stored.
  • the embodiment of the present application The provided data processing method does not need to save each convolution value obtained by calculation in the process of processing the input data of a convolutional layer to obtain the output, which can effectively reduce the occupied storage space.
  • multiple convolution kernels can be used to read the input data at the same time.
  • multiple convolution kernels can calculate an element of the input data at the same time to obtain the effect of multiple convolution values, which effectively improves the data processing speed.
  • the foregoing embodiment uses one convolutional layer of a convolutional neural network as an example to introduce the data processing method provided in this application, and those skilled in the art can directly extend the foregoing method to all convolutional layers of a convolutional neural network.
  • the method for one convolutional layer in the foregoing embodiment is extended to the entire convolutional neural network.
  • each 8 ⁇ 8 square area is filled with 3 elements at the corresponding positions in the square matrix of 224 steps.
  • the first elements of three 224-order square matrices can be denoted as R1, G1, and B1 respectively.
  • the three elements are combined into ⁇ R1G1B1 ⁇ , and ⁇ R1G1B1 ⁇ is copied into 64 copies as the 64 elements in the first 8 ⁇ 8 square area.
  • the first convolutional layer use the data processing method provided in the previous embodiment to process the input data of the first convolutional layer to obtain the output of the first convolutional layer, considering that the first convolutional layer includes 64 Convolution kernels, and in a convolution layer, the number of rows and columns of the calculation result of the convolution kernel is the same as the number of rows and columns of the input matrix. Therefore, the output of the first convolution layer is 224 ⁇ 224 ⁇ 64, that is, 64 square matrices of order 224, the output of the first convolutional layer can be converted into the corresponding second square matrix shown in Fig. 5. It can be found that each area in the second square matrix has 8 rows and 8 columns, which can fill 64 elements at the corresponding positions of the output matrix. Therefore, in the second square matrix corresponding to the first convolutional layer, the output elements are not Copy, each output element has only one in the second square matrix.
  • the first pooling layer connected to the first convolutional layer is used to perform a pooling operation on the output of the first convolutional layer to obtain the output of the pooled first convolutional layer, that is, 64 square 112-order matrices.
  • the output of the first convolutional layer after pooling is used as the input of the second convolutional layer.
  • the input of the second convolutional layer is converted into the first square matrix of the second convolutional layer shown in Fig. 5, this The data processing method provided by the application is processed, and the output of the second convolutional layer is obtained, which is 128 square 112-order matrices.
  • the corresponding second matrix of the converted second convolutional layer is shown in FIG. 5.
  • the output of the pooled second convolutional layer is obtained, that is, 128 square matrices of 56 orders are input to the third convolutional layer for processing.
  • the first square matrix obtained by the third convolutional layer conversion according to the input data includes 56 ⁇ 56 square areas, and each square area is 32 ⁇ 32.
  • each square area is filled with 128 elements at corresponding positions in the input matrix, and each element is copied into 8 copies.
  • the first row is filled with the first elements of the four input matrices, and each element is copied into 8 copies, that is, the first 32 ⁇ 32 square area
  • the first 8 elements of the first row of are the first elements of the same input matrix
  • the 9th to 16th elements are the first elements of another input matrix, and so on.
  • the output obtained by the third convolutional layer is 256 56-order square matrices
  • the second matrix corresponding to the third convolutional layer is still divided into a plurality of 32 ⁇ 32 square regions, where the elements in the output of the third convolutional layer are copied into 4 copies in the corresponding second matrix. That is, in the second matrix, the first 4 elements of the first row of the first 32 ⁇ 32 square area are the first elements of the same output matrix, and the 5th to 8th elements are another output The first element of the matrix, and so on.
  • the data processing method provided by the embodiment of the present application mainly improves the processing process of each convolutional layer, and the processing process of the fully connected layer and the probability classification function can refer to the prior art and will not be described in detail.
  • the area division method of the first square matrix and the second square matrix of each convolution layer is based on The number of rows and columns of the input matrix of the corresponding convolutional layer is determined. Assuming that the number of rows and columns of the input matrix of a convolutional layer are both a, then the corresponding first square matrix is divided into multiple b ⁇ b square regions, where b is equal to 1792 divided by a. Similarly, this The second square matrix corresponding to the convolutional layer is also divided into a plurality of b ⁇ b square regions.
  • the copy score is determined according to the number of input matrices and the number of convolution kernels in the convolutional layer. For example, for the first square matrix divided into multiple b ⁇ b square regions, if you enter The number of input matrices included in the data is c, then each element in the input data is copied into d parts, d is equal to the square of b divided by c.
  • each output element composing the output of the convolutional layer is copied into f parts in the second square matrix, and f is equal to the square of b divided by e.
  • a, b, c, d, e and f are all positive integers.
  • calculating the convolution value of each element in the input data of the convolution layer can be implemented by a multiplier, and the accumulation of the convolution value can be implemented by combining an adder and a register.
  • An optional way to configure the device can be, for a convolutional layer, if in the first square matrix of the convolutional layer, each element of the input data is copied as d parts, and the convolution of this convolutional layer
  • the number of cores is e, so for the data processing process of this convolutional layer, the number of multipliers that can be configured is g, g is equal to e divided by d, and g is a positive integer.
  • each multiplier is configured with a corresponding adder and a corresponding register.
  • the input data is 64 input matrices of order 112.
  • the first square matrix of order 1792 is Each element of the input data is copied into 4 copies. Therefore, for this convolutional layer, 128 divided by 4 can be configured, that is, 32 multipliers, and 32 adders and 32 registers are correspondingly configured.
  • FIG. 6 for the connection mode of the device, where the RAM in FIG. 6 represents memory, which is used to store the calculation results of all the convolution kernels of this convolution layer, that is, the output of the convolution layer.
  • the data processing method provided by this application for the above-mentioned convolutional layer processing process includes the following steps:
  • 64 input matrices of order 112 are converted into a first square matrix of order 1792.
  • the convolution kernel of the convolution layer is pre-allocated with multiple multipliers.
  • the elements in the input data are copied into 4 copies in the first square matrix, so that one multiplier runs once, and four convolution kernels can be used to calculate four convolution values at the same time. Therefore, this embodiment
  • the 128 convolution kernels of the convolution layer are equally divided into 32 multipliers, and each multiplier corresponds to four convolution kernels.
  • each multiplier After inputting the first square matrix, 32 multipliers run at the same time. Each time it runs, each multiplier outputs four convolution values corresponding to one element in the input data. Specifically, the 32 multipliers first calculate the first element of the input matrix 1. Therefore, the first multiplier obtains the four convolution values of the first element of the input matrix 1. These four convolution values use the first The four convolution kernels of one multiplier are calculated, and the second multiplier also outputs the four convolution values calculated by the four convolution kernels of the second multiplier, and so on, that is, 32 A multiplier runs once, and a 32 ⁇ 4 element corresponding to an element of the input data can be obtained, a total of 128 convolution values.
  • the first square matrix is input to the multiplier, and the elements of the first square matrix can be input to the multiplier row by row, and the input signal used to input data can refer to FIG. 8.
  • clk represents the clock signal
  • the vsync signal represents the input of a 1792-order square matrix.
  • the input signal of a 1792-order square matrix shows multiple de signals, and each de signal corresponds to one row of the 1792-order square matrix.
  • Step S703 means that after 128 convolution values are calculated, the 128 convolution values are input to the corresponding adder, and each adder receives 4 convolution values among them.
  • the adder After receiving the convolution value, the adder reads the previously stored convolution sum corresponding to these convolution values from the register. For example, a multiplier uses its own four convolution kernels, A, B, C, and D. Calculate the element of an input data stored in the second square area of the first square matrix. To be precise, these four convolution kernels are used to calculate four copies of this element, one convolution kernel corresponds to one copy, and The four convolution values corresponding to this element. After the adder obtains these four convolution values, it reads the four corresponding convolution sums in the register.
  • a multiplier uses its own four convolution kernels, A, B, C, and D. Calculate the element of an input data stored in the second square area of the first square matrix. To be precise, these four convolution kernels are used to calculate four copies of this element, one convolution kernel corresponds to one copy, and The four convolution values corresponding to this element. After the adder obtains these four convolution values, it reads the four corresponding convolution sum
  • step S702 Each time the adder completes an accumulation, it continues to perform step S702, and then calculates an element in the input data to obtain 128 convolution values again.
  • each convolution kernel will be stored in the register.
  • one register corresponds to four convolution kernels, and each convolution kernel corresponds to 112 ⁇ 112 outputs. Element, so 112 ⁇ 112 ⁇ 4 output elements are stored in a register.
  • step S702 and step S703 both execute 112 ⁇ After 112 ⁇ 64 times, all elements in the input data are calculated to obtain corresponding convolution values, and then step S704 can be executed.
  • step S702 and step S703 After the calculation process of step S702 and step S703 is completely completed, the output elements stored in the register are stored in RAM, and the calculation results of all convolution kernels are combined according to the corresponding positions in the RAM, thereby obtaining the output of the convolution layer.
  • the method provided in this embodiment configures a corresponding number of multipliers, adders and registers according to the situation that the input data is copied in the first square matrix and the number of convolution kernels in the convolutional layer, so that multiple multipliers can be Multiple convolution values are calculated at the same time, and multiple adders are used for accumulation at the same time, thereby further improving the efficiency of the data processing method of this embodiment.
  • this embodiment can ensure that for any convolutional layer, only Traverse each element of the input data once to get the output of this convolutional layer, thereby achieving a pipeline processing effect.
  • the device configured for one convolutional layer can be directly used in the data processing process of other convolutional layers of the convolutional neural network, or it can be applied to other volumes after adjusting the number of devices according to the specific applicable convolutional layer. Layered processing.
  • another embodiment of the present application also provides a data processing device based on the convolutional neural network.
  • the device includes the following structure:
  • the conversion unit 901 is configured to convert the input data of the convolutional layer into a first square matrix for any convolutional layer of the convolutional neural network.
  • the first square matrix is an N-order square matrix, and N is a positive integer set according to the parameters of the convolutional layer;
  • the input data includes multiple input matrices, the first square matrix is divided into multiple regions, and each region includes elements All have the same matrix position, and the matrix position of the element refers to the position of the element in its corresponding input matrix.
  • the calculation unit 902 is configured to use the convolution kernel to calculate each element in the input data for each convolution kernel of the convolution layer to obtain the convolution value of each element in the input data, and to use the convolution kernel
  • the convolution value of each element is calculated by the convolution kernel, and the convolution value of the element is calculated with the same convolution kernel and the convolution of the elements belonging to the same area
  • the values are accumulated to obtain the output element of a convolution kernel corresponding to a region.
  • the above-mentioned area is each area of the first square matrix.
  • the combining unit 903 is configured to combine the output elements of each region corresponding to the convolution kernel for each convolution kernel of the convolution layer to obtain the calculation result of the convolution kernel.
  • the calculation unit 902 includes multiple multipliers.
  • the calculation unit 902 uses all the convolution kernels of the convolution layer to calculate each element in the input data, including:
  • Multiple multipliers use their own corresponding convolution kernels to calculate each element of the input data at the same time; among them, the convolution kernels of the convolution layer are pre-allocated to multiple multipliers.
  • the calculation unit 902 includes an adder and a register.
  • the calculation unit 902 is configured to calculate the convolution value of an element and accumulate the convolution value of the element and the convolution value of the element that belongs to the same area calculated by using the same convolution kernel to obtain a convolution kernel.
  • the output elements corresponding to a region include:
  • the adder Each time the convolution value of an element is calculated, the adder accumulates the convolution value and the convolution value of the elements that belong to the same area calculated by the same convolution kernel to obtain the output of a convolution kernel corresponding to one area element.
  • Registers are used to store output elements.
  • the calculation result of each convolution kernel of the convolution layer is an output matrix, and the output matrix of all convolution kernels of the convolution layer is used as the output of the convolution layer.
  • the conversion unit 901 is also used for:
  • the second square matrix is an N-order square matrix
  • the second square matrix is divided into multiple regions, and the elements included in each region have the same matrix position.
  • the matrix position of refers to the position of the element in its corresponding output matrix.
  • the data processing device further includes:
  • the pooling unit 904 is used to process the output of the convolutional layer by using the pooling layer to obtain the pooled output of the convolutional layer, and the pooled output of the convolutional layer is used as the next convolutional layer of the convolutional layer Input data
  • the present invention provides a data processing device based on a convolutional neural network.
  • the calculation unit 902 uses the convolution kernel of the convolutional layer to calculate the input data of the convolutional layer one by one. Element, get the convolution value of each element, each time a convolution value is calculated, the convolution value is calculated with the same convolution kernel, and the convolution value of the elements belonging to the same area is accumulated to obtain a convolution
  • the product core corresponds to the output element of a region.
  • the present invention in the process of calculating the convolution value, each time a convolution value is calculated, the convolution value is added to the corresponding convolution sum, and finally the output of the convolution layer is directly obtained Therefore, the present invention can obtain the output of the convolutional layer after calculating all the convolution values. There is no need to read the convolution value in the storage device for calculation, which effectively reduces the process of calculating the output of the convolutional layer and the storage device. Interaction to improve data processing efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Algebra (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Complex Calculations (AREA)

Abstract

一种基于卷积神经网络的数据处理方法，对于所述卷积神经网络的任意一个卷积层，利用卷积层的卷积核逐个计算卷积层的输入数据中的元素，得到每个元素的卷积值，每计算得到一个卷积值，将卷积值与利用同一个卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个卷积核对应一个区域的输出元素。上述数据处理方法，在计算卷积值的过程中，每计算得到一个卷积值，就将这个卷积值累加至对应的卷积和中，最后直接得到卷积层的输出中的元素，因此计算完所有卷积值就可以得到卷积层的输出，不必再读取存储设备中的卷积值进行计算，有效的减少了计算卷积层的输出的过程中与存储设备的交互，提高数据处理效率。

Description

基于卷积神经网络的数据处理方法和装置
本申请要求于2019年06月28日提交中国专利局、申请号为201910580367.8、发明名称为“基于卷积神经网络的数据处理方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及深度学习技术，特别涉及一种基于卷积神经网络的数据处理方法和装置。
背景技术
随着深度学习技术的发展，卷积神经网络目前已经被广泛应用于生活中的多个领域，例如，利用卷积神经网络处理视频数据，音频数据或图像数据等，可以自动检测出相似的视频，相似的音频或相似的图像。
卷积神经网络一般包括多个卷积层，并且每一个卷积层均包括多个卷积核，现有的基于卷积神经网络的数据处理方法中，针对一个卷积层，一般是先利用卷积核根据该卷积层的输入计算多个对应的卷积值，每次计算得到的卷积值均被保存在存储设备中，计算所有卷积值后，在根据存储的卷积值计算得到这个卷积层的输出。因此，现有的方法在运行过程中需要频繁的对存储设备进行读操作和写操作，导致处理效率较低。
发明内容
基于上述现有技术的不足，本发明提出一种基于卷积神经网络的数据处理方法和装置，以提高数据处理效率。
本发明第一方面公开一种基于卷积神经网络的数据处理方法，包括：
对于所述卷积神经网络的任意一个卷积层，将所述卷积层的输入数据转换成第一方阵；其中，所述第一方阵是一个N阶方阵，所述N是根据所述卷积层的参数设置的正整数；所述输入数据包括多个输入矩阵，所述第一方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输入矩阵中的位置；
针对所述卷积层的每一个卷积核，利用所述卷积核计算所述输入数据中的 每一个元素，得到所述输入数据中的每一个所述元素的卷积值；其中，在利用所述卷积核计算所述输入数据中的每一个元素的过程中，每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素；其中，所述区域为所述第一方阵的每一个区域；
针对所述卷积层的每一个卷积核，将所述卷积核对应的每一个区域的输出元素进行组合，得到所述卷积核的计算结果；所述卷积层的所有卷积核的计算结果，作为所述卷积层的输出。
可选的，利用所述卷积层的所有卷积核计算所述输入数据中的每一个元素的方式，包括：
将所述第一方阵输入多个乘法器，使所述多个乘法器同时利用自身对应的卷积核计算所述输入数据的每一个元素；其中，所述卷积层的所有卷积核被预先分配给所述多个乘法器。
可选的，所述每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素，包括：
每计算得到一个元素的卷积值，利用加法器将所述卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素，所述输出元素保存在预设的寄存器中。
可选的，所述卷积层的每一个卷积核的计算结果均为一个输出矩阵，所述卷积层的所有卷积核的输出矩阵作为所述卷积层的输出；
其中，所述针对所述卷积层的每一个卷积核，将利用所述卷积核计算得到的所有输出元素组合成所述卷积核的计算结果之后，还包括：
将所述卷积层的输出转换成第二方阵；其中，所述第二方阵是一个N阶方阵，所述第二方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输出矩阵中的位置。
可选的，所述针对所述卷积层的每一个卷积核，将所述卷积核对应的每一个区域的输出元素进行组合，得到所述卷积核的计算结果；所述卷积层的所有 卷积核的计算结果，作为所述卷积层的输出之后，还包括：
利用所述池化层对所述卷积层的输出进行处理，得到所述卷积层的池化后的输出，所述卷积层的池化后的输出作为所述卷积层的下一个卷积层的输入数据。
本发明第二方面公开一种基于卷积神经网络的数据处理装置，包括：
转换单元，用于对于所述卷积神经网络的任意一个卷积层，将所述卷积层的输入数据转换成第一方阵；其中，所述第一方阵是一个N阶方阵，所述N是根据所述卷积层的参数设置的正整数；所述输入数据包括多个输入矩阵，所述第一方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输入矩阵中的位置；
计算单元，用于针对所述卷积层的每一个卷积核，利用所述卷积核计算所述输入数据中的每一个元素，得到所述输入数据中的每一个所述元素的卷积值；其中，在利用所述卷积核计算所述输入数据中的每一个元素的过程中，每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素；其中，所述区域为所述第一方阵的每一个区域；
组合单元，用于针对所述卷积层的每一个卷积核，将所述卷积核对应的每一个区域的输出元素进行组合，得到所述卷积核的计算结果；所述卷积层的所有卷积核的计算结果，作为所述卷积层的输出。
可选的，所述计算单元包括多个乘法器；
所述计算单元利用所述卷积层的所有卷积核计算所述输入数据中的每一个元素，包括：
所述多个乘法器同时利用自身对应的卷积核计算所述输入数据的每一个元素；其中，所述卷积层的卷积核被预先分配给所述多个乘法器。
可选的，所述计算单元包括加法器和寄存器；
所述计算单元用于每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素，包括：
每计算得到一个元素的卷积值，所述加法器将所述卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素；
所述寄存器用于保存所述输出元素。
可选的，所述卷积层的每一个卷积核的计算结果均为一个输出矩阵，所述卷积层的所有卷积核的输出矩阵作为所述卷积层的输出；
其中，所述转换单元还用于：
将所述卷积层的输出转换成第二方阵；其中，所述第二方阵是一个N阶方阵，所述第二方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输出矩阵中的位置。
可选的，所述数据处理装置，还包括：
池化单元，用于利用所述池化层对所述卷积层的输出进行处理，得到所述卷积层的池化后的输出，所述卷积层的池化后的输出作为所述卷积层的下一个卷积层的输入数据。
本发明提供一种基于卷积神经网络的数据处理方法和装置，对于所述卷积神经网络的任意一个卷积层，利用卷积层的卷积核逐个计算卷积层的输入数据中的元素，得到每个元素的卷积值，每计算得到一个卷积值，将卷积值与利用同一个卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个卷积核对应一个区域的输出元素。本发明提供的数据处理方法，在计算卷积值的过程中，每计算得到一个卷积值，就将这个卷积值累加至对应的卷积和中，最后直接得到卷积层的输出中的元素，因此本发明计算完所有卷积值就可以得到卷积层的输出，不必再读取存储设备中的卷积值进行计算，有效的减少了计算卷积层的输出的过程中与存储设备的交互，提高数据处理效率。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本发明实施例提供的一种卷积神经网络的模型结构示意图；
图2a为矩阵的卷积运算示意图；
图2b为对矩阵进行池化操作的示意图；
图3为本发明实施例提供的一种基于卷积神经网络的数据处理方法的流程图；
图4为本发明实施例提供的一种存储卷积神经网络的卷积层的输入数据的数据格式和卷积层的输出的数据格式示意图；
图5为本发明实施例提供的一种卷积神经网络的多个卷积层的数据格式示意图；
图6为用于实现本发明另一实施例提供的一种基于卷积神经网络的数据处理方法的器件配置图；
图7为本发明另一实施例提供的一种基于卷积神经网络的数据处理方法的流程图；
图8为本发明另一实施例提供的一种基于卷积神经网络的数据处理方法的输入信息示意图；
图9为本发明实施例提供的一种基于卷积神经网络的数据处理装置的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
首先需要说明，本发明提供的基于卷积神经网络的数据处理方法与现有的处理方法的关键区别在于，现有的数据处理方法中每一个卷积层的输入数据均由多个特征矩阵(也可以认为是特征图)构成，并且不同卷积层之间特征图的数量和大小不一致；而本申请任一实施例提供的基于卷积神经网络的数据处理方法，针对组成卷积神经网络的每一个卷积层，将输入数据统一的转换成一个大小一致的特征矩阵(特征图)，使得每个卷积层均根据同一形式的输入数据 进行数据处理。基于上述转换，在针对各个卷积层的数据处理过程中，可以对每个卷积层使用相同的或相似的数据存储格式，并且可以使用相同的或者相似的处理时序，因此在基于整个卷积神经网络的数据处理过程中可以不用调整数据存储格式和处理时序，从而有效的提高数据处理效率。
卷积神经网络是一种在深度学习领域被广泛使用的模型，利用训练好的卷积神经网络处理输入数据，可以获得输入数据的某一类特征，从而对输入数据进行分析。卷积神经网络主要包括由若干个卷积层，若干个池化层，一个全连接层，以及一个概率分类函数。卷积神经网络的输入首先经过第一个卷积层处理，然后第一个卷积层的输出又被输入至与第一个卷积层对应的第一个池化层，第一个池化层对输入数据进行池化操作后得到第一个池化层的输出，紧接着第一个池化层的输出又作为第二个卷积层的输入，然后是第二个池化层，第三个卷积层，等等，以此类推，直至得到全连接层的输入，全连接层的输入经过全连接层处理后得到全连接层的输出，最后再利用概率分类函数对全连接层的输出进行处理，就得到卷积神经网络的输出，也就是输入数据的特征。其中，一个卷积神经网络包括的卷积层的数量可以根据实际情况确定，如前文所述，卷积神经网络中池化层的数量等于卷积层的数量。
如图1所示，一种常见的卷积神经网络包括5个卷积层，5个池化层，一个全连接层和一个概率分类函数。其中，每个卷积层均包括多个卷积核，具体的，在图1所示的常用的卷积神经网络中，第一卷积层包括64个卷积核，第二卷积层包括128个卷积核，第三卷积层包括256个卷积核，第四卷积层和第五卷积层分别包括512个卷积核。卷积层对输入数据进行处理，就是利用自身包括的卷积核对输入数据进行运算。
卷积神经网络中，卷积层和池化层的输入数据和输出数据均可以认为由一个或多个矩阵构成(构成输入数据的矩阵可以记为输入矩阵)，例如，图1所示的卷积神经网络，就可以用3个224阶的方阵作为这个卷积神经网络的输入数据，或者说，作为这个卷积神经网络的第一卷积层的输入。第一卷积层对输入数据的处理，就是利用自身的64个卷积核分别对这3个224阶的方阵进行计算。
具体的，第一卷积层的64个卷积核各不相同，其中的每一个卷积核均需要对输入数据(即三个224阶的方阵)进行计算，得到这个卷积核对应的计算结果。也就是说，第一卷积层的卷积核1用于对输入数据进行计算，得到卷积核1的计算结果，第一卷积层的卷积核2也用于对输入数据进行计算，得到卷积核2的计算结果，以此类推，最后得到64个计算结果，这64个计算结果，就构成第一卷积层的输出。其他的卷积神经网络中的卷积层的数据过程与上述过程类似。
其中，卷积核对输入数据的计算，实际是利用这个卷积核自身的系数矩阵对输入矩阵进行卷积运算，然后将卷积运算的结果相加就得到这个卷积核的计算结果。具体的，一个卷积核包括若干个系数矩阵，系数矩阵的数量等于构成输入数据的输入矩阵的数量，例如，对于前述3个224阶方阵构成的输入数据，用于计算的卷积核就包括3个系数矩阵；系数矩阵一般是3阶方阵或5阶方阵，当然也可以根据实际需要增加系数矩阵的阶数。另外，一个卷积核中的系数矩阵与输入数据中的输入矩阵一一对应，即，上述3个系数矩阵中，每个系数矩阵对应一个输入矩阵。一个卷积核包括的所有系数矩阵中的元素的取值，在利用样本数据训练卷积神经网络的过程中确定。
一个系数矩阵对一个输入矩阵进行卷积运算，是指用这个系数矩阵计算输入矩阵的每一个元素的卷积值，这些卷积值按照对应的元素在输入矩阵中的位置排列，就构成本次卷积运算的结果。
参考图2a，对输入矩阵进行卷积运算，就是指，将位于系数矩阵的中间位置的元素(在图2a中，就是系数矩阵的第2行第2列的元素)，与输入矩阵的一个元素对齐(例如，与输入矩阵的第1行第1列的元素对齐)，使得系数矩阵的部分或全部元素与输入矩阵中的部分元素对应(如图2a所示，表示系数矩阵的元素的4个方框内，每个方框均包括一个表示输入矩阵的元素的点)，然后将系数矩阵的元素与对应的元素相乘，得到多个乘积，例如，在图2a所示的情况下，系数矩阵的第2行第2列的元素(记为X22)与输入矩阵的第1行第1列的元素(记为Y11)相乘，元素X23与Y12相乘，元素X32与Y21相乘，元素X33与Y22相乘，得到四个乘积，这四个乘积相加，就得 到输入矩阵的第1行第1列的元素的与这个系数矩阵对应的一个卷积值。以此类推，按上述过程利用这个系数矩阵计算输入矩阵的每一个元素后，得到的卷积值按对应的元素在输入矩阵中的位置排列，就得到本次卷积运算的结果。也就是说，在上述卷积运算的结果(也是一个矩阵)中，输入矩阵的第1行第1列的元素计算得到的卷积值，也是卷积运算的结果中的第1行第1列的元素，其他卷积值以此类推。
需要说明的是，如图2a所示，对输入矩阵进行卷积运算时，可能出现系数矩阵中的部分元素未与输入矩阵的元素对应的情况，这种情况下，只考虑存在对应关系的元素的乘积即可，因此前述计算过程只对4个乘积求和，当然，若系数矩阵的9个元素均对应有输入矩阵的元素，则需要计算9个乘积，并对9个乘积进行求和。
还需要说明的是，前面已经指出，一个卷积层包括多个各不相同的卷积核，每个卷积核都需要用自身的系数矩阵对输入矩阵进行卷积运算。并且，对于输入数据中的每一个元素，一个卷积核中只有与这个元素所在的输入矩阵相对应的系数矩阵可以用于计算该元素。综上所述，对于输入数据中的每一个元素，这个元素会计算得到多个卷积值，其中的每一个卷积值均对应一个卷积核，卷积值的数量等于卷积层包括的卷积核的数量。
对于一个卷积核，这个卷积核的每个系数矩阵都和对应的输入矩阵完成卷积运算后，这些卷积运算的结果中的元素对应的相加，就得到这个卷积核的计算结果。
例如，一个卷积核中包括3个系数矩阵，那么这3个系数矩阵均完成对应的卷积运算后，就得到3个卷积运算的结果，也就是3个矩阵，这3个矩阵相加，就得到这个卷积核的计算结果，当然，一个卷积核的计算结果仍然是一个矩阵。
对于一个卷积层，这个卷积层的所有卷积核的计算结果的集合，就是这个卷积层的输出。
池化层对输入数据的池化操作，就是指对输入数据中的每一个输入矩阵进行池化操作。对一个矩阵的池化操作，如图2b所示，就是指，将一个矩阵分 割为多个2行2列的区域，各个区域互不重叠，然后分别提取该区域内的值最大的元素作为该区域的输出，最后各个区域的输出按对应的位置排列，就构成池化操作的结果。输入数据中每一个输入矩阵的池化操作的结果，就构成一个池化层的输出。
从前面对卷积神经网络的介绍可以发现，利用卷积神经网络处理数据，需要计算得到多个卷积值并计算这些卷积值的和。现有的基于卷积神经网络的数据处理方法中，一般是先计算得到一个卷积层的所有卷积值，然后将这些卷积值对应的组合成多个卷积运算的结果，最后再对这些卷积运算的结果求和得到各个卷积核的计算结果。显然，在计算过程中，每次计算得到的卷积值都需要保存在用于处理数据的计算机的内存中，所有卷积值计算完毕后，又需要从内存中读取这些卷积值用于求和，因此现有的基于卷积神经网络的数据处理方法需要多次对计算机内存进行读写，极大的降低了数据处理效率。
另外，现有的基于卷积神经网络的处理方法中，各个卷积层的输入数据分别是数量不同的多个矩阵(输入数据中包括的矩阵也可以认为是特征图)，并且矩阵的行数和列数也大不相同，导致基于整个卷积神经网络的数据处理过程中需要根据输入数据的格式频繁的修改数据存储格式以及对应的处理时序，进一步降低了数据处理效率。
有鉴于此，本申请实施例提供一种基于卷积神经网络的数据处理方法，请参考图3，该方法包括以下步骤：
首先需要说明的是，卷积神经网络中的各个卷积层，除了用于处理数据的参数的数量和取值不同以外，处理数据的过程基本一致。而本申请实施例提供的数据处理方法，主要是通过改进卷积神经网络的各个卷积层的处理过程实现的。并且，在后续介绍本申请实施例的过程中会发现，本申请提供的方法对任意一个卷积层的改进，只需要调整相关参数，就可以直接应用于其他卷积层。因此，在介绍本申请实施例提供的基于卷积神经网络的数据处理方法的过程中，仅基于其中的一个卷积层进行介绍，根据一个卷积层的处理过程，本领域技术人员能够直接的将针对一个卷积层的数据处理方法扩展至任意一个卷积神经网络的任意一个卷积层，因此，通过结合多个卷积神经网络的多个卷积层 执行本申请提供的数据处理方法得到的实施例，也在本申请要求保护的范围内。
为了方便理解，本实施例基于下述例子进行介绍：
本实施例提供的数据处理方法主要应用于一个包括128个卷积核的卷积层，这个卷积层的输入数据包括64个输入矩阵，每一个输入矩阵均为一个112阶的方阵，对应的，结合前述对卷积神经网络的介绍，这个卷积层的每个卷积核均包括64个系数矩阵，每个系数矩阵都是一个3阶方阵。
S301、对于卷积神经网络的任意一个卷积层，将卷积层的输入数据转换成第一方阵。
其中，第一方阵是一个N阶方阵，N是根据卷积层的参数设置的正整数。
本申请实施例提供的数据处理方法，针对每一个卷积层，将输入数据统一的转换成一个行数和列数均固定为N的第一方阵，相当于将现有技术中卷积层的不同格式的多个特征图装换成一个相同大小的第一方阵(第一方阵也可以认为是一个大小为N×N的特征图)。基于上述转换，在针对各个卷积层的数据处理过程中，可以对每个卷积层使用相同的或相似的数据存储格式，并且可以使用相同的或者相似的处理时序，因此在基于整个卷积神经网络的数据处理过程中可以不用调整数据存储格式和处理时序，从而有效的提高数据处理效率。
需要说明的是，步骤S301所述的将输入数据转换成第一方阵，可以理解为以一个方阵的形式保存输入数据中的每一个元素。
第一方阵的阶数N主要根据卷积层的输入矩阵的行数，输入矩阵的列数，以及卷积层的卷积核的数量确定。可选的，结合前述例子，本实施例中可以将第一方阵设置为一个1792阶的方阵。
需要说明的是，第一方阵是一个预先划分为多个区域的方阵，区域的数量等于一个输入矩阵包含的元素的数量，也就是说，在本实施例中，第一方阵被划分为112的平方，12544个区域。具体的，若第一方阵设置为一个1792阶的方阵，那么第一方阵可以按照图4所示的方法划分。
图4中，每一个小方框表示一个16×16，即16行16列的方形区域，可 以发现，水平方向的一行方形区域包括1792除以16，即112个方形区域，竖直方向的一列方形区域也包括112个区域，也就是整个第一方阵包括12544个方形区域。
将输入数据转换成图4所示的1792阶的第一方阵的过程包括：
首先将输入数据包括的64个输入矩阵编号为1至64号，其中，具体哪一个输入矩阵分配哪一个编号不做限定，只需要确保64个编号恰好分配完，每一个输入矩阵均对应一个编号即可。
从输入矩阵1至输入矩阵64，依次获取每个输入矩阵的第一个元素，将这些元素按下述表1的形式填充在第一方阵的第一个方形区域，即第一行第一列的方形区域中(讨论方形区域的位置时，行和列以方形区域为间隔划分)。
表1
1 1 2 2 …… …… 8 8
1 1 2 2 …… …… 8 8
9 9 …… …… …… …… 16 16
9 9 …… …… …… …… 16 16
…… …… …… …… …… …… …… ……
…… …… …… …… …… …… …… ……
57 57 …… …… 63 63 64 64
57 57 …… …… 63 63 64 64
表1中的数字表示此处的元素属于对应编号的输入矩阵，也就是说，第一个方形区域中，第一行从左到右依次是两个输入矩阵1的第一个元素，两个输入矩阵2的第一个元素，两个输入矩阵3的第一个元素，以此类推，直至两个输入矩阵8的第一个元素为止，第二行与第一行完全相同，第三行则从输入矩阵9开始，按前一行的形式填充，直至输入矩阵16的第一个元素，第四行又与第三行完全相同。相当于第一个方形区域包括的是64个输入矩阵的第一个元素，每个元素均被复制为4份。
可选的，前述表1只是一种填充方式，在另一种填充方式中，可以将一个输入矩阵的第一元素的四个复制排列均排列在一行中。
需要说明的是，上述第一个元素是指矩阵的第一行第一列的元素，为了方便，本申请中涉及元素在输入矩阵中的位置时，直接将输入矩阵中的元素按先左后右，先上后下的顺序编号，因此，对于上述输入矩阵，第2个元素是指输入矩阵的第一行第二列的元素，第113个元素是指输入矩阵的第二行第一列的元素，对第一方阵中区域的位置的定义类似。
以前述第一个方形区域为例，填充第一矩阵的其他方形区域。最后得到的第一矩阵中，对于任意一个方形区域(如，第i个方形区域)，区域内的元素为64个输入矩阵中，每一个输入矩阵的第i个元素，并且每个元素复制为4份。
S302、针对卷积层的每一个卷积核，利用该卷积核计算输入数据中的每一个元素，得到输入数据中的每一个元素的卷积值。
需要说明的是，步骤S302和步骤S303都是需要反复多次执行的步骤，并且，在步骤S302的执行过程中，每计算得到一个元素的卷积值，需要执行一次步骤S303，然后在执行步骤S302。
需要说明的是，步骤S302关键在于指出每一个卷积核都需要执行如步骤S302所述的计算输入数据的每一个元素的过程，而并未限定一次只能用一个卷积核进行计算。具体的，在本申请实施例提供的方法中，根据输入数据中的同一个元素在第一方阵中被复制的份数，以及用于计算卷积值的乘法器的数量，可以同时利用多个卷积核进行步骤S302所述的计算。
在本实施例中，假设只有一个乘法器用于执行步骤S302，结合前文可以发现，输入数据中的每一个元素，在第一方阵中均被复制为4份，因此，本实施例中可以同时利用卷积层的四个卷积核进行前述计算。
也就是说，结合第一方阵，可以利用四个卷积核同时计算第一方阵中存储的输入矩阵1的第一个元素的4份复制，得到四个不同的卷积值，接着执行步骤S303，步骤S303执行完后，又利用四个卷积核同时计算第一方阵中存储的输入矩阵2的第一个元素的4份复制，得到四个卷积值后再执行步骤S303，以此类推。
如前文所述，利用卷积核计算输入数据中的元素的卷积值，实际是利用这 个卷积核中，与被计算的元素所属的输入矩阵相对应的系数矩阵进行卷积运算。
步骤S302中，用于计算的元素是从第一方阵中读取的，但是计算这个元素的卷积值时，使用的仍然是这个元素所属的输入矩阵中的元素，而不是直接将系数矩阵中的元素与第一方阵中的元素一一对应。
具体的，步骤S302中，计算一个元素的卷积值，首先需要从用于计算的卷积核中确定出这个元素所属的输入矩阵对应的系数矩阵，然后将这个系数矩阵的中心的元素与被计算的元素对应，接着按前文介绍的卷积运算的过程，从保存在内存中的第一方阵中，读取计算的这个元素所属的输入矩阵的其他元素，然后按照前述卷积运算过程进行计算。
例如，本实施例中，针对第一方阵存储的输入矩阵1的第一个元素，利用卷积核(假设为卷积核A)计算这个元素的卷积值时，首先查找出卷积核A中与输入矩阵1对应的系数矩阵，然后用这个系数矩阵的中心的元素与输入矩阵1的第一个元素对应，根据前文对卷积运算的介绍，这种情况下这个系数矩阵的第二行第三列的元素(元素X23)应该与输入矩阵1的第一行第二列的元素(元素Y12)对应，元素X32与元素Y21对应，元素X33与元素Y22对应，所以从第一方阵中读取上述三个输入矩阵1的元素，然后利用读取的三个元素，前述被计算元素(即输入矩阵1的第一个元素)以及输入矩阵1对应的系数矩阵，计算得到输入矩阵1的第一个元素的卷积值，并且，如前文所述，这个卷积值是与卷积核A对应的卷积值。
S303、每计算得到一个元素的卷积值，将元素的卷积值与利用同一个卷积核计算得到，且属于同一个区域的元素的卷积值进行累加。
下面结合前述实例介绍步骤S303的具体实现过程：
以一个卷积核(记为卷积核A)为例，步骤S302第一次执行时，卷积核A从第一方阵中读取输入矩阵1的第一个元素，然后计算得到该元素的卷积值，此时还未计算出其他元素的卷积值，故步骤S303中的累加，就是将卷积核A计算得到的输入矩阵1的第一个元素的卷积值保存；
保存后，步骤S302再次执行，从第一方阵的第一个方形区域中读取输入 矩阵2的元素，计算输入矩阵2的元素的卷积值，然后执行步骤S303，步骤S303中，确认此次计算得到的卷积值，与前一个计算得到的卷积值，属于第一方阵的同一个方形区域(均位于第一个方形区域内)，并且都是利用卷积核A计算得到的卷积值，所以将这个卷积值与前一个卷积值相加，得到一个卷积和，并删除计算得到的两个卷积值；
在后续计算过程中，每次出现利用卷积核A计算得到的卷积值，并且这个卷积值是第一方阵的第一个方形区域内存储的元素的卷积值时，就在步骤S303中将满足以上条件的卷积值累加至前述卷积和，保留累加后的卷积和并删除原有的卷积和以及卷积值，以此类推，直至第一方阵的第一个方形区域内保存的，分别属于64个输入矩阵的64个不同的元素均利用卷积核A计算得到对应的卷积值后，这64个卷积值累加得到的卷积和，就是卷积核A针对第一方阵的第一个方形区域计算得到的输出元素。
可选的，步骤S303所述的累加过程，可以利用加法器实现，累加得到的卷积核可以保存在寄存器中。
其他卷积核在第一方阵的其他方形区域内的计算过程与上述过程基本一致，不再赘述。
综上所述，通过反复执行步骤S302和步骤S303，最终，对于卷积层的每一个卷积核，都能得到该卷积核对第一方阵的每一个方形区域的输出元素。具体的，本实施例中，卷积层包括128个卷积核，第一方阵中有112×112个方形区域，所以最后能得到112×112×128个输出元素。
S304、针对卷积层的每一个卷积核，将卷积核对应的每一个区域的输出元素进行组合，得到卷积核的计算结果。
卷积层的所有卷积核的计算结果，作为卷积层的输出。
前面已经指出，卷积核的计算结果也是一个矩阵。本实施例中，对于一个卷积核，最后能够计算得到112×112个输出元素，并且每一个输出元素均对应于第一方阵的一个方形区域，将这些输出元素，根据各自对应的方形区域在第一方阵内的位置组合成的矩阵，就是这个卷积核的计算结果。
例如，卷积核A的计算结果中，第一个元素是卷积核A的对应于第一方 阵的第一个方形区域的输出元素，第二个元素是对应于第一方阵的第二个方形区域的输出元素，以此类推，即可组合得到卷积核A的计算结果。可以看出，作为一个矩阵，卷积核的计算结果的元素的数量和排列方式，与第一方阵中各个方形区域的数量和排列方式一致，因此，一个卷积核的计算结果，是一个112×112的方阵。
本实施例中，卷积层包括128个卷积核，因此对应的有128个卷积核的计算结果，也就是说，128个卷积核计算得到的128个112阶的方阵，就是这个卷积层的输出。
可选的，本实施例提供的方法还包括下述步骤：
S305、将卷积层的输出转换成第二方阵。
其中，第二方阵是一个N阶方阵，第二方阵分割为多个区域，每个区域包括的元素均具有相同的矩阵位置，元素的矩阵位置，指代元素在其对应的输出矩阵中的位置。
已知，本实施例中卷积层的输出是128个112阶的方阵，将这些方阵转换成第二方阵的过程与前述将卷积层的输入数据转换成第一方阵类似，此处不再赘述。
需要说明的是，本实施例中，针对128个112阶的方阵转换得到的第二方阵仍然以16行16列的方形区域划分，按照前述方法，第二方阵的第一个方形区域内填充128个输出矩阵的第一个元素，第二个方形区域内填充128个输出矩阵的第二个元素，以此类推。其中，一个方形区域内包括128个输出中的元素，因此，第二方阵中每个输出元素只复制一次。也就是说，第一个方形区域的第一行填充8个输出矩阵的第一个元素，每个元素复制为两份，第二行填充另外8个输出矩阵的第一个元素，同样的，每个元素复制为两份。其他区域以此类推。
从上述计算过程可以看出，本申请实施例提供的基于卷积神经网络的数据处理方法，在计算卷积核的输出元素时，基于步骤S302和步骤S303的过程，每次计算得到一个卷积值，就将这个卷积值与利用同一个卷积核计算得到的，属于第一方阵的同一个区域的元素的卷积值累加，并保存累加结果，通过这种 方式，本申请实施例提供的数据处理方法在处理一个卷积层的输入数据以获得输出的过程中，不必保存计算得到的每一个卷积值，能够有效的减少占用的存储空间。并且，基于这种方式，一个卷积核对输入数据中的每一个元素都进行计算后，就可以直接得到这个卷积核对应的所有输出元素，从而可以直接组合得到该卷积核的计算结果，而不必再次从内存中读取所有的卷积值进行计算，因此能够有效的减少计算卷积层的输出的过程中对内存的访问次数，提高数据处理效率。
进一步的，通过在第一方阵中复制输入数据的同一个元素，使得从第一方阵中读取用于计算的输入数据的元素时，能够利用多个卷积核同时读取输入数据的同一个元素，达到多个卷积核同时对输入数据的一个元素进行计算，得到多个卷积值的效果，有效提高数据处理速度。
前述实施例以一个卷积神经网络的一个卷积层为例介绍了本申请提供的数据处理方法，本领域技术人员能够将上述方法直接扩展至一个卷积神经网络的所有卷积层。下面以前述图1所示的卷积神经网络为例，介绍前述实施例中针对一个卷积层的方法扩展至整个卷积神经网络后的方法。
首先，获取第一卷积层的输入数据，假设为3个224阶的方阵，然后将这3个224阶的方阵按前述实施例的方法转换成1792阶的第一卷积层的第一方阵，其形式如图5所示，每个8×8的方形区域内均填充有3个224阶的方阵中的对应位置的元素。具体的，以第一个8×8的方形区域为例，可以将3个224阶的方阵的第一个元素分别记为R1，G1，B1，三个元素组合成{R1G1B1}，将{R1G1B1}复制为64份作为第一个8×8的方形区域内的64个元素。
然后对第一个卷积层，利用前一实施例提供的数据处理方法处理第一个卷积层的输入数据，得到第一个卷积层的输出，考虑到第一个卷积层包括64个卷积核，并且一个卷积层中，卷积核的计算结果的行数和列数与输入矩阵的行数和列数一致，因此，第一个卷积层的输出是224×224×64，即64个224阶的方阵，第一个卷积层的输出可以转换成图5所示的对应的第二方阵。可以发现，第二方阵中的每个区域为8行8列，恰好能填充64个输出矩阵的对应位置的元素，因此，第一卷积层对应的第二方阵中，输出元素并未复制，每个输 出元素在第二方阵中均只有一个。
利用与第一卷积层连接的第一池化层对第一卷积层的输出进行池化操作，得到池化后的第一卷积层的输出，也就是64个112阶的方阵。池化后的第一卷积层的输出有作为第二卷积层的输入，第二卷积层的输入转换为图5所示的第二卷积层的第一方阵后，又利用本申请提供的数据处理方法进行处理，得到第二卷积层的输出，也就是128个112阶的方阵。转换得到的第二卷积层的对应的第二矩阵如图5所示。
第二卷积层的输出经过第二池化层后，得到池化后的第二卷积层的输出，即128个56阶的方阵，又输入至第三卷积层进行处理。第三卷积层根据输入数据转换得到的第一方阵包括56×56个方形区域，每个方形区域为32×32。第三卷积层的第一方阵中，每个方形区域填充有128个输入矩阵中对应位置的元素，并且每个元素被复制为8份。具体的，第一个32×32的方形区域中，第一行填充有四个输入矩阵的第一个元素，每个元素复制为8份，也就是说，第一个32×32的方形区域的第一行的前8个元素均为同一个输入矩阵的第一个元素，第9至第16个元素为另一个输入矩阵的第一个元素，以此类推。
第三卷积层对自身的输入数据基于前述实施例的方法进行处理后，由于第三卷积层包括256个卷积核，因此第三卷积层得到的输出是256个56阶的方阵，第三卷积层对应的第二矩阵仍然被划分为多个32×32的方形区域，其中，第三卷积层的输出中的元素在对应的第二矩阵中被复制为4份，也就是说，在第二矩阵中，第一个32×32的方形区域的第一行的前4个元素均为同一个输出矩阵的第一个元素，第5至第8个元素为另一个输出矩阵的第一个元素，以此类推。
第四卷积层的第一方阵和第二方阵，以及第五卷积层的第一方阵和第二方阵的形式如图5，具体计算过程可参考前文，不再赘述。
另外，本申请实施例提供的数据处理方法主要对每个卷积层的处理过程进行改进，全连接层和概率分类函数的处理过程可参考现有技术，不再详述。
总而言之，在确定每个卷积层对应的第一方阵和第二方阵均为1792阶的方阵的基础上，各个卷积层的第一方阵和第二方阵的区域划分方式根据对应的 卷积层的输入矩阵行数和列数确定。假设一个卷积层的输入矩阵的行数和列数均为a，那么对应的第一方阵就划分为多个b×b的方形区域，其中，b等于1792除以a，同样的，这个卷积层对应的第二方阵也划分为多个b×b的方形区域。确定第一方阵和第二方阵的划分方式后，就可以根据方形区域与元素在输入矩阵或计算结果中的位置之间的对应关系向第一方阵和第二方阵填充元素，其中，涉及对元素进行复制时，复制的分数根据输入矩阵的数量，以及卷积层的卷积核的数量确定，例如，对于划分为多个b×b的方形区域的第一方阵，若输入数据包括的输入矩阵的数量为c，那么输入数据中每个元素均被复制为d份，d等于b的平方除以c，对于划分为多个b×b的方形区域的第二方阵，若对应的卷积层包括的卷积核数量为e，那么组成该卷积层的输出的每一个输出元素，在第二方阵中被复制为f份，f等于b的平方除以e。a，b，c，d，e和f均为正整数。
前文已经提及，对于一个卷积层，计算该卷积层的输入数据中各个元素的卷积值具体可以由乘法器实现，并且，对卷积值的累积可以结合加法器和寄存器实现。
特别的，在针对一个卷积层的处理过程中，可以设置多个乘法器以及配套的加法器和寄存器，通过使上述器件同时工作，可以有效的提高该卷积层的数据处理效率。
一种可选的配置器件的方式可以是，针对一个卷积层，如果这个卷积层的第一方阵中，输入数据的每个元素被复制为d份，并且这个卷积层的卷积核的数量为e，那么针对这个卷积层的数据处理过程，可以配置的乘法器的数量为g，g等于e除以d，并且g为正整数。同时，每一个乘法器，均配置一个对应的加法器和一个对应的寄存器。
结合前述图3所示的实施例中的例子，对于一个包括128个卷积核的卷积层，输入数据为64个112阶的输入矩阵，如前文所述，在1792阶的第一方阵中，输入数据的每一个元素被复制为4份，因此，针对这个卷积层，可以配置128除以4，也就是32个乘法器，同时对应的配置32个加法器和32个寄存器，这些器件的连接方式参考图6，其中，图6的RAM表示内存，用于存储这个 卷积层的所有卷积核的计算结果，即该卷积层的输出。
结合图6所示的器件配置情况，参考图7，本申请提供的数据处理方法针对上述卷积层的处理过程包括下述步骤：
S701、将卷积层的输入数据转换成第一方阵。
本实施例中，就是将64个112阶的输入矩阵转换成一个1792阶的第一方阵。
S702、将第一方阵输入多个乘法器，使多个乘法器同时利用自身对应的卷积核计算输入数据的每一个元素。
其中，卷积层的卷积核被预先分配个多个乘法器。在本实施例中，输入数据中的元素在第一方阵中被复制为4份，使得一个乘法器运行一次，可以同时利用四个卷积核计算得到四个卷积值，因此本实施例中，卷积层的128个卷积核被均分至32个乘法器，每个乘法器对应四个卷积核。
输入第一方阵后，32个乘法器同时运行，每运行一次，每个乘法器就输出输入数据中的一个元素对应的四个卷积值。具体的，32个乘法器首先计算输入矩阵1的第一个元素，于是，第一个乘法器得到输入矩阵1的第一个元素的四个卷积值，这四个卷积值分别利用第一个乘法器的四个卷积核计算得到，第二个乘法器也输出分别利用第二乘法器的四个卷积核计算得到的四个卷积值，以此类推，也就是说，32个乘法器运行一次，就能得到输入数据的一个元素对应的32×4，共128个卷积值。
可选的，第一方阵输入乘法器，可以将第一方阵的元素逐行输入乘法器，用于输入数据的输入信号可以参考图8。其中的clk表示时钟信号，vsync信号表示输入一个1792阶的方阵，其中，一个1792阶的方阵的输入信号图示的多个de信号，每个de信号对应1792阶的方阵的一行。
S703、乘法器每计算一次，得到的卷积值由加法器对应的累加，并将累加的结果保存在对应的寄存器中。
步骤S703是指，计算得到128个卷积值后，这128个卷积值输入对应的加法器，每个加法器接收其中的4个卷积值。
收到卷积值后，加法器从寄存器中读取之前存储的与这些卷积值对应的卷 积和，例如，一个乘法器利用自身的A、B、C和D共四个卷积核，计算第一方阵的第二个方形区域内存储的一个输入数据的元素，准确的说，是利用这四个卷积核分别计算这个元素的四个复制，一个卷积核对应一个复制，得到这个元素对应的四个卷积值。加法器获得这四个卷积值后，读取寄存器中的四个对应的卷积和，这四个卷积和都是根据第一方阵的第二个方形区域中存储的输入数据的元素计算得到的卷积和，并且四个卷积和分别利用A、B、C和D四个卷积核计算得到，然后将这四个卷积值与对应的四个卷积和相加，得到的结果作为新的卷积和，替代原本保存在寄存器中的四个卷积和。
加法器每完成一次累加，就继续执行一次步骤S702，再计算一个输入数据中的元素，再次得到128个卷积值。
通过重复上述过程，输入数据的所有元素均被计算后，寄存器中会保存各个卷积核的输出元素，具体的，一个寄存器对应四个卷积核，每个卷积核对应112×112个输出元素，所以一个寄存器中会保存112×112×4个输出元素。
如前文所述，步骤S702和步骤S703所述的过程，每执行一次，就完成一次对于输入数据的一个元素计算128个卷积值并累加的过程，因此，步骤S702和步骤S703均执行112×112×64次后，输入数据中的所有元素均被计算得到对应的卷积值，然后就可以执行步骤S704。
S704、输入数据中的所有元素均被计算后，卷积核的输出元素在内存中组合为卷积核的计算结果。
步骤S702和步骤S703的计算过程完全结束后，寄存器中保存的输出元素存入RAM，在RAM中按对应的位置组合得到所有卷积核的计算结果，从而得到卷积层的输出。
本实施例提供的方法，根据输入数据在第一方阵中被复制的情况，以及卷积层的卷积核的数量，配置对应数量的乘法器，加法器和寄存器，使得多个乘法器能够同时计算得到多个卷积值，并且同时利用多个加法器进行累加，从而进一步提高了本实施例的数据处理方法的效率。
具体的，通过使乘法器的数量与第一方阵中输入数据被复制的份数的乘积等于卷积层的卷积核的数量，本实施例能够保证，对于任意一个卷积层，只需 要将输入数据的每个元素遍历一次，就可以得到这个卷积层的输出，从而达到一种流水线处理的效果。
当然，前面针对一个卷积层配置的器件，可以直接用于卷积神经网络的其他卷积层的数据处理过程，也可以根据具体适用的卷积层调整器件的数量后，再应用于其他卷积层的处理过程。
结合前述实施例提供的基于卷积神经网络的数据处理方法，本申请另一实施例还提供一种基于卷积神经网络的数据处理装置，如图9所示，该装置包括以下结构：
转换单元901，用于对于卷积神经网络的任意一个卷积层，将卷积层的输入数据转换成第一方阵。
其中，第一方阵是一个N阶方阵，N是根据卷积层的参数设置的正整数；输入数据包括多个输入矩阵，第一方阵分割为多个区域，每个区域包括的元素均具有相同的矩阵位置，元素的矩阵位置，指代元素在其对应的输入矩阵中的位置。
计算单元902，用于针对卷积层的每一个卷积核，利用卷积核计算输入数据中的每一个元素，得到输入数据中的每一个元素的卷积值，并且，用于在利用卷积核计算输入数据中的每一个元素的过程中，每计算得到一个元素的卷积值，将元素的卷积值与利用同一个卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个卷积核对应一个区域的输出元素。
其中，上述区域为第一方阵的每一个区域。
组合单元903，用于针对卷积层的每一个卷积核，将卷积核对应的每一个区域的输出元素进行组合，得到卷积核的计算结果。
卷积层的所有卷积核的计算结果，作为卷积层的输出。
可选的，计算单元902包括多个乘法器。
计算单元902利用卷积层的所有卷积核计算输入数据中的每一个元素，包括：
多个乘法器同时利用自身对应的卷积核计算输入数据的每一个元素；其 中，卷积层的卷积核被预先分配给多个乘法器。
计算单元902包括加法器和寄存器。
计算单元902用于每计算得到一个元素的卷积值，将元素的卷积值与利用同一个卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个卷积核对应一个区域的输出元素，包括：
每计算得到一个元素的卷积值，加法器将卷积值与利用同一个卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个卷积核对应一个区域的输出元素。
寄存器用于保存输出元素。
卷积层的每一个卷积核的计算结果均为一个输出矩阵，卷积层的所有卷积核的输出矩阵作为卷积层的输出。
其中，转换单元901还用于：
将卷积层的输出转换成第二方阵；其中，第二方阵是一个N阶方阵，第二方阵分割为多个区域，每个区域包括的元素均具有相同的矩阵位置，元素的矩阵位置，指代元素在其对应的输出矩阵中的位置。
可选的，数据处理装置，还包括：
池化单元904，用于利用池化层对卷积层的输出进行处理，得到卷积层的池化后的输出，卷积层的池化后的输出作为卷积层的下一个卷积层的输入数据
本发明提供一种基于卷积神经网络的数据处理装置，对于所述卷积神经网络的任意一个卷积层，计算单元902利用卷积层的卷积核逐个计算卷积层的输入数据中的元素，得到每个元素的卷积值，每计算得到一个卷积值，将卷积值与利用同一个卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个卷积核对应一个区域的输出元素。本发明提供的数据处理方法，在计算卷积值的过程中，每计算得到一个卷积值，就将这个卷积值累加至对应的卷积和中，最后直接得到卷积层的输出中的元素，因此本发明计算完所有卷积值就可以得到卷积层的输出，不必再读取存储设备中的卷积值进行计算，有效的减少了计算卷积层的输出的过程中与存储设备的交互，提高数据处理效率。
专业技术人员能够实现或使用本申请。对这些实施例的多种修改对本领域 的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下，在其它实施例中实现。因此，本申请将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (10)

  1. 一种基于卷积神经网络的数据处理方法，其特征在于，包括：
    对于所述卷积神经网络的任意一个卷积层，将所述卷积层的输入数据转换成第一方阵；其中，所述第一方阵是一个N阶方阵，所述N是根据所述卷积层的参数设置的正整数；所述输入数据包括多个输入矩阵，所述第一方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输入矩阵中的位置；
    针对所述卷积层的每一个卷积核，利用所述卷积核计算所述输入数据中的每一个元素，得到所述输入数据中的每一个所述元素的卷积值；其中，在利用所述卷积核计算所述输入数据中的每一个元素的过程中，每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素；其中，所述区域为所述第一方阵的每一个区域；
    针对所述卷积层的每一个卷积核，将所述卷积核对应的每一个区域的输出元素进行组合，得到所述卷积核的计算结果；所述卷积层的所有卷积核的计算结果，作为所述卷积层的输出。
  2. 根据权利要求1所述的数据处理方法，其特征在于，利用所述卷积层的所有卷积核计算所述输入数据中的每一个元素的方式，包括：
    将所述第一方阵输入多个乘法器，使所述多个乘法器同时利用自身对应的卷积核计算所述输入数据的每一个元素；其中，所述卷积层的所有卷积核被预先分配给所述多个乘法器。
  3. 根据权利要求1所述的数据处理方法，其特征在于，所述每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素，包括：
    每计算得到一个元素的卷积值，利用加法器将所述卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素，所述输出元素保存在预设的寄存器中。
  4. 根据权利要求1所述的数据处理方法，其特征在于，所述卷积层的每一个卷积核的计算结果均为一个输出矩阵，所述卷积层的所有卷积核的输出矩阵作为所述卷积层的输出；
    其中，所述针对所述卷积层的每一个卷积核，将利用所述卷积核计算得到的所有输出元素组合成所述卷积核的计算结果之后，还包括：
    将所述卷积层的输出转换成第二方阵；其中，所述第二方阵是一个N阶方阵，所述第二方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输出矩阵中的位置。
  5. 根据权利要求1所述的数据处理方法，其特征在于，所述针对所述卷积层的每一个卷积核，将所述卷积核对应的每一个区域的输出元素进行组合，得到所述卷积核的计算结果；所述卷积层的所有卷积核的计算结果，作为所述卷积层的输出之后，还包括：
    利用所述池化层对所述卷积层的输出进行处理，得到所述卷积层的池化后的输出，所述卷积层的池化后的输出作为所述卷积层的下一个卷积层的输入数据。
  6. 一种基于卷积神经网络的数据处理装置，其特征在于，包括：
    转换单元，用于对于所述卷积神经网络的任意一个卷积层，将所述卷积层的输入数据转换成第一方阵；其中，所述第一方阵是一个N阶方阵，所述N是根据所述卷积层的参数设置的正整数；所述输入数据包括多个输入矩阵，所述第一方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输入矩阵中的位置；
    计算单元，用于针对所述卷积层的每一个卷积核，利用所述卷积核计算所述输入数据中的每一个元素，得到所述输入数据中的每一个所述元素的卷积值；其中，在利用所述卷积核计算所述输入数据中的每一个元素的过程中，每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素；其中，所述区域为所述第一方阵的每一个区域；
    组合单元，用于针对所述卷积层的每一个卷积核，将所述卷积核对应的每 一个区域的输出元素进行组合，得到所述卷积核的计算结果；所述卷积层的所有卷积核的计算结果，作为所述卷积层的输出。
  7. 根据权利要求6所述的数据处理装置，其特征在于，所述计算单元包括多个乘法器；
    所述计算单元利用所述卷积层的所有卷积核计算所述输入数据中的每一个元素，包括：
    所述多个乘法器同时利用自身对应的卷积核计算所述输入数据的每一个元素；其中，所述卷积层的卷积核被预先分配给所述多个乘法器。
  8. 根据权利要求6所述的数据处理装置，其特征在于，所述计算单元包括加法器和寄存器；
    所述计算单元用于每计算得到一个元素的卷积值，将所述元素的卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素，包括：
    每计算得到一个元素的卷积值，所述加法器将所述卷积值与利用同一个所述卷积核计算得到，且属于同一个区域的元素的卷积值进行累加，得到一个所述卷积核对应一个区域的输出元素；
    所述寄存器用于保存所述输出元素。
  9. 根据权利要求6所述的数据处理装置，其特征在于，所述卷积层的每一个卷积核的计算结果均为一个输出矩阵，所述卷积层的所有卷积核的输出矩阵作为所述卷积层的输出；
    其中，所述转换单元还用于：
    将所述卷积层的输出转换成第二方阵；其中，所述第二方阵是一个N阶方阵，所述第二方阵分割为多个区域，每个所述区域包括的元素均具有相同的矩阵位置，所述元素的矩阵位置，指代所述元素在其对应的输出矩阵中的位置。
  10. 根据权利要求6所述的数据处理装置，其特征在于，所述数据处理装置，还包括：
    池化单元，用于利用所述池化层对所述卷积层的输出进行处理，得到所述卷积层的池化后的输出，所述卷积层的池化后的输出作为所述卷积层的下一个 卷积层的输入数据。
PCT/CN2019/108928 2019-06-28 2019-09-29 基于卷积神经网络的数据处理方法和装置 Ceased WO2020258568A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/281,267 US20220004840A1 (en) 2019-06-28 2019-09-29 Convolutional neural network-based data processing method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910580367.8A CN110399591B (zh) 2019-06-28 2019-06-28 基于卷积神经网络的数据处理方法和装置
CN201910580367.8 2019-06-28

Publications (1)

Publication Number Publication Date
WO2020258568A1 true WO2020258568A1 (zh) 2020-12-30

Family

ID=68323666

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/108928 Ceased WO2020258568A1 (zh) 2019-06-28 2019-09-29 基于卷积神经网络的数据处理方法和装置

Country Status (3)

Country Link
US (1) US20220004840A1 (zh)
CN (1) CN110399591B (zh)
WO (1) WO2020258568A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111882029A (zh) * 2020-06-22 2020-11-03 华控清交信息科技(北京)有限公司 一种数据处理方法及装置
CN111898081B (zh) * 2020-07-09 2024-02-27 上海兆芯集成电路股份有限公司 卷积运算方法及卷积运算装置
CN111967582B (zh) * 2020-08-07 2022-07-08 苏州浪潮智能科技有限公司 一种cnn卷积层运算方法及cnn卷积层运算加速器
CN112668709B (zh) * 2020-12-30 2023-06-13 上海壁仞智能科技有限公司 计算装置以及用于数据重用的方法
CN115034351B (zh) * 2021-03-04 2025-04-08 杭州海康威视数字技术股份有限公司 数据处理方法、卷积神经网络训练方法、装置和fpga
CN115481713A (zh) * 2021-06-15 2022-12-16 瑞昱半导体股份有限公司 改进卷积神经网络进行计算的方法
CN113641952B (zh) * 2021-10-14 2022-02-08 北京壁仞科技开发有限公司 卷积设备、卷积方法、矩阵拆聚装置以及矩阵拆聚方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106951395A (zh) * 2017-02-13 2017-07-14 上海客鹭信息技术有限公司 面向压缩卷积神经网络的并行卷积运算方法及装置
US20170208081A1 (en) * 2016-01-01 2017-07-20 Tata Consultancy Services Limited Systems and methods for analyzing sensor data using incremental autoregression techniques
CN108205702A (zh) * 2017-12-29 2018-06-26 中国人民解放军国防科技大学 一种多输入多输出矩阵卷积的并行处理方法
CN109086244A (zh) * 2018-07-11 2018-12-25 中国人民解放军国防科技大学 一种基于向量处理器的矩阵卷积向量化实现方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100298327B1 (ko) * 1999-06-30 2001-11-01 구자홍 고속 컨벌루션 처리 방법 및 그 장치
US10185891B1 (en) * 2016-07-08 2019-01-22 Gopro, Inc. Systems and methods for compact convolutional neural networks
US11003985B2 (en) * 2016-11-07 2021-05-11 Electronics And Telecommunications Research Institute Convolutional neural network system and operation method thereof
US10417560B2 (en) * 2016-12-01 2019-09-17 Via Alliance Semiconductor Co., Ltd. Neural network unit that performs efficient 3-dimensional convolutions
CN107145939B (zh) * 2017-06-21 2020-11-24 北京图森智途科技有限公司 一种低计算能力处理设备的计算机视觉处理方法及装置
CN108537330B (zh) * 2018-03-09 2020-09-01 中国科学院自动化研究所 应用于神经网络的卷积计算装置及方法
US11481612B1 (en) * 2018-04-20 2022-10-25 Perceive Corporation Storage of input values across multiple cores of neural network inference circuit
CN109190756B (zh) * 2018-09-10 2022-02-18 中国科学院计算技术研究所 基于Winograd卷积的运算装置及包含该装置的神经网络处理器
CN109214506B (zh) * 2018-09-13 2022-04-15 深思考人工智能机器人科技（北京）有限公司 一种基于像素的卷积神经网络建立装置及方法
US11455370B2 (en) * 2018-11-19 2022-09-27 Groq, Inc. Flattened input stream generation for convolution with expanded kernel
CN109784489B (zh) * 2019-01-16 2021-07-30 北京大学软件与微电子学院 基于fpga的卷积神经网络ip核

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170208081A1 (en) * 2016-01-01 2017-07-20 Tata Consultancy Services Limited Systems and methods for analyzing sensor data using incremental autoregression techniques
CN106951395A (zh) * 2017-02-13 2017-07-14 上海客鹭信息技术有限公司 面向压缩卷积神经网络的并行卷积运算方法及装置
CN108205702A (zh) * 2017-12-29 2018-06-26 中国人民解放军国防科技大学 一种多输入多输出矩阵卷积的并行处理方法
CN109086244A (zh) * 2018-07-11 2018-12-25 中国人民解放军国防科技大学 一种基于向量处理器的矩阵卷积向量化实现方法

Also Published As

Publication number Publication date
US20220004840A1 (en) 2022-01-06
CN110399591A (zh) 2019-11-01
CN110399591B (zh) 2021-08-31

Similar Documents

Publication Publication Date Title
WO2020258568A1 (zh) 基于卷积神经网络的数据处理方法和装置
US11249721B2 (en) Multiplication circuit, system on chip, and electronic device
CN111240746B (zh) 一种浮点数据反量化及量化的方法和设备
US12159223B2 (en) Processing data of a neural network
US20180150721A1 (en) Convolution processing apparatus and method
JP6767660B2 (ja) プロセッサ、情報処理装置及びプロセッサの動作方法
CN111507473B (zh) 一种基于Crossbar架构的剪枝方法及系统
CN111382833A (zh) 多层神经网络模型的训练和应用方法、装置及存储介质
US11823418B2 (en) Method and system for improving compression ratio by difference between blocks of image file
CN108897716B (zh) 通过存储器读写操作来缩减计算量的数据处理装置及方法
WO2023123919A1 (zh) 数据处理电路、数据处理方法及相关产品
CN113313247A (zh) 基于数据流架构的稀疏神经网络的运算方法
CN116051345A (zh) 图像数据处理方法、装置、计算机设备及可读存储介质
US11539955B2 (en) Method and system for improving compression ratio through pixel conversion of image file
CN115309333A (zh) 一种带状稀疏矩阵的数据存储格式及其乘法加速方法
CN111476101A (zh) 视频镜头切换检测方法及装置、计算机可读存储介质
CN115034351B (zh) 数据处理方法、卷积神经网络训练方法、装置和fpga
CN116957018B (zh) 一种逐通道卷积的实现方法
CN114140618B (zh) 一种卷积特征缓存方法、装置、电子设备及可读存储介质
WO2022221982A1 (zh) 一种图像重建方法、装置、终端设备和存储介质
CN106909320A (zh) 一种多维数据扩充传输的方法、装置以及系统
WO2024114505A1 (zh) 一种通用、可配置的图像滤波计算多行输出系统和方法
CN115660055A (zh) 降低存储器带宽的方法
WO2022000324A1 (zh) 数据编码方法、数据解码方法、数据处理方法、编码器、解码器、系统、可移动平台与计算机可读介质
CN119728879B (zh) 一种视频降噪方法、装置、电子设备以及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19934932

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19934932

Country of ref document: EP

Kind code of ref document: A1