WO2022179075A1 - Data processing method and apparatus, computer device and storage medium - Google Patents

Abstract

A data processing method and apparatus, a computer device and a storage medium. The method comprises: determining, from an image feature matrix to be processed and weight matrices, target feature elements and target weight elements corresponding to a plurality of processing cycles, wherein said image feature matrix corresponds to a plurality of weight matrices; in response to the arrival of any processing cycle, each processing engine (PE) in a PE array acquiring a corresponding target feature element and a corresponding target weight element of the processing cycle, and performing a preset operation, so as to obtain intermediate processing data, wherein for any processing cycle, target feature elements in the PE array comprise repeated feature elements, and target weight elements corresponding to the repeated feature elements are weight elements, corresponding to the repeated feature elements, in different weight matrices; and on the basis of intermediate processing data corresponding to the plurality of processing cycles, obtaining result data of the processing of said image feature matrix.

Description

A data processing method, device, computer equipment and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims priority to Chinese Patent Publication No. 202110221235.3 filed on February 26, 2021, the entire contents of which are incorporated herein by reference.

technical field

The present disclosure relates to the field of computer technology, and in particular, to a data processing method, apparatus, computer device, and storage medium.

Background technique

Convolutional Neural Networks (CNN), as an important model of deep learning, have a wide range of applications in image recognition, natural language processing, etc. In a convolutional neural network, there are many different network layers such as convolutional layers, pooling layers, activation layers, and fully connected layers.

Among them, the fully connected layer is composed of a large number of nodes. When calculating each node of the fully connected layer, due to the large amount of processing data such as input data and related parameters, it takes a long time to calculate each time. The required data is transmitted to the operation unit, resulting in low processing efficiency.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide at least a data processing method, apparatus, computer device, and storage medium.

In a first aspect, an embodiment of the present disclosure provides a data processing method, including: from a feature matrix and a weight matrix of an image to be processed, determining target feature elements and target weight elements corresponding to multiple processing cycles respectively; The processing image feature matrix corresponds to multiple weight matrices; in response to the arrival of any processing cycle, each PE in the processing engine PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation to obtain an intermediate Processing data; wherein, for any processing cycle, the target feature elements in the PE array include repeated feature elements, and the target weight elements corresponding to the repeated feature elements are respectively the weight elements corresponding to the repeated feature elements in different weight matrices ; Based on the intermediate processing data corresponding to a plurality of processing cycles, the result data of processing the feature matrix of the to-be-processed image is obtained.

In this way, by multiplexing the feature elements transmitted to the PE array in each processing cycle, the amount of data that needs to be read into the PE array in each processing cycle is reduced, and the amount of data that needs to be read into the PE array is reduced. Time-consuming and improve the processing efficiency of the PE array.

In a possible implementation manner, before the determining of the target feature elements and target weight elements corresponding to the multiple processing cycles respectively, the method further includes: based on the size of the PE array, performing the processing on the original to-be-processed image feature matrix and the original weight matrix. The size is transformed to obtain the feature matrix of the to-be-processed image and the weight matrix.

In this way, the size of the original to-be-processed image feature matrix and the original weight matrix can be transformed to match the PE array, which can simplify the processing logic in the subsequent processing process and simplify the processing process.

In a possible implementation manner, the target feature elements corresponding to the multiple processing cycles respectively include at least one image feature element in the to-be-processed image feature matrix; the target feature element is in the to-be-processed image feature matrix. The position in , is consistent with the position of the corresponding target weight element in the corresponding weight matrix.

In a possible implementation manner, each row of the PE array includes repeated feature elements; in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element. and performing a preset operation to obtain intermediate processing data, including: in response to the arrival of any processing cycle, transmitting the target feature elements of the PE array rows in the to-be-processed image feature matrix to a column of PEs in the PE array , and copy the target feature elements in one column of PEs to PEs in other columns as the first operand of the corresponding PEs; and copy the target feature elements from different weight matrices corresponding to the target feature elements in each column of PEs respectively The weight element is transferred to the PE in the PE array corresponding to the position of the target feature element as the second operand of the corresponding PE; the first operand and the second operand stored in the PE array are processed by the PE array. A preset operation is performed on the operands to obtain intermediate processing data corresponding to the processing cycle.

In this way, a column of feature elements in the feature data of the image to be processed is multiplexed in the PE array, thereby reducing the quantity of feature data that needs to be transmitted to the PE array, thereby improving the data processing efficiency.

In a possible implementation manner, the use of the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain intermediate processing data corresponding to the processing cycle, Including: in the processing cycle, weighted summation is performed on each column of target feature elements in the first operand and each column of weight elements in the second operand to obtain the different weight matrices Corresponding intermediate data; based on the intermediate data corresponding to the different weight matrices, obtain the intermediate processing data corresponding to the processing period.

In this way, the parallel processing of the processing tasks corresponding to multiple weight data in one processing cycle is realized, and the processing of the processing tasks corresponding to the multiple weight data is completed through multiple processing cycles, so that the elements in the feature matrix of the image to be processed are in Each cycle can be multiplexed to improve the efficiency of data processing.

In a possible implementation manner, each column of the PE array includes repeated feature elements; in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element. and performing a preset operation to obtain intermediate processing data, including: in response to the arrival of any processing cycle, transmitting the target feature elements of the PE array columns in the to-be-processed image feature matrix to a row of PEs in the PE array , and copy the target feature elements in the PE in the row to the PEs in other rows as the first operand of the corresponding PE; and copy the target feature elements from different weight matrices corresponding to the target feature elements in each row PE respectively The weight element is transferred to the PE in the PE array corresponding to the position of the target feature element as the second operand of the corresponding PE; the first operand and the second operand stored in the PE array are processed by the PE array. A preset operation is performed on the operands to obtain intermediate processing data corresponding to the processing cycle.

In this way, a row of feature elements in the feature data of the image to be processed is multiplexed in the PE array, reducing the quantity of feature data that needs to be transmitted to the PE array, thereby improving data processing efficiency.

In a possible implementation manner, the use of the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain intermediate processing data corresponding to the processing cycle, Including: in the processing cycle, weighted summation is performed on each row of target feature elements in the first operand and each row of weight elements in the second operand to obtain the different weight matrices Corresponding intermediate data; based on the intermediate data corresponding to the different weight matrices, obtain the intermediate processing data corresponding to the processing period.

In a possible implementation manner, each PE of the PE array includes repeated feature elements; in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and perform a preset operation to obtain intermediate processing data, including: in response to the arrival of any processing cycle, transferring a target feature element in the feature matrix of the to-be-processed image to a PE in the PE array, and transferring The target feature element in the one PE is copied to other PEs as the first operand of the corresponding PE; the weight matrix corresponding to the one target feature element from all the PEs in the PE array with different numbers The target weight element is transferred to each PE of the PE array as the second operand of the corresponding PE; the first operand and the second operand stored in the PE array are pre-processed by using the PE array. Set the operation to obtain the intermediate processing data corresponding to the processing cycle.

In a possible implementation manner, the obtaining result data of processing the feature matrix of the to-be-processed image based on the intermediate processing data corresponding to the plurality of processing cycles respectively includes: In the intermediate processing data, the intermediate data belonging to the same weight matrix are accumulated to obtain the result value corresponding to each weight matrix; the result data of processing the feature matrix of the image to be processed is obtained based on the result values corresponding to the multiple weight matrices respectively.

In a possible implementation manner, the preset operation corresponding to any processing cycle includes: a sub-operation of performing a fully connected operation on the feature matrix of the to-be-processed image.

In this way, the full connection processing of the feature matrix of the image to be processed is realized, so that the efficiency of the full connection processing is higher, and the processing speed of the neural network using this method for the full connection processing is improved.

In a second aspect, an embodiment of the present disclosure provides a data processing apparatus, including: a controller and a processing engine PE array; the controller is configured to determine targets corresponding to multiple processing cycles from a feature matrix and a weight matrix of an image to be processed Feature element and target weight element; wherein, the feature matrix of the image to be processed corresponds to multiple weight matrices; the PE array is used to respond to the arrival of any processing cycle, and each PE in the PE array obtains the corresponding processing cycle. The target feature element and the corresponding target weight element are pre-set to obtain intermediate processing data; based on the intermediate processing data corresponding to multiple processing cycles, the result data of processing the feature matrix of the to-be-processed image is obtained; wherein, For any processing cycle, the target feature elements in the PE array include repeated feature elements, and the target weight elements corresponding to the repeated feature elements are weight elements corresponding to the repeated feature elements in different weight matrices, respectively.

In a possible implementation manner, the controller, before determining the target feature elements and target weight elements corresponding to the multiple processing cycles respectively, is further configured to: and the original weight matrix to perform size transformation to obtain the feature matrix of the to-be-processed image and the weight matrix.

In a possible implementation manner, the target feature elements corresponding to the multiple processing cycles respectively include at least one image feature element in the to-be-processed image feature matrix; the target feature element is in the to-be-processed image feature matrix. The position in , is consistent with the position of the corresponding target weight element in the corresponding weight matrix.

In a possible implementation manner, each row of the PE array includes repeated feature elements; in the PE array, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle. and the corresponding target weight elements and perform a preset operation to obtain intermediate processing data, used for: in response to the arrival of any processing cycle, the PE array row in the to-be-processed image feature matrix is transmitted to several target feature elements. In a row of PEs in the PE array, copy the target feature elements in the one row of PEs to PEs in other rows as the first operand of the corresponding PE; and separate the target feature elements from each row of PEs. Corresponding target weight elements from different weight matrices are transferred to the PE in the PE array corresponding to the position of the target feature element as the second operand of the corresponding PE; A preset operation is performed on the first operand and the second operand to obtain intermediate processing data corresponding to the processing cycle.

In a possible implementation manner, the PE array uses the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the corresponding value of the processing cycle. During the intermediate processing of data, it is used for: in the processing cycle, weighted summation is performed on each column of target feature elements in the first operand and each column of weight elements in the second operand, Obtain intermediate data corresponding to the different weight matrices; and obtain intermediate processing data corresponding to the processing period based on the intermediate data corresponding to the different weight matrices.

In a possible implementation manner, each column of the PE array includes repeated feature elements; in the PE array, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle. and the corresponding target weight elements and perform a preset operation to obtain intermediate processing data, used for: in response to the arrival of any processing cycle, the number of target feature elements in the PE array column in the image feature matrix to be processed is transmitted to In a row of PEs in the PE array, copy the target feature elements in the row of PEs to PEs in other rows as the first operand of the corresponding PE; and separate the target feature elements from each row of PEs. Corresponding target weight elements from different weight matrices are transferred to the PE in the PE array corresponding to the position of the target feature element as the second operand of the corresponding PE; A preset operation is performed on the first operand and the second operand to obtain intermediate processing data corresponding to the processing cycle.

In a possible implementation manner, the PE array uses the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the corresponding value of the processing cycle. During the intermediate processing of data, it is used for: in the processing cycle, weighted summation is performed on each row of target feature elements in the first operand and each row of weight elements in the second operand to obtain The intermediate data corresponding to the different weight matrices; based on the intermediate data corresponding to the different weight matrices, the intermediate processing data corresponding to the processing period is obtained.

In a possible implementation manner, each PE of the PE array includes repeated feature elements; in the PE array, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature corresponding to the processing cycle. element and the corresponding target weight element and perform a preset operation to obtain intermediate processing data, used for: in response to the arrival of any processing cycle, a target feature element in the feature matrix of the to-be-processed image is transmitted to the PE array in one PE, and copy the target feature element in the one PE to other PEs as the first operand of the corresponding PE; copy the target feature element corresponding to the one from all the PE arrays. The target weight elements of weight matrices with different numbers of PEs are transferred to each PE of the PE array as the second operand of the corresponding PE; the first operand stored in the PE array is processed by the PE array. and perform a preset operation on the second operand to obtain intermediate processing data corresponding to the processing cycle.

In a possible implementation manner, the PE array, when obtaining the result data of processing the image feature matrix to be processed based on the intermediate processing data corresponding to the plurality of processing cycles, is used to: Among the intermediate processing data corresponding respectively in the processing cycle, the intermediate data belonging to the same weight matrix are accumulated to obtain the result value corresponding to each weight matrix; based on the result values corresponding to the multiple weight matrices, the feature matrix of the image to be processed is obtained. The resulting data for processing.

In a possible implementation manner, the preset operation corresponding to any processing cycle includes: a sub-operation of performing a fully connected operation on the feature matrix of the to-be-processed image.

In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory, and the data processing apparatus according to the second aspect.

In a fourth aspect, an optional implementation manner of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program executes the first aspect, or any of the first aspect, when the computer program is run. steps in one possible implementation.

For the description of the effects of the above-mentioned data apparatus, computer equipment, and computer-readable storage medium, reference may be made to the description of the above-mentioned data processing method, which will not be repeated here.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

FIG. 1 shows a flowchart of a data processing method provided by an embodiment of the present disclosure;

FIG. 2 shows an example of processing target feature elements by using PE matrix in the data processing method provided by the embodiment of the present disclosure;

FIG. 3 shows another example of processing the target feature element by using the PE matrix in the data processing method provided by the embodiment of the present disclosure;

FIG. 4 shows another example of processing the target feature element by using the PE matrix in the data processing method provided by the embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of a data processing apparatus provided by an embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

After research, it is found that the commonly used artificial intelligence (Artificial Intelligence, AI) accelerator hardware architecture mainly includes storage unit, computing unit, control unit, etc., among which the core computing unit is generally composed of processing engine (Processing Engine, PE) array and register array ( local register file) composition. In the full connection processing in the neural network, it is usually necessary to use multiple sets of full connection weights to perform weighted summation processing on the eigenvalues of each feature point in the feature map, so as to obtain the fully connected processing result of the feature map. When using the AI accelerator to perform full-connection processing on image data, it is necessary to read at least part of the image feature elements in the feature map into the PE array in each processing cycle in multiple processing cycles, and combine a fully-connected weight with the read-in The weight elements corresponding to the image feature elements of the PE array are read into the PE array, and the PE array performs weighted summation processing on the read image feature elements and the weight elements. After multiple processing cycles, the result of the fully connected processing of the feature map is obtained. However, due to the limited bandwidth of the PE array to read data, for a PE array of size m*n, each processing cycle needs to read m*n image feature elements, and read the corresponding m*n weight elements, data The read-in efficiency is low, which increases the time required for full connection processing of the feature map, which in turn leads to the problem of low processing efficiency of the PE array.

Based on the above research, the present disclosure provides a data processing method. By multiplexing the image feature elements transmitted to the PE array in each processing cycle, the data of the data that needs to be read into the PE array in each processing cycle is reduced. It reduces the time required to read data into the PE array and improves the processing efficiency of the PE array.

The defects existing in the above solutions are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions to the above problems proposed by the present disclosure hereinafter should be the inventors Contributions made to this disclosure during the course of this disclosure.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The data processing method provided by the embodiments of the present disclosure will be described below.

Referring to FIG. 1, which is a flowchart of a data processing method provided by an embodiment of the present disclosure, the method includes steps S101-S103, wherein:

S101: From the image feature matrix to be processed and the weight matrix, determine target feature elements and target weight elements corresponding to multiple processing cycles respectively; wherein, the image feature matrix to be processed corresponds to multiple weight matrices;

S102: In response to the arrival of any processing cycle, each PE in the processing engine PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation to obtain intermediate processing data;

Wherein, for any processing cycle, the target feature element in the PE array includes a repeating feature element, and the target weight element corresponding to this repeating feature element is respectively a weight element corresponding to this repeating feature element in different weight matrices;

S103: Obtain result data of processing the feature matrix of the image to be processed based on the intermediate processing data corresponding to the multiple processing cycles respectively.

In this embodiment of the present disclosure, target feature elements and target weight elements corresponding to multiple processing cycles are determined from the feature matrix and weight matrix of the image to be processed, and in response to the arrival of any processing cycle, each PE in the PE array obtains The target feature element corresponding to the processing cycle and the corresponding target weight element are subjected to a preset operation, and the result data of processing the feature matrix of the image to be processed is obtained based on the intermediate processing data corresponding to the multiple processing cycles. In this process, for In any processing cycle, the target feature elements in the PE array include repeated feature elements, and the target weight elements corresponding to the repeated feature elements are respectively the weight elements corresponding to the repeated feature elements in different weight matrices, so that in each processing cycle, The image feature elements transmitted to the PE array are multiplexed to reduce the amount of data that needs to be read into the PE array in each processing cycle, reduce the time required for data to be read into the PE array, and improve the processing efficiency of the PE array.

The above S101 to S103 will be described in detail below.

For the above S101, when determining target feature elements and target weight elements corresponding to multiple processing cycles from the image feature matrix and weight matrix to be processed, for example, based on the number of image feature matrices and weight matrices to be processed, determine the required The number of processing cycles, and then target feature elements and target weight elements are determined for each processing cycle.

Exemplarily, the number of target feature elements determined for each processing cycle is less than the number of PEs in the PE array; the target feature elements determined for each processing cycle are transferred from the external memory to some PEs in the PE array , the data required by other PEs in the PE array is the repeated feature elements that multiplex the target feature elements transmitted to the PE array, so it does not need to be transmitted externally, but from the PE that stores these target feature elements. Just make a copy.

In practical applications, the feature map that needs to be fully connected is usually the feature map output by the convolution layer, which can include multiple ones. The feature matrix of the image to be processed here can be one of multiple feature maps, or multiple feature maps. For a combined feature map composed of feature maps, in the case of one of multiple feature maps, the data processing method provided by the present disclosure can be separately executed for each feature map, and then the data obtained by each execution can be combined to obtain the final result. The result of the full join operation.

In a possible implementation, the target feature elements corresponding to multiple processing cycles respectively include at least one image feature element in the to-be-processed image feature matrix; the target feature element is in the to-be-processed image feature matrix. position, which is consistent with the position of the corresponding target weight element in the corresponding weight matrix.

In this embodiment of the present disclosure, for the convenience of processing, for example, before determining the target feature elements and target weight elements corresponding to each processing cycle, the original image feature matrix to be processed and the original weight matrix may be size-transformed based on the size of the PE array , to obtain the feature matrix and weight matrix of the image to be processed.

Exemplarily, if the size of the original to-be-processed image feature matrix is M*N*S, the size of the corresponding original weight matrix is also M*N*S. If the size of the PE array is A*A, when performing size transformation on the original image feature matrix to be processed, the size of the obtained image feature matrix to be processed is: A*A*W, where W=(M*N* S)/(A*A). The dimensions of each of the multiple weight matrices are also A*A*W.

In addition, if the size of the feature map represented by the original to-be-processed image feature matrix is smaller than or equal to the size of the PE array, the original to-be-processed image feature matrix may be size-transformed, or not. In the case of no size transformation of the original image feature matrix to be processed, if the size of the feature map is smaller than the size of the PE array, then only a part of the PE in the PE array will be used during the process of using the PE array, while Not all will be used.

After the target feature elements and target weight elements corresponding to each processing period are determined, after any processing period arrives, preset processing can be performed on the target feature elements and target weight elements corresponding to the processing period to obtain intermediate processing data.

Here, the preset processing performed on the target feature element and the target weight element corresponding to the processing period includes, for example, a sub-operation of performing a full connection operation on the feature matrix of the to-be-processed image.

Here, the sub-operations performed are, for example, sub-operations corresponding to the weight matrices respectively.

Exemplarily, the image feature matrix ImgC to be processed is expressed as

The weight matrix Wi is expressed as:

Among them, i represents the ith weight matrix. Taking the first group of weight parameters W1 as an example, there are 16 weight data, which are expressed as W1_1, W1_2, W1_3, ..., W1_16 for the convenience of expression, which can form a weight matrix

Then when using the weight matrix Wi to perform the full connection operation on the feature matrix ImgC of the image to be processed, the full connection operation corresponding to the i-th weight matrix can be expressed as:

O1=a1×wi_1+a2×wi_2+a3×wi_3+a4×wi_4

+a5×wi_5+a6×wi_6+a7×wi_7+a8×wi_8

+a9×wi_9+a10×wi_10+a11×wi_11+a12×wi_12

+a13×wi_13+a14×wi_14+a15×wi_15+a16×wi_16.

The sub-operations corresponding to the weight matrix include, for example:

O1 ¹ =a1×wi_1+a2×wi_2+a3×wi_3+a4×wi_4;

O1 ² =a5×wi_5+a6×wi_6+a7×wi_7+a8×wi_8;

O1 ³ =a9×wi_9+a10×wi_10+a11×wi_11+a12×wi_12;

O1 ⁴ =a13×wi_13+a14×wi_14+a15×wi_15+a16×wi_16.

in,

O1=O1 ¹ +O1 ² +O1 ³ +O1 ⁴ .

For the above S102 and S103: in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation, for example: in any processing cycle After arrival, read the target feature elements and target weight elements corresponding to the processing cycle from the external memory, store them in some PEs of the PE array, and then copy the target feature elements to other PEs in the PE array to form the first Operand, and then use the read target weight element as the second operand, and use the first operand and the second operand to perform a preset operation to obtain the intermediate processing data corresponding to the processing cycle.

In the embodiment of the present disclosure, the processing of each processing cycle is described in detail by taking as an example that the size of the feature matrix of the image to be processed is the same as the size of the PE array. Here, the examples in the following (1) to (3) are only examples of determining the target feature element and storing it in the PE array. Other methods can also be used to determine the target feature element. When determining the target feature element can not be determined according to the order of the image feature elements, as long as it is ensured that each image feature element in the image feature matrix to be processed is processed by the corresponding weight elements in the N weight matrices, and each final weight matrix corresponding to The processing result can be the result of weighted summation of all image feature elements in the feature matrix of the image to be processed and the weight matrix.

(1) Each row of the PE array includes repeating feature elements. In this case, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation to obtain intermediate processing data, including: responding When any processing cycle arrives, transfer the target feature elements of the PE array row in the image feature matrix to be processed to a column of PEs in the PE array, and copy the target feature elements in the PE array In the PE of other columns, as the first operand of the corresponding PE; and transfer the target weight elements from different weight matrices corresponding to the target feature elements in each column of PEs to the position of the target feature element in the PE array. In the corresponding PE, as the second operand of the corresponding PE; use the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the corresponding processing cycle. intermediate processing data.

When using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the intermediate processing data corresponding to the processing cycle, for example, the following method may be used: In the processing cycle, each column of target feature elements in the first operand and each column of weight elements in the second operand are weighted and summed to obtain the middle corresponding to the different weight matrices. data; based on the intermediate data corresponding to the different weight matrices, obtain the intermediate processing data corresponding to the processing period.

In this case, the target feature elements determined for multiple processing cycles include, for example, a row of image feature elements in the image feature matrix to be processed. Since the number of image feature elements in a row of the image feature matrix to be processed is the same as the number of columns in the PE array, when reading a row of image feature elements into the PE array, it can be read into the PE array. in a column of PE.

For example, continue to take the above to-be-processed image feature matrix ImgC and weight matrix Wi as an example, where i is an integer from 1 to 4.

In the first processing cycle, the image feature elements a1, a2, a3, and a4 included in the first row of the image feature matrix to be processed are stored as target feature elements in the first row PE, and copied in other PE rows, Get the first operand of the PE array in the first processing cycle, that is, it can form a matrix

The corresponding weight data can form a matrix

That is, the second operand.

When the first operand and the second operand are stored in the PE array, as shown in (a) in Figure 2, PE1, PE2, PE3, and PE4 all store the image feature element a1, and store the weight matrix W1~ Weight values corresponding to the image feature element a1 in W4: w1_1, w2_1, w3_1, w4_1.

Similarly, PE5-PE8 all store the image feature element a2, and respectively store the weight values corresponding to the image feature element a2 in the weight matrices W1-W4: w1_2, w2_2, w3_2, w4_2.

PE9-PE12 all store the image feature element a3, and respectively store the weight values corresponding to the image feature element a3 in the weight matrices W1-W4: w1_3, w2_3, w3_3, w4_3.

PE13-PE16 all store the image feature element a4, and respectively store the weight values corresponding to a4 in the weight matrices W1-W4: w1_4, w2_4, w3_4, w4_4.

Then, using w1_1, w1_2, w1_3, and w1_4 as weights, the first columns a1, a2, a3, and a4 are weighted and summed to obtain the intermediate data O1 ¹ corresponding to the weight matrix W1.

Taking w2_1, w2_2, w2_3, and w2_4 as weights, weighted summation is performed on the second columns a1, a2, a3, and a4 to obtain the intermediate data O2 ¹ corresponding to the weight matrix W2.

Taking w3_1, w3_2, w3_3, and w3_4 as weights, the third column a1, a2, a3, and a4 are weighted and summed to obtain the intermediate data O3 ¹ corresponding to the weight matrix W3.

Taking w4_1, w4_2, w4_3, and w4_4 as weights, the fourth column a1, a2, a3, and a4 are weighted and summed to obtain the intermediate data O4 ¹ corresponding to the weight matrix W4.

Then, combine the intermediate data O1 ¹ , O2 ¹ , O3 ¹ and O4 ¹ as the intermediate processing data corresponding to the first processing cycle.

In addition, in the second processing cycle, the image feature elements a5, a6, a7, and a8 included in the second row of the image feature matrix ImgC to be processed are stored as target feature elements in the first row PE, and in other PE rows Copy to get the first operand in the second processing cycle, that is, a matrix can be formed

The corresponding weight data can form a matrix

That is, the second operand.

When the first operand and the second operand are stored in the PE array, as shown in (b) of Figure 2, PE1, PE2, PE3, and PE4 all store the image feature element a5, and store the weight matrix W1~ Weight values corresponding to the image feature element a5 in W4: w1_5, w2_5, w3_5, w4_5.

Similarly, PE5-PE8 all store the image feature element a6, and respectively store the weight values corresponding to the image feature element a6 in the weight matrices W1-W4: w1_6, w2_6, w3_6, w4_6.

PE9-PE12 all store the image feature element a7, and respectively store the weight values corresponding to the image feature element a7 in the weight matrices W1-W4: w1_7, w2_7, w3_7, w4_7.

PE13-PE16 all store the image feature element a8, and respectively store the weight values corresponding to the image feature element a8 in the weight matrices W1-W4: w1_8, w2_8, w3_8, w4_8.

Then, using w1_5, w1_6, w1_7, and w1_8 as weights, the first columns a5, a6, a7, and a8 are weighted and summed to obtain the intermediate data O1 ² corresponding to the weight matrix W1.

Taking w2_5, w2_6, w2_7, and w2_8 as weights, weighted summation is performed on the second columns a5, a6, a7, and a8 to obtain the intermediate data O2 ² corresponding to the weight matrix W2.

Taking w3_5, w3_6, w3_7, and w3_8 as weights, the third column a5, a6, a7, and a8 are weighted and summed to obtain the intermediate data O3 ² corresponding to the weight matrix W3.

Taking w4_5, w4_6, w4_7, and w4_8 as weights, weighted summation is performed on the fourth columns a5, a6, a7, and a8 to obtain the intermediate data O4 ² corresponding to the weight matrix W4.

Then combine the intermediate data O1 ² , O2 ² , O3 ² and O4 ² as the intermediate processing data corresponding to the processing cycle.

In the third processing cycle, in a similar manner, the intermediate data O1 ³ generated by w1_9, w1_10, w1_11, w1_12 in the weight matrix W1 and the corresponding image feature elements a9, a10, a11 and a12, w2_9, w2_10, w2_11, w2_12 and the intermediate data O2 ³ generated by the corresponding image feature elements a9, a10, a11 and a12, and w3_9, w3_10, w3_11, w3_12 in the weight matrix W3 and the corresponding image feature elements a9, a10, a11 and a12 are generated The intermediate data O3 ³ , the intermediate data O4 ³ generated by w4_9, w4_10, w4_11, w4_12 in the weight matrix W4 and the corresponding image feature elements a9, a10, a11 and a12.

In the fourth processing cycle, the intermediate data O1 ⁴ generated by w1_13, w1_14, w1_15, w1_16 in the weight matrix W1 and the corresponding image feature elements a13, a14, a15 and a16, w2_13, w2_14 in the weight matrix W2 are obtained in a similar manner , w2_15, w2_16 and the intermediate data O2 ⁴ generated by the corresponding image feature elements a13, a14, a15 and a16, w3_13, w3_14, w3_15, w3_16 in the weight matrix W3 and the corresponding image feature elements a13, a14, a15 and a16 are generated The corresponding intermediate data O3 ⁴ , the corresponding intermediate data O4 ⁴ generated by w4_13, w4_14, w4_15, w4_16 in the weight matrix W4 and the corresponding image feature elements a13, a14, a15 and a16.

After 4 processing cycles, the processing of the feature matrix ImgC of the image to be processed using the weight matrices W1, W2, W3 and W4 is completed, and then the intermediate data O1 ¹ , O1 ² , O1 ³ , and O1 ⁴ are added to obtain the corresponding weight matrix W1. The resulting value is O1. Add the intermediate data O2 ¹ , O2 ² , O2 ³ and O2 ⁴ to obtain the result value O2 corresponding to the weight matrix W2. In a similar way, the result value O3 corresponding to the weight matrix W3 and the result value O4 corresponding to the weight matrix W4 are obtained. . If there are more weight matrices in addition to W1, W2, W3 and W4, use other weight matrices and perform similar processing. Finally, the result data corresponding to all the weight matrices are combined to obtain the result data of processing the feature matrix of the image to be processed, that is, the result of the full connection operation.

It should be noted here that when adding the intermediate data Oi ¹ , Oi ² , Oi ³ and Oi ⁴ corresponding to the weight matrix Wi, the intermediate data Oi ¹ corresponding to the weight matrix Wi may be added after all processing cycles are completed. , Oi ² , Oi ³ and Oi ⁴ are added.

In each processing cycle except the first processing cycle, the result values corresponding to each weight matrix obtained in this cycle can be added to the result values and values of all cycles obtained in the previous cycle. Then in the last processing cycle, the result of adding the intermediate data Oi ¹ , Oi ² , Oi ³ and Oi ⁴ corresponding to the weight matrix Wi can be directly output.

For example, taking the weight matrix W2 as an example, after the end of the first processing cycle, the PE stores the obtained intermediate data O2 ¹ in the register; after obtaining the intermediate data O2 ² in the second processing cycle, the intermediate data is read from the register. O2 ¹ , add the intermediate data O2 ¹ and O2 ² to obtain the result value and value O2 ¹ +O2 ² , and store the result value and value in the register. After the intermediate data O2 ³ is obtained in the third processing cycle, the result value and the value O2 ¹ +O2 ² obtained in the second processing cycle are taken out from the register, and O2 ¹ +O2 ² and O2 ³ are added to obtain the third The result value and value O2 ¹ +O2 ² +O2 ³ , .

(2) Each column of the PE array includes repeating feature elements. In this case, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation to obtain intermediate processing data, including: responding When any processing cycle arrives, transfer the target feature elements of the PE array columns in the image feature matrix to be processed to a row of PEs in the PE array, and copy the target feature elements in the row of PEs to In the PEs of other rows, as the first operand of the corresponding PE; and transfer the target weight elements from different weight matrices corresponding to the target feature elements in each row of PEs to the PE array corresponding to the target feature element position. In the PE, as the second operand of the corresponding PE; use the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the corresponding processing cycle Intermediate processing data.

When using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the intermediate processing data corresponding to the processing cycle, for example, the following method may be used: In the processing cycle, each row of target feature elements in the first operand and each row of weight elements in the second operand are weighted and summed to obtain the middle corresponding to the different weight matrices. data; based on the intermediate data corresponding to the different weight matrices, obtain the intermediate processing data corresponding to the processing period.

In this case, the target feature elements determined for multiple processing cycles include, for example, a column of image feature elements in the image feature matrix to be processed. Since the number of image feature elements in a column of the image feature matrix to be processed is the same as the number of rows in the PE array, when reading a column of image feature elements into the PE array, you can read a column of image feature elements into the PE array. in a line of PE.

For example, continue to take the above to-be-processed image feature matrix ImgC and weight matrix Wi as an example, where i is an integer from 1 to 4.

In the first processing cycle, the image feature elements a1, a5, a9, and a13 included in the first column of the image feature matrix to be processed are used as the target feature elements, and the target feature elements are stored in the first row PE, and in other The PE row is copied to obtain the first operand of the PE array in the first processing cycle, that is, a matrix can be formed

The corresponding weight data can form a matrix

That is, the second operand.

When the first operand and the second operand are stored in the PE array, as shown in (a) in Figure 3, PE1, PE2, PE3, PE4 store image feature elements a1, a5, a9, a13 respectively, and The weight values corresponding to the image feature elements a1, a5, a9, and a13 in the weight matrix W1 are stored: w1_1, w1_5, w1_9, and w1_13.

Similarly, PE5 to PE8 store the image feature elements a1, a5, a9, and a13, respectively, and store the weight values corresponding to the image feature elements a1, a5, a9, and a13 in the weight matrix W2: w2_1, w2_5, w2_9, w2_13.

PE9 to PE12 all store the image feature elements a1, a5, a9, and a13, respectively, and respectively store the weight values in the weight matrix W3 corresponding to the image feature elements a1, a5, a9, and a13: w3_1, w3_5, w3_9, w3_13.

PE13 to PE16 store image feature elements a1, a5, a9, and a13, respectively, and store weight values corresponding to image feature elements a1, a5, a9, and a13 in the weight matrix W4: w4_1, w4_5, w4_9, and w4_13.

Then, using w1_1, w1_5, w1_9, and w1_13 as weights, the first row a1, a5, a9, and a13 are weighted and summed to obtain the intermediate data O1 ¹ corresponding to the weight matrix W1.

Taking w2_1, w2_5, w2_9, and w2_13 as weights, the second row a1, a5, a9, and a13 are weighted and summed to obtain the intermediate data O2 ¹ corresponding to the weight matrix W2.

Taking w3_1, w3_5, w3_9, and w3_13 as weights, the third row a1, a5, a9, and a13 are weighted and summed to obtain the intermediate data O3 ¹ corresponding to the weight matrix W3.

Taking w4_1, w4_5, w4_9, and w4_13 as weights, the fourth row a1, a5, a9, and a13 are weighted and summed to obtain the intermediate data O4 ¹ corresponding to the weight matrix W4.

Then, combine the intermediate data O1 ¹ , O2 ¹ , O3 ¹ and O4 ¹ as the intermediate processing data corresponding to the first processing cycle.

In addition, in the second processing cycle, the image feature elements a2, a6, a10, and a14 included in the second column of the image feature matrix to be processed are stored as target feature elements in the first row PE, and are processed in other PE rows. Copy to get the first operand of the PE array in the second processing cycle, that is, a matrix can be formed

The corresponding weight data can form a matrix

When the first operand and the second operand are stored in the PE array, as shown in (b) in Figure 3, PE1, PE2, PE3, PE4 store image feature elements a2, a6, a10, a14 respectively, and The weight values corresponding to the image feature elements a2, a6, a10, and a14 in the weight matrix W1 are stored: w1_2, w1_6, w1_10, and w1_14.

Similarly, PE5 to PE8 store the image feature elements a2, a6, a10, and a14, respectively, and store the weight values corresponding to the image feature elements a2, a6, a10, and a14 in the weight matrix W2: w2_2, w2_6, w2_10, and w2_14.

PE9 to PE12 all store the image feature elements a2, a6, a10, and a14, respectively, and store the weight values corresponding to the image feature elements a2, a6, a10, and a14 in the weight matrix W3: w3_2, w3_6, w3_10, w3_14.

PE13-PE16 store the image feature elements a2, a6, a10, and a14 respectively, and store the weight values corresponding to the image feature elements a2, a6, a10, and a14 in the weight matrix W4: w4_2, w4_6, w4_10, and w4_14.

Then, using w1_2, w1_6, w1_10, and w1_14 as weights, perform weighted summation on the first row a2, a6, a10, and a14 to obtain the intermediate data O1 ² corresponding to the weight matrix W1.

Taking w2_2, w2_6, w2_10, and w2_14 as weights, the second row a2, a6, a10, and a14 are weighted and summed to obtain the intermediate data O2 ² corresponding to the weight matrix W2.

Taking w3_2, w3_6, w3_10, and w3_14 as weights, the third row a2, a6, a10, and a14 are weighted and summed to obtain the intermediate data O3 ² corresponding to the weight matrix W3.

Taking w4_2, w4_6, w4_10, and w4_14 as weights, the fourth row a2, a6, a10, and a14 are weighted and summed to obtain the intermediate data O4 ² corresponding to the weight matrix W4.

Then combine the intermediate data O1 ² , O2 ² , O3 ² and O4 ² as the intermediate processing data corresponding to the processing cycle.

In the third processing cycle, in a similar manner, the intermediate data O1 ³ generated by w1_3, w1_7, w1_11, w1_15 in the weight matrix W1 and the corresponding image feature elements a3, a7, a11 and a15, w2_3, w2_3, The intermediate data O2 ³ generated by w2_7, w2_11, w2_15 and the corresponding image feature elements a3, a7, a11 and a15, w3_3, w3_7, w3_11, w3_15 in the weight matrix W3 and the corresponding image feature elements a3, a7, a11 and a15 are generated The intermediate data O3 ³ , the intermediate data O4 ³ generated by w4_3, w4_7, w4_11, w4_15 in the weight matrix W4 and the corresponding image feature elements a3, a7, a11 and a15.

In the fourth processing cycle, the intermediate data O1 ⁴ generated by w1_4, w1_8, w1_12, w1_16 in the weight matrix W1 and the corresponding image feature elements a4, a8, a12 and a16, w2_4, w2_8 in the weight matrix W2 are obtained in a similar manner , w2_12, w2_16 and the intermediate data O2 ⁴ generated by the corresponding image feature elements a4, a8, a12 and a16, w3_4, w3_8, w3_12, w3_16 in the weight matrix W3 and the corresponding image feature elements a4, a8, a12 and a16 are generated Intermediate data O3 ⁴ , intermediate data O4 ⁴ generated by w4_4, w4_8, w4_12, w4_16 and corresponding image feature elements a4, a8, a12 and a16 in the weight matrix W4.

After 4 processing cycles, the processing of the feature matrix ImgC of the image to be processed using the weight matrices W1, W2, W3 and W4 is completed, and then the intermediate data O1 ¹ , O1 ² , O1 ³ , and O1 ⁴ are added to obtain the corresponding weight matrix W1. The resulting value is O1. Add the intermediate data O2 ¹ , O2 ² , O2 ³ and O2 ⁴ to obtain the result value O2 corresponding to the weight matrix W2. In a similar way, the result value O3 corresponding to the weight matrix W3 and the result value O4 corresponding to the weight matrix W4 are obtained. , if the weight matrix only has W1, W2, W3 and W4, then combine the result values O1, O2, O3 and O4 as the result of processing the feature matrix of the image to be processed. If there are more weight matrices in addition to W1, W2, W3 and W4, use other weight matrices and perform similar processing. Finally, the result data corresponding to all the weight matrices are combined to obtain the result data of processing the feature matrix of the image to be processed, that is, the result of the full connection operation. Here, the process of adding the result values corresponding to the same weight matrix in different processing cycles is similar to that in (1) above, and details are not repeated here.

(3) Each PE of the PE array includes repeating feature elements. In this case, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation to obtain intermediate processing data, including: responding When any processing cycle comes, transfer a target feature element in the image feature matrix to be processed to a PE in the PE array, and copy the target feature element in the one PE to other PEs , as the first operand of the corresponding PE; transfer the target weight elements from the weight matrix with different Acl numbers corresponding to the one target feature element to each PE of the PE array, as the second corresponding PE Operands, where Acl represents the number of all PEs in the PE array; use the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the Intermediate processing data corresponding to the processing cycle.

In this case, the target feature element determined for multiple processing cycles includes, for example, one feature element in the image features to be processed, and in the corresponding processing cycle, the one feature element is read into a PE in the PE array , and copied from the one PE to the other PEs.

For example, continue to take the above to-be-processed image feature matrix ImgC and weight matrix Wi as an example, where i is an integer ranging from 1 to 16.

In the first processing cycle, take an image feature element a1 in the image feature matrix to be processed as the target feature element, store the target feature element in the first PE, and copy it to other PEs to obtain PE The first operand of the array in the first processing cycle, that is, it can form a matrix

The corresponding weight data can form a matrix

That is, the second operand.

When the first operand and the second operand are stored in the PE array, as shown in (a) in Figure 4, PE1~PE16 all store the image feature element a1, and respectively store the weight matrix W1~W16 and the image The weight value corresponding to the feature element a1: w1_1~w16_1.

Then calculate the product of w1_1 and a1 to obtain the intermediate data O1 ¹ corresponding to the weight matrix W1;

Calculate the product of w2_1 and a1 to obtain the intermediate data O2 ¹ corresponding to the weight matrix W2;

...

Calculate the product of w16_1 and a1 to obtain the intermediate data O16 ¹ corresponding to the weight matrix W16.

In the second processing cycle, the image feature element a2 is used as the target feature element of the second processing cycle, and the first operand of the PE array in the second processing cycle is:

The corresponding weight data can form a matrix

That is, the second operand.

When the first operand and the second operand are stored in the PE array, PE1~PE16 all store the image feature element a2, and respectively store the weight value corresponding to the image feature element a2 in the weight matrix W1~W16: w1_2~ w16_2.

Then calculate the product of w1_2 and a2 to obtain the intermediate data O1 ² corresponding to the weight matrix W1;

Calculate the product of w2_2 and a2 to obtain the intermediate data O2 ² corresponding to the weight matrix W2;

...

Calculate the product of w16_2 and a2 to obtain the intermediate data O16 ² corresponding to the weight matrix W16.

By analogy, in the 16th processing cycle, the image feature element a16 is used as the target feature element of the PE array in the 16th processing cycle, and the first operand constituting the 16th processing cycle is:

The corresponding weight data can form a matrix

That is, the second operand.

As shown in (b) in Figure 4, then calculate the product of w1_16 and a16 to obtain the intermediate data O1 ¹⁶ corresponding to the weight matrix W1;

Calculate the product of w2_16 and a16 to obtain the intermediate data O2 ¹⁶ corresponding to the weight matrix W2;

...

Calculate the product of w16_16 and a16 to obtain the intermediate data O16 ¹⁶ corresponding to the weight matrix W16.

After 16 processing cycles, the processing of the feature matrix ImgC of the image to be processed using the weight matrices W1, W2, W3, ..., W16 is completed, and then the intermediate data O1 ¹ -O1 ¹⁶ are added to obtain the result value O1 corresponding to the weight matrix W1 ; Add the intermediate data O2 ¹ to O2 ¹⁶ to obtain the result value O2 corresponding to the weight matrix W2; ...; Add the intermediate data O16 ¹ to O16 ¹⁶ to obtain the result value O16 corresponding to the weight matrix W16. Finally, combining the result values O1-O16 corresponding to the 16 weight matrices, the result data of processing the feature matrix of the image to be processed is obtained, that is, the result of the full connection operation.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Based on the same inventive concept, the embodiment of the present disclosure also provides a data processing apparatus corresponding to the data processing method. Reference may be made to the implementation of the method, and repeated descriptions will not be repeated.

5 , which is a schematic diagram of a data processing apparatus provided by an embodiment of the present disclosure, the apparatus includes: a controller 51 and a PE array 52 ; the controller 51 is used to obtain the image feature matrix to be processed and the PE array 52 ; In the weight matrix, the target feature elements and target weight elements corresponding to multiple processing cycles are determined respectively; wherein, the image feature matrix to be processed corresponds to multiple weight matrices; the PE array 52 is used to respond to the arrival of any processing cycle, Each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element, and performs a preset operation to obtain intermediate processing data; based on the intermediate processing data corresponding to multiple processing cycles, obtain the The result data of processing the image feature matrix for processing; wherein, for any processing cycle, the target feature elements in the PE array include repeated feature elements, and the target weight elements corresponding to the repeated feature elements are respectively in different weight matrices. Repeat the weight element corresponding to the feature element.

The data processing apparatus provided by the embodiments of the present disclosure may include a chip, an AI chip, and the like. The computer device provided by the embodiment of the present disclosure may include a smart terminal such as a mobile phone, or may be other devices, servers, etc. that can be used for data processing, which is not limited here.

In a possible implementation manner, the controller 51, before determining the target feature elements and target weight elements corresponding to the multiple processing cycles respectively, is further configured to: The matrix and the original weight matrix are subjected to size transformation to obtain the to-be-processed image feature matrix and the weight matrix.

In a possible implementation manner, the target feature elements corresponding to the multiple processing cycles respectively include at least one image feature element in the to-be-processed image feature matrix; the target feature element is in the to-be-processed image feature matrix. The position in , is consistent with the position of the corresponding target weight element in the corresponding weight matrix.

In a possible implementation manner, each row of the PE array includes repeated feature elements; in the PE array 52, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature corresponding to the processing cycle. element and the corresponding target weight element and perform a preset operation to obtain intermediate processing data, which is used to: in response to the arrival of any processing cycle, transmit several target feature elements of the PE array row in the image feature matrix to be processed. to a row of PEs in the PE array, and copy the target feature elements in the one row of PEs to PEs in other rows as the first operand of the corresponding PE; and combine the target feature elements in each row of PEs with the The corresponding target weight elements from different weight matrices are transferred to the PE in the PE array corresponding to the position of the target feature element as the second operand of the corresponding PE; the PE array is used to store data in the PE array. Perform preset operations on the first operand and the second operand of , to obtain intermediate processing data corresponding to the processing cycle.

In a possible implementation manner, the PE array 52 uses the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the corresponding processing cycle. When the intermediate processing data of , is used for: in the processing cycle, weighted summation is performed on each column of target feature elements in the first operand and each column of weight elements in the second operand , to obtain the intermediate data corresponding to the different weight matrices; and based on the intermediate data corresponding to the different weight matrices, obtain the intermediate processing data corresponding to the processing period.

In a possible implementation manner, each column of the PE array includes repeated feature elements; in the PE array 52, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature corresponding to the processing cycle. element and the corresponding target weight element and perform a preset operation to obtain intermediate processing data, which is used to: in response to the arrival of any processing cycle, transmit the number of target feature elements in the PE array column in the image feature matrix to be processed. To a row of PEs in the PE array, and copy the target feature elements in the row of PEs to the PEs of other rows, as the first operand of the corresponding PE; and match the target feature elements in each row of PEs The corresponding target weight elements from different weight matrices are transferred to the PE in the PE array corresponding to the position of the target feature element as the second operand of the corresponding PE; the PE array is used to store data in the PE array. Perform preset operations on the first operand and the second operand of , to obtain intermediate processing data corresponding to the processing cycle.

In a possible implementation manner, the PE array 52 uses the PE array to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the corresponding processing cycle. The intermediate processing data is used for: in the processing cycle, weighted summation is performed on each row of target feature elements in the first operand and each row of weight elements in the second operand, Obtain intermediate data corresponding to the different weight matrices; and obtain intermediate processing data corresponding to the processing period based on the intermediate data corresponding to the different weight matrices.

In a possible implementation manner, each PE in the PE array includes repeating feature elements; in the PE array 52, in response to the arrival of any processing cycle, each PE in the PE array obtains the target corresponding to the processing cycle. The feature elements and the corresponding target weight elements are subjected to a preset operation to obtain intermediate processing data, which are used for: in response to the arrival of any processing cycle, a target feature element in the feature matrix of the to-be-processed image is transmitted to the PE in one PE in the array, and copy the target feature element in the one PE to other PEs as the first operand of the corresponding PE; copy the target feature element corresponding to the one from the PE array The target weight elements of all weight matrices with different numbers of PEs are transferred to each PE of the PE array as the second operand of the corresponding PE; the first operation stored in the PE array is performed by the PE array A preset operation is performed on the number and the second operand to obtain the intermediate processing data corresponding to the processing cycle.

In a possible implementation manner, when the PE array 52 obtains the result data of processing the image feature matrix to be processed based on the intermediate processing data corresponding to the plurality of processing cycles, the PE array 52 is used to: Among the intermediate processing data corresponding to each processing cycle, the intermediate data belonging to the same weight matrix are accumulated to obtain the result value corresponding to each weight matrix; based on the result values corresponding to the multiple weight matrices, the feature of the image to be processed is obtained. Matrix to process the resulting data.

In a possible implementation manner, the preset operation corresponding to any processing cycle includes: a sub-operation of performing a fully connected operation on the feature matrix of the to-be-processed image.

For the description of the processing flow of each module in the apparatus and the interaction flow between the modules, reference may be made to the relevant descriptions in the foregoing method embodiments, which will not be described in detail here.

An embodiment of the present disclosure also provides a computer device. As shown in FIG. 6 , a schematic structural diagram of the computer device provided by the embodiment of the present disclosure includes: a processor 61 , a memory 62 , and a data processing apparatus 63 provided by the present disclosure.

The above-mentioned memory 62 includes a memory 621 and an external memory 622; the memory 621 here is also called an internal memory, and is used to temporarily store the operation data in the processor 61 and the data exchanged with the external memory 622 such as the hard disk. The external memory 622 performs data exchange.

For the specific execution process of the above instruction, reference may be made to the steps of the data processing method described in the embodiments of the present disclosure, and details are not repeated here.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the data processing method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be used to execute the steps of the data processing methods described in the foregoing method embodiments. For details, please refer to the foregoing methods. The embodiments are not repeated here.

Wherein, the above-mentioned computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure rather than limit them. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing The embodiments describe the present disclosure in detail. Those of ordinary skill in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims (22)

1. A data processing method, comprising:

   From the feature matrix of the image to be processed and the corresponding multiple weight matrices, determine the target feature element and the target weight element corresponding to the multiple processing cycles respectively;

   In response to the arrival of any processing cycle, each PE in the processing engine PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation to obtain intermediate processing data; wherein, for any processing cycle , the target feature elements in the PE array include repeated feature elements, and the target weight elements corresponding to the repeated feature elements are respectively the weight elements corresponding to the repeated feature elements in different weight matrices;

   Based on the intermediate processing data corresponding to the plurality of processing cycles, the result data of processing the feature matrix of the image to be processed is obtained.
2. The data processing method according to claim 1, wherein before determining the target feature elements and target weight elements corresponding to the multiple processing cycles respectively, the method further comprises:

   Based on the size of the PE array, size transformation is performed on the original to-be-processed image feature matrix and the original weight matrix to obtain the to-be-processed image feature matrix and the weight matrix.
3. The data processing method according to claim 1 or 2, characterized in that:

   The target feature elements corresponding to the multiple processing cycles respectively include at least one image feature element in the to-be-processed image feature matrix;

   The position of the target feature element in the feature matrix of the image to be processed is consistent with the position of the corresponding target weight element in the corresponding weight matrix.
4. The data processing method according to any one of claims 1 to 3, wherein, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target Weight elements and perform preset operations to obtain intermediate processing data, including:

   In response to the arrival of any processing cycle, the Ar target feature elements in the feature matrix of the to-be-processed image are transferred to one column of PEs in the PE array, and the target feature elements in the one column of PEs are copied to PEs in other columns , as the first operand of the corresponding PE, where Ar represents the row number of the PE array; and

   The target weight elements corresponding to the target feature elements in each column of PEs and from different weight matrices are transferred to the PEs corresponding to the target feature element positions in the PE array, as the second operand of the corresponding PE;

   Using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array, intermediate processing data corresponding to the processing cycle is obtained.
5. The data processing method according to claim 4, wherein the PE array is used to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the processing The intermediate processing data corresponding to the period, including:

   In the processing cycle, weighted summation is performed on each column of target feature elements in the first operand and each column of weight elements in the second operand to obtain the corresponding value of the different weight matrices. intermediate data;

   Based on the intermediate data corresponding to the different weight matrices, the intermediate processing data corresponding to the processing period is obtained.
6. The data processing method according to any one of claims 1 to 3, wherein, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target Weight elements and perform preset operations to obtain intermediate processing data, including:

   In response to the arrival of any processing cycle, the Ac target feature elements in the feature matrix of the image to be processed are transferred to a row of PEs in the PE array, and the target feature elements in the row of PEs are copied to PEs in other rows , as the first operand of the corresponding PE, where Ac represents the number of columns of the PE array; and

   The target weight elements corresponding to the target feature elements in each row of PEs and from different weight matrices are transferred to the PEs corresponding to the target feature element positions in the PE array, as the second operand of the corresponding PE;

   Using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array, intermediate processing data corresponding to the processing cycle is obtained.
7. The data processing method according to claim 6, wherein the PE array is used to perform a preset operation on the first operand and the second operand stored in the PE array to obtain the processing The intermediate processing data corresponding to the period, including:

   In the processing cycle, weighted summation is performed on each row of target feature elements in the first operand and each row of weight elements in the second operand to obtain the corresponding value of the different weight matrices. intermediate data;

   Based on the intermediate data corresponding to the different weight matrices, the intermediate processing data corresponding to the processing period is obtained.
8. The data processing method according to claim 3, wherein, in response to the arrival of any processing cycle, each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element, and performs pre-processing steps. Set the operation to obtain the intermediate processing data, including:

   In response to the arrival of any processing cycle, a target feature element in the feature matrix of the image to be processed is transferred to a PE of the PE array, and the target feature element in the one PE is copied to other PEs , as the first operand of the corresponding PE;

   Transfer the target weight elements from weight matrices with different Acl numbers corresponding to the one target feature element to each PE of the PE array as the second operand of the corresponding PE, where Acl represents the PE array The number of all PEs in;

   Using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array, intermediate processing data corresponding to the processing cycle is obtained.
9. The data processing method according to any one of claims 1 to 8, wherein the result data of processing the image feature matrix to be processed is obtained based on the intermediate processing data corresponding to the plurality of processing cycles respectively ,include:

   Accumulate the intermediate data belonging to the same weight matrix in the intermediate processing data corresponding to the multiple processing cycles respectively, to obtain the result value corresponding to each weight matrix;

   Based on the result values corresponding to the multiple weight matrices respectively, the result data of processing the feature matrix of the image to be processed is obtained.
10. The data processing method according to any one of claims 1 to 9, wherein the preset operation includes a sub-operation of performing a full connection operation on the feature matrix of the image to be processed.
11. A data processing device, comprising a controller and a processing engine PE array;

    The controller is configured to determine, from the feature matrix of the image to be processed and a plurality of corresponding weight matrices, target feature elements and target weight elements corresponding to a plurality of processing cycles respectively;

    The PE array is used to respond to the arrival of any processing cycle, and each PE in the PE array obtains the target feature element corresponding to the processing cycle and the corresponding target weight element and performs a preset operation to obtain intermediate processing data; based on The intermediate processing data corresponding to the multiple processing cycles respectively, to obtain the result data of processing the feature matrix of the to-be-processed image;

    Wherein, for any processing cycle, the target feature elements in the PE array include repeated feature elements, and the target weight elements corresponding to the repeated feature elements are weight elements corresponding to the repeated feature elements in different weight matrices, respectively.
12. The data processing device according to claim 11, wherein the controller, before determining the target feature elements and target weight elements corresponding to the plurality of processing cycles respectively, is further configured to:

    Based on the size of the PE array, size transformation is performed on the original to-be-processed image feature matrix and the original weight matrix to obtain the to-be-processed image feature matrix and the weight matrix.
13. The data processing device according to claim 11 or 12, wherein the target feature elements corresponding to the multiple processing cycles respectively include at least one image feature element in the to-be-processed image feature matrix;

    The position of the target feature element in the feature matrix of the image to be processed is consistent with the position of the corresponding target weight element in the corresponding weight matrix.
14. The data processing device according to any one of claims 11 to 13, wherein each PE in the PE array, in response to the arrival of any processing cycle, acquires the target feature element corresponding to the processing cycle and the corresponding Target weight elements and perform preset operations to obtain intermediate processing data, used for:

    In response to the arrival of any processing cycle, the Ar target feature elements in the feature matrix of the to-be-processed image are transferred to one column of PEs in the PE array, and the target feature elements in the one column of PEs are copied to PEs in other columns , as the first operand of the corresponding PE, where Ar represents the row number of the PE array; and

    The target weight elements corresponding to the target feature elements in each column of PEs and from different weight matrices are transferred to the PEs corresponding to the target feature element positions in the PE array, as the second operand of the corresponding PE;

    Using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array, intermediate processing data corresponding to the processing cycle is obtained.
15. The data processing apparatus according to claim 14, wherein, in the PE array, the PE array is used to perform a preset operation on the first operand and the second operand stored in the PE array to obtain When the intermediate processing data corresponding to the processing cycle is used, it is used for:

    In the processing cycle, weighted summation is performed on each column of target feature elements in the first operand and each column of weight elements in the second operand to obtain the corresponding value of the different weight matrices. intermediate data;

    Based on the intermediate data corresponding to the different weight matrices, the intermediate processing data corresponding to the processing period is obtained.
16. The data processing apparatus according to any one of claims 11 to 13, wherein each PE in the PE array, in response to the arrival of any processing cycle, acquires the target feature element corresponding to the processing cycle and the corresponding The target weight element of , and the preset operation is performed to obtain intermediate processing data, which is used for:

    In response to the arrival of any processing cycle, the Ac target feature elements in the feature matrix of the image to be processed are transferred to a row of PEs in the PE array, and the target feature elements in the row of PEs are copied to PEs in other rows , as the first operand of the corresponding PE, where Ac represents the number of columns of the PE array; and

    The target weight elements corresponding to the target feature elements in each row of PEs and from different weight matrices are transferred to the PEs corresponding to the target feature element positions in the PE array, as the second operand of the corresponding PE;

    Using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array, intermediate processing data corresponding to the processing cycle is obtained.
17. The data processing apparatus according to claim 16, wherein the PE array is used to perform a preset operation on the first operand and the second operand stored in the PE array by using the PE array, When obtaining the intermediate processing data corresponding to the processing cycle, it is used for:

    In the processing cycle, weighted summation is performed on each row of target feature elements in the first operand and each row of weight elements in the second operand to obtain the corresponding value of the different weight matrices. intermediate data;

    Based on the intermediate data corresponding to the different weight matrices, the intermediate processing data corresponding to the processing period is obtained.
18. The data processing apparatus according to claim 13, wherein each PE in the PE array, in response to the arrival of any processing cycle, acquires the target feature element and the corresponding target weight element corresponding to the processing cycle, and When performing preset operations to obtain intermediate processing data, it is used to:

    In response to the arrival of any processing cycle, a target feature element in the feature matrix of the image to be processed is transferred to a PE of the PE array, and the target feature element in the one PE is copied to other PEs , as the first operand of the corresponding PE;

    Transfer the target weight elements from weight matrices with different Acl numbers corresponding to the one target feature element to each PE of the PE array as the second operand of the corresponding PE, where Acl represents the PE array The number of all PEs in;

    Using the PE array to perform a preset operation on the first operand and the second operand stored in the PE array, intermediate processing data corresponding to the processing cycle is obtained.
19. The data processing apparatus according to any one of claims 11 to 18, wherein the PE array is based on the intermediate processing data corresponding to a plurality of the processing cycles to obtain the feature matrix of the image to be processed. When processing the resulting data, use:

    Accumulate the intermediate data belonging to the same weight matrix in the intermediate processing data corresponding to the multiple processing cycles respectively, to obtain the result value corresponding to each weight matrix;

    Based on the result values corresponding to the multiple weight matrices respectively, the result data of processing the feature matrix of the image to be processed is obtained.
20. The data processing apparatus according to any one of claims 11 to 19, wherein the preset operation includes: a sub-operation of a fully connected operation on the feature matrix of the image to be processed.
21. A computer device, characterized in that it comprises: a processor, a memory, and the data processing device according to any one of claims 11 to 20.
22. A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and the computer program is executed by the controller and the PE array when running as described in any one of claims 1 to 10. The steps of the data processing method described.

Images