WO2021179224A1

WO2021179224A1 - Data processing device, data processing method and accelerator

Info

Publication number: WO2021179224A1
Application number: PCT/CN2020/078876
Authority: WO
Inventors: 韩峰; 李鹏
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-03-11
Filing date: 2020-03-11
Publication date: 2021-09-16
Also published as: CN112602094A

Abstract

The embodiments of the present application provide a data processing device, a data processing method, and an accelerator. Said device comprises a control module, a data loading module and a processing module; the data loading module loads, in response to a control instruction of the control module, data to be processed so as to allow the processing module to process same; the processing module processes, in response to a control instruction of the control module, said data; and the control module controls the data loading module and the processing module to execute different control instructions at the same time. In this embodiment, the control module controls the data loading module and the processing module to execute different control instructions at the same time, increasing the utilization rate of processing resources and avoiding waste of processing resources caused by a waiting process.

Description

Data processing device, data processing method and accelerator

Technical field

This application relates to the field of computer data processing, and in particular to a data processing device, a data processing method, and an accelerator.

Background technique

With the advancement of technology, all kinds of products realize algorithms or program products are developing in the direction of refinement, making the processing process of all kinds of products realizing algorithms or program products more complicated and cumbersome, which requires a lot of calculation or processing resources, so how to ensure the processing Or the comprehensive utilization of computing resources has become a technical problem to be solved urgently.

As an example, a convolutional neural network (Convolutional Neural Network, CNN) is a complex and non-linear hypothetical model. The model parameters used are obtained through training and learning, and it has the ability to fit data. Convolutional neural network algorithms can be applied to scenarios such as machine vision and natural language processing. When the CNN algorithm is implemented in an embedded system, due to the large resource consumption of neural network processing, it is necessary to fully consider computing resources and real-time performance. Therefore, it is necessary to improve the computing resource utilization rate of neural network processing.

Summary of the invention

In view of this, one of the objectives of the embodiments of the present application is to provide a data processing device, a data processing method, and an accelerator.

First, according to the first aspect of the embodiments of the present application, a data processing device is provided, which includes a control module, a data loading module, and a processing module;

The data loading module, in response to the control instruction of the control module, loads the data to be processed for processing by the processing module;

The processing module responds to the control instruction of the control module to process the data to be processed;

The control module controls the data loading module and the processing module to execute different control instructions at the same time.

According to a second aspect of the embodiments of the present application, a data processing method is provided, which is applied to a data processing device, the data processing device includes a data loading module and a processing module; the method includes:

In response to the control instruction, load data through the data loading module for the processing module to perform data processing; and,

In response to the control instruction, data processing is performed by the processing module; wherein the data loading module and the processing module execute different control instructions at the same time.

According to a third aspect of the embodiments of the present application, an accelerator is provided, including the device described in any one of the first aspect.

The embodiments of the present application have the following beneficial effects:

In this embodiment, the control module controls the data loading module and the processing module to execute different control instructions at the same time. When the processing module processes the data to be processed corresponding to the current control instruction, the data loading The module may be loading the to-be-processed data corresponding to the next control instruction, which is beneficial to improve the utilization of processing resources and avoid the waste of processing resources caused by the waiting process.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.

Fig. 1 is a schematic structural diagram of a first data processing device according to an exemplary embodiment of the present application.

Fig. 2 is a schematic structural diagram of a second data processing device according to an exemplary embodiment of the present application.

Fig. 3 is a schematic diagram showing an execution flow of a control instruction according to an exemplary embodiment of the present application.

Fig. 4 is a schematic structural diagram of a third data processing device according to an exemplary embodiment of the present application.

Fig. 5 is a schematic structural diagram of a fourth data processing device according to an exemplary embodiment of the present application.

Fig. 6 is a schematic diagram showing the execution of the first control instruction according to an exemplary embodiment of the present application.

Fig. 7 is a schematic diagram showing the execution of a second type of control instruction according to an exemplary embodiment of the present application.

Fig. 8 is a schematic structural diagram of a fifth data processing device according to an exemplary embodiment of the present application.

Fig. 9 is a schematic diagram showing an execution flow of a control instruction according to an exemplary embodiment of the present application.

Fig. 10 is a schematic diagram showing the execution of a third control instruction according to an exemplary embodiment of the present application.

Fig. 11 is a schematic flowchart of a data processing method according to an exemplary embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Based on a problem in the related art, please refer to FIG. 1. An embodiment of the present application provides a data processing apparatus. FIG. 1 is a schematic structural diagram of a first data processing apparatus according to an exemplary embodiment of the present application. The device includes: a control module 11, a data loading module 12, and a processing module 13. In another embodiment, the data loading module is not limited to one. In other words, there may be multiple data loading modules. For example, if the data processing device is used for data processing of convolution operation, the data processing device includes two data loading modules, namely a feature map loading module and a weight loading module.

The data loading module 12, in response to the control instruction of the control module 11, loads the data to be processed for the processing module 13 to process.

The processing module 13 responds to the control instruction of the control module 11 to process the data to be processed.

The control module 11 controls the data loading module 12 and the processing module 13 to execute different control instructions at the same time. In another embodiment, the control module 11 controls the data loading module 12 to process the control instruction in advance, without waiting for the processing of the previous control instruction in the entire data processing device to end. For example, the control module 11 does not need to wait for the end of the processing of the control instruction x0 in the processing module 13, the control module 11 can control the data loading module 12 to process the operation corresponding to the control instruction x1 in advance, where the control instruction x0 and the control instruction x1 are executed in sequence Control instruction.

In this embodiment, the control module 11 receives a control instruction from an external module, and sends the control instruction to the data loading module 12 and the processing module 13; the data loading module 12 responds to the control Instruction to load the to-be-processed data for processing by the processing module 13; the processing module 13 to process the to-be-processed data in response to the control instruction; wherein, in order to further improve the comprehensive utilization rate of processing resources, the The control module 11 can control the data loading module 12 and the processing module 13 to execute different control instructions at the same time. That is, the control module 11 can control the data loading module 12 and the processing module 13 to execute at the same time. Not the same control instruction, after the data loading module 12 has loaded the data to be processed corresponding to the current control instruction, there is no need to wait for the processing module 13 to process the data to be processed corresponding to the current control instruction. The loading module 12 can directly load the data corresponding to the next control instruction based on the control of the control module 11, that is, when the processing module 13 processes the data to be processed corresponding to the current control instruction, the data is loaded The module 12 is loading the to-be-processed data corresponding to the next control instruction, thereby helping to improve the utilization rate of processing resources and avoid the waste of processing resources caused by the waiting process.

In one embodiment, the control module 11 sends the i+1th control instruction to the data loading module 12 in response to the data loading module 12 completing the execution of the i-th control instruction; and in response to the The processing module 13 finishes executing the i-th control instruction, and sends the i+1-th control instruction to the processing module 13; where i is an integer.

It should be noted that the completion of the execution of the i-th control instruction by the data loading module 12 means that the data loading module 12 has finished loading the data to be processed corresponding to the i-th control instruction; the processing module 13 controls the i-th control instruction. The completion of the instruction execution means that the processing module 13 has processed the to-be-processed data corresponding to the i-th control instruction.

The data loading module 12 loads the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction. In response to the i-th control instruction, the processing module 13 processes the to-be-processed data corresponding to the i-th control instruction to obtain a processing result. Wherein, when the processing module 13 processes the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction, if the data loading module 12 has finished loading the i-th control instruction After the data to be processed corresponding to the instruction, there is no need to wait for the processing module 13 to complete the execution of the i-th control instruction, and the data loading module 12 can directly receive the i+1-th control instruction sent by the control module 11 , And load the data to be processed corresponding to the i+1th control instruction in response to the i+1th control instruction; this embodiment further reduces the time for the data loading module 12 to wait for the next control instruction to avoid waiting time To deal with the waste of resources.

In one embodiment, the processing module 13 includes a systolic array; in response to the i-th control instruction, the processing module 13 writes the to-be-processed data corresponding to the i-th control instruction into the systolic array. The systolic array performs operations on the to-be-processed data to obtain the processing result. In this embodiment, the data process of the data to be processed is realized through the hardware structure, which is beneficial to improve the processing efficiency.

Please refer to FIG. 2, which is a schematic structural diagram of a second data processing device according to an exemplary embodiment of this application. The device includes: a control module 11, a data loading module 12, a processing module 13, and a data writing back module 14.

The data write-back module 14 writes the processing result of the data to be processed into the external storage module in response to the control instruction of the control module 11.

The control module 11 controls the data loading module 12, the processing module 13 and the data writing back module 14 to execute different control instructions at the same time.

In this embodiment, the control module 11 receives a control instruction from an external module, and sends the control instruction to the data loading module 12, the processing module 13, and the data write-back module 14; the data The loading module 12 loads the data to be processed by the processing module 13 in response to the control instruction; the processing module 13 processes the data to be processed in response to the control instruction to obtain a processing result; In response to the control instruction, the data write-back module 14 writes the processing result into the external storage module.

Among them, in order to further improve the comprehensive utilization of processing resources, the control module 11 may control the data loading module 12, the processing module 13 and the data write-back module 14 to execute different control instructions at the same time, that is, The control module 11 can control the data loading module 12, the processing module 13 and the data write-back module 14 to execute a non-same control instruction at the same time; the data loading module 12 is different from the current control instruction after loading. After the corresponding data to be processed, there is no need to wait for the processing module 13 to process the data to be processed corresponding to the current control instruction, and the data loading module 12 can directly load the next control instruction based on the control of the control module 11 Corresponding to-be-processed data; after the processing module 13 processes the data corresponding to the current control instruction, there is no need to wait for the data write-back module 14 to write the processing result corresponding to the current control instruction into the external storage module, so The processing module 13 can directly process the data to be processed corresponding to the next control instruction based on the control of the control module 11; that is, the data write-back module 14 is writing the processing result corresponding to the previous control instruction When the external storage module is used, the processing module 13 may be processing the data corresponding to the current control instruction, and the data loading module 12 may be loading the data corresponding to the next control instruction, thereby helping to improve the utilization of processing resources and avoid waiting The process causes a waste of processing resources.

In one embodiment, the control module 11 sends the i+1th control instruction to the data loading module 12 in response to the data loading module 12 completing the execution of the i-th control instruction; and in response to the The processing module 13 completes the execution of the i-th control instruction and sends the i+1th control instruction to the processing module 13; and, in response to the data write-back module 14 completing the execution of the i-th control instruction, sends the i+1 control instructions are sent to the data write-back module 14; where i is an integer.

It should be noted that the completion of the execution of the i-th control instruction by the data loading module 12 means that the data loading module 12 has finished loading the data to be processed corresponding to the i-th control instruction; the processing module 13 controls the i-th control instruction. The completion of the instruction execution means that the processing module 13 has processed the to-be-processed data corresponding to the i-th control instruction; the completion of the data write-back module 14 execution of the i-th control instruction means that the data write-back module 14 is completed The operation of writing the processing result corresponding to the i-th control instruction into the external storage module.

The data loading module 12 loads the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction. In response to the i-th control instruction, the processing module 13 processes the to-be-processed data corresponding to the i-th control instruction to obtain a processing result. In response to the i-th control instruction, the data write-back module 14 writes the processing result corresponding to the i-th control instruction into the external storage module.

Wherein, when the processing module 13 processes the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction, if the data loading module 12 has finished loading the i-th control instruction After the data to be processed corresponding to the instruction, there is no need to wait for the processing module 13 to complete the execution of the i-th control instruction, and the data loading module 12 can directly receive the i+1-th control instruction sent by the control module 11 , And load the to-be-processed data corresponding to the i+1th control instruction; this embodiment further reduces the waiting time of the data loading module 12, and avoids waste of processing resources caused by the waiting time.

Further, when the data write-back module 14 writes the processing result corresponding to the i-th control instruction into the external storage module in response to the i-th control instruction, if the processing module 13 has processed the The data to be processed corresponding to the i-th control instruction does not need to wait for the data write-back module 14 to complete the execution of the i-th control instruction, and the processing module 13 can directly receive the i+th sent by the control module 11. 1 control instruction, and in response to the i+1 control instruction, process the data to be processed corresponding to the i+1 control instruction; this embodiment further reduces the waiting time of the data processing module 13 , To avoid waste of processing resources caused by waiting time.

In an example, please refer to FIG. 3, which is a schematic diagram showing the processing time of the data loading module 12, the processing module 13, and the data writing back module 14; the data loading module 12 has finished loading the control instruction 1 After the corresponding to-be-processed data, there is no need to wait for the processing module 13 to process the to-be-processed data corresponding to the control instruction 1 and the data write-back module 14 to write the processing result corresponding to the control instruction 1, you can directly obtain control Instruction 2 and load the data to be processed corresponding to the control instruction 2, thereby further reducing the time for the data loading module 12 to wait for the next control instruction, and avoiding the waste of processing resources caused by the waiting time; accordingly, the processing module 13 is processing After finishing the to-be-processed data corresponding to the control instruction 1, after obtaining the processing result, without waiting for the data write-back module 14 to write the processing result into the external storage module, you can directly obtain the control instruction 2 and process the pending data corresponding to the control instruction 2. The data is processed, thereby further reducing the time for the processing module 13 to wait for the next control instruction, and avoiding the waste of processing resources caused by the waiting time.

It can be understood that the embodiment of the present application does not impose any restriction on the specific type of the external storage module, and specific settings can be made according to actual application scenarios. For example, it can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable and programmable Read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

In an exemplary embodiment, the data processing device can be applied to related processing based on a convolutional neural network, which is a machine learning algorithm, which is widely used in target recognition, target detection, and image semantics. Computer vision tasks such as segmentation. The structure of a neural neural network usually includes an input layer, one or more hidden layers, and an output layer. The operations in the hidden layer include, but are not limited to, convolution operations, pooling operations, or activation operations; in general, convolution The hidden layers in the neural network can be named according to the type of operation. For example, the hidden layer for convolution operation can be classified as a convolutional layer, and the hidden layer for pooling operation can be classified as a pooling layer or a hidden layer for activation operation. Can be classified as an active layer. The data processing device provided by the embodiment of the present application can perform convolution operation of the convolutional layer, pooling operation of the pooling layer, and activation operation of the activation layer, so as to accelerate the operation process of the deep neural network through hardware and reduce the depth. The calculation time of the neural network improves the calculation efficiency.

Wherein, the objects processed by the convolutional neural network include but are not limited to images, audio or text, etc., and different types of hidden layers in the convolutional neural network correspond to different operating parameters, such as the operating parameters of the convolutional layer It is a convolution kernel, the operating parameter of the activation layer is an activation function, and the operating parameter of the pooling layer is a pooling parameter.

Hereinafter, the convolutional neural network is applied to the field of image processing, and the data processing device is used to perform the convolution operation of the convolutional layer in the convolutional neural network as an example for description: the convolutional operation process of the convolutional layer is Perform vector inner product operations on the image to be processed based on the convolution kernel to obtain a feature map; then in this scenario, the control module 11 sends the data loading module 12, the processing module 13, and the data write-back module 14 respectively Sending a convolution operation control instruction, the data loading module 12 loads the image to be processed and the convolution kernel in response to the convolution operation control instruction of the control module 11; the processing module 13 responds to the convolution operation of the control module 11 The product operation control instruction is to perform a convolution operation on the image to be processed through the convolution kernel to obtain a feature map; the data write-back module 14 responds to the convolution operation control instruction of the control module 11 to convert the feature Figure is written into an external storage module; where the control module 11 controls the data loading module 12, the processing module 13 and the data write-back module 14 to execute different convolution operation control instructions at the same time, through Convolution operation controls the parallel processing of instructions, which effectively improves operation efficiency and at the same time increases the utilization rate of computing resources.

Further, in order to improve the efficiency of data loading, the data loading module 12 may include an object loading unit and a parameter loading unit. The object loading unit is used to load the image to be processed, and the parameter loading unit is used to load the volume. The product core, the object loading unit and the parameter loading unit are loaded at the same time, which is beneficial to improve the loading efficiency.

In an embodiment, the control module 11 includes at least one instruction slot for caching the control instruction. In one embodiment, the instruction slot includes a set of instruction cache flags and control status signals. Among them, the instruction cache flag is used to indicate whether the instruction cached in the instruction slot is valid. For example, the instruction cache flag can indicate which control instruction is cached by the instruction slot and whether the cached control instruction is valid. The set of control state signals is used to indicate the working state of the corresponding module, or indicate the working state of other instruction slots related to the instruction slot. In one embodiment, the working status refers to whether the processing or operation of the corresponding module is completed, or whether the instruction cache operation of the corresponding instruction slot is completed. Through the setting of at least one instruction slot, the execution of each instruction can be coordinated, so that each sub-module in the data processing device can process operations corresponding to different instructions at the same time. Therefore, the work efficiency of the data processing device is improved.

In response to the completion of the execution of the i-th control instruction by the data loading module 12, the control module 11 sends the i+1th control instruction buffered in the instruction slot to the data loading module 12; and in response to all The processing module 13 completes the execution of the i-th control instruction, and sends the i+1-th control instruction cached in the instruction slot to the processing module 13; and responds to the data write-back module 14 for the i-th control instruction After the control instruction is executed, the i+1th control instruction buffered in the instruction slot is sent to the data write-back module 14.

Further, the command slot can record the status of the control command cached in the command slot, and the control module 11 can control the execution process of the control command according to the status of the control command recorded in at least one command slot. In an example, it is assumed that the data loading module 12, the processing module 13 and the data writing back module 14 each correspond to an instruction slot, and one of the instruction slots corresponds to the data loading module 12 as an example for description. When the control After the module 11 sends the i-th control instruction in the instruction slot to the data loading module 12, correspondingly, the state of the control instruction recorded in the instruction slot indicates that the i-th control instruction is invalid. At this time, The control module 11 can cache the i+1th control instruction to the instruction slot. Correspondingly, after the i+1th control instruction caches the value of the instruction slot, the state of the control instruction recorded in the instruction slot is changed to indicate that the i+1th control instruction is valid, which indicates that the i+th control instruction is valid. One control command has not been sent to the data loading module 12 yet. In this embodiment, the state of the control instruction recorded in the instruction slot ensures that the control module 11 can cache different control instructions in an orderly manner, so as to ensure that the data loading module 12, the processing module 13 and the data The write-back module 14 sends different control commands in an orderly manner.

In an implementation manner, the state of the control module 11 recorded in the instruction slot may be represented by a control instruction state signal, which indicates whether the control instruction buffered in the corresponding instruction slot is valid. In an example, when the control instruction status signal in the instruction slot indicates that the i-th control instruction cached in the instruction slot is invalid, the control module 11 can cache the i+1-th control instruction to the instruction At the same time, the status value of the control command status signal is set to a value that characterizes validity, which is used to indicate that the i+1th control command cached in the command slot is valid.

Wherein, the data loading module 12, the processing module 13 and the data writing back module 14 respectively correspond to an instruction slot, whether the control instruction cached in the instruction slot is valid, and whether the module corresponding to the at least one instruction slot is completed The operation of the control instruction corresponding to the instruction slot is related.

As one of the implementation manners, a control instruction completion signal may be used to indicate whether the module corresponding to the at least one instruction slot has completed the operation of the control instruction corresponding to the at least one instruction slot; wherein, the "at least one instruction slot corresponds to "Whether the module of "has completed the operation of the control instruction corresponding to the at least one instruction slot" refers to whether the module corresponding to the at least one instruction slot has completed receiving the control instruction corresponding to the instruction slot.

The control module 11 includes at least one instruction slot for buffering the control instruction; the instruction slot includes a control instruction status signal and a control instruction completion signal; wherein, when the control instruction completion signal instructs the corresponding module to complete the corresponding When the control instruction is operated, the control instruction status signal indicates that the control instruction cached in the corresponding instruction slot is invalid; otherwise, the control instruction status signal indicates that the control instruction cached in the corresponding instruction slot is valid. In this embodiment, through the control command status signal and the control command completion signal, it is ensured that the control module 11 can cache different control commands in an orderly manner, so as to ensure that the data loading module 12, the processing module 13 and the data writing back module 14 are available. Send different control commands in sequence.

Further, the data loading module 12, the processing module 13 and the data write-back module 14 respectively correspond to an instruction slot, and the caching process of the control instruction in the instruction slot corresponds to the data loading module 12, The processing module 13 and the data write-back module 14 are transferred in sequence; in one example, the control module 11 includes a second instruction slot 111 corresponding to the data loading module 12, corresponding to the first instruction slot of the data processing module 13 The third instruction slot 112 and the fourth instruction slot 113 corresponding to the data write-back module 14; when the i-th control instruction in the second instruction slot 111 is invalid, the i+1th control instruction is cached to the second instruction slot 111. When the i-th control instruction in the third instruction slot 112 is invalid, the i+1-th control instruction cached in the second instruction slot 111 is cached to the third instruction slot 112, and when the i-th control instruction in the fourth instruction slot 113 is When a control instruction is invalid, the i+1th control instruction cached in the third instruction slot 112 is cached to the fourth instruction slot 113. It can be seen that the i+1th control instruction is sequentially cached in the second instruction slot 111. In the third instruction slot 112 and the fourth instruction slot 113.

Then whether the control instruction cached in the instruction slot is valid, in addition to whether the module corresponding to the at least one instruction slot completes the operation of the control instruction corresponding to the instruction slot, is also related to whether the at least one instruction slot completes the corresponding control The operation of the instruction is related; the "whether at least one instruction slot completes the operation of the corresponding control instruction" refers to whether the control instruction in the instruction slot is cached in another instruction slot.

In an implementation manner, two control instruction completion signals can be set, one of which is used to indicate whether the module corresponding to the at least one instruction slot has completed the operation of the control instruction corresponding to the at least one instruction slot, and the other A control instruction completion signal is used to indicate whether the at least one instruction slot has completed the operation of the corresponding control instruction.

The control module 11 includes at least one instruction slot for buffering the control instruction; the instruction slot includes a control instruction status signal and at least two control instruction completion signals; wherein, when one of the control instruction completion signals indicates the corresponding When the module completes the operation of the corresponding control instruction, and another control instruction completion signal indicates that the at least one instruction slot completes the operation of the corresponding control instruction, the control instruction status signal indicates that the control instruction buffered in the corresponding instruction slot is invalid; Otherwise, the control command status signal indicates that the control command buffered in the corresponding command slot is valid. This embodiment uses the control command status signal and the two control command completion signals to ensure that the control module 11 can cache different control commands in an orderly manner to ensure that the data loading module 12, the processing module 13 and the data write-back module 14 Send different control commands in an orderly manner.

In an example, referring to FIG. 4, the control module 11 includes a second instruction slot 111 corresponding to the data loading module 12, a third instruction slot 112 corresponding to the data processing module 13 and corresponding to the The data is written back to the fourth instruction slot 113 of the module 14. In one embodiment, the second instruction slot 111, the third instruction slot 112, and the fourth instruction slot 113 are all used to: after the control module 11 has sent the i-th control instruction at different times, respectively, cache the corresponding The i+1th control instruction. In another embodiment, the instruction slot can cache the instruction control word corresponding to the control instruction after decoding, and the corresponding control instruction does not need to be cached. The second instruction slot 111 is used to buffer the i+1th control instruction sent to the data loading module 12 after the control module 11 has sent the i-th control instruction; The three instruction slot 112 is used to buffer the i+1th control instruction sent to the processing module 13 after the control module 11 has sent the i-th control instruction; the fourth instruction slot 113 It is used to buffer the (i+1)th control instruction sent to the data write-back module 14 after the control module 11 finishes sending the i-th control instruction. It should be noted that the time points at which the i+1th control instruction is cached in each of the above-mentioned instruction slots may be different. The "sending the i-th control instruction" means that the control module 11 is sending the i-th control instruction to the data loading module 12 and the third instruction slot 111 corresponding to the After the data write-back module 14 corresponding to the processing module 13 or the fourth instruction slot 113 corresponding to the instruction slot 112, the data loading module 12, the processing module 13, or the data write-back module 14 is completed Receive the i-th control instruction.

For the second instruction slot 111, the control module 11 caches the i+1th control instruction to the second instruction slot 111 in response to the i-th control instruction in the second instruction slot 111 being invalid; And in response to the completion of the execution of the i-th control instruction by the data loading module 12, the i+1-th control instruction in the second instruction slot 111 is sent to the data loading module 12.

Wherein, the second command slot 111 includes a second valid signal, a data loading module completion signal, and a third command slot completion signal; the second valid signal is used to instruct the control of the buffer in the second command slot 111 Whether the instruction is valid; the data loading module completion signal is used to indicate whether the data loading module 12 has completed receiving the control instruction (that is, whether the control module 11 has sent the control instruction to the data loading module 12) The third instruction slot completion signal is used to indicate whether the third instruction slot 112 has completed caching the control instruction (that is, whether the control module 11 has cached the cache instruction in the second instruction slot 111 To the third instruction slot 112).

When the i-th control instruction has been sent to the data loading module 12 and cached in the third instruction slot 112, the status values of the data loading module completion signal and the third instruction slot completion signal are set to At the same time, the state value of the second valid signal is set to a value representing invalid. At this time, the control module 11 may cache the i+1th control instruction to the first The second instruction slot 111; when the i+1th control instruction is cached in the second instruction slot 111, the i+1th control instruction has not been sent to the data loading module 12 and cached to the The third command slot 112, therefore, the state value of the second valid signal is set to a value representing valid, and the state values of the data loading module completion signal and the third command slot completion signal are set to be representing not The completed value.

For the third instruction slot 112, the control module 11, in response to the i-th control instruction in the third instruction slot 112 being invalid, caches the i+1-th control instruction in the second instruction slot 111 to all The third instruction slot 112; and in response to the processing module 13 completing the execution of the i-th control instruction, the i+1th control instruction in the third instruction slot 112 is sent to the processing module 13.

Wherein, the third instruction slot 112 includes a third valid signal, a processing module completion signal, and a fourth instruction slot 113 completion signal. The third valid signal is used to indicate whether the control instruction buffered in the third instruction slot 112 is valid; the processing module completion signal is used to indicate whether the processing module 13 has completed receiving the control instruction (that is, the Whether the control module 11 has sent the control instruction to the processing module 13); the fourth instruction slot 113 completion signal is used to indicate whether the fourth instruction slot 113 has completed buffering the control instruction (that is, the Whether the control module 11 has cached the cache instruction in the third instruction slot 112 to the fourth instruction slot 113).

When the i-th control instruction has been sent to the processing module 13 and has been cached in the fourth instruction slot 113, the status values of the processing module completion signal and the fourth instruction slot 113 completion signal are set to represent At the same time, the state value of the third valid signal is set to a value characterizing invalidity, and the control module 11 may cache the i+1th control instruction to the third instruction slot 112 ; When the i+1th control instruction is cached to the third instruction slot 112, the i+1th control instruction has not been sent to the processing module 13 and cached to the fourth instruction slot 113 The state value of the third valid signal is set to a value that characterizes validity, and the state value of the processing module completion signal and the state value of the fourth command slot 113 completion signal is set to a value that characterizes incompleteness.

For the fourth instruction slot 113, in response to the i-th control instruction in the fourth instruction slot 113 being invalid, the control module 11 caches the i+1-th control instruction in the third instruction slot 112 to all The fourth instruction slot 113; and in response to the completion of the execution of the i-th control instruction by the data write-back module 14, the i+1th control instruction in the fourth instruction slot 113 is sent to the processing module 13 .

Wherein, the fourth command slot 113 includes a fourth valid signal and a data write-back module 14 completion signal; the fourth valid signal is used to indicate whether the control command buffered in the fourth command slot 113 is valid; The data write-back module 14 completion signal is used to indicate whether the data write-back module 14 has completed receiving the control instruction (that is, whether the control module 11 has sent the control instruction to the data write-back module 14).

When the i-th control command has been sent to the data write-back module 14, the status value of the completion signal of the data write-back module 14 is set to the value that characterizes the completion, and at the same time, the status of the fourth valid signal The value is set to a value that characterizes invalidity. The control module 11 may cache the i+1th control instruction to the fourth instruction slot 113; when the i+1th control instruction is cached to the fourth instruction slot 113; In the case of four command slots 113, the i+1 has not been sent by the control module 11 to the data write-back module 14. At this time, the state value of the fourth valid signal is set to a valid value, and The state value of the completion signal of the data write-back module 14 is set to a value representing incompleteness.

In this embodiment, by setting the second instruction slot 111 corresponding to the data loading module 12 in the control module 11, the third instruction slot 112 corresponding to the processing module 13 and corresponding to the data write-back The fourth instruction slot 113 of the module 14 enables the control module 11 to load the data module 12, the processing module 13 and the data through the second instruction slot 111, the third instruction slot 112, and the fourth instruction slot 113, respectively. The data write-back module 14 performs control to ensure that the data loading module 12, the processing module 13 and the data write-back module 14 can execute different control commands at the same time, thereby reducing the corresponding waiting time and benefiting Improve the utilization of processing resources.

In another example, please refer to FIG. 5, which is a schematic structural diagram of a fourth data processing device according to an exemplary embodiment of this application. The embodiment shown in FIG. 5 is based on the embodiment shown in FIG. 4 Above, the control module 11 also includes a first instruction slot 114, which is used to cache decoded control instructions; wherein, the decoded control instructions can be represented by corresponding instruction control words.

For the first instruction slot 114, the control module 11 decodes the i-th control instruction; in response to the decoded i-th control instruction in the first instruction slot 114 being invalid, the decoded i+1-th control instruction is invalid The control instruction is cached in the first instruction slot 114; and in response to the i-th control instruction decoded in the second instruction slot 111 being invalid, the i+1-th control instruction decoded in the first instruction slot 114 Cache to the second instruction slot 111.

The first command slot 114 includes a first valid signal and a second command slot completion signal; the first valid signal is used to indicate whether the control command buffered in the first command slot 114 is valid; The second instruction slot completion signal is used to indicate whether the second instruction slot 111 has completed caching the control instruction (that is, whether the control module 11 has cached the cache instruction in the first instruction slot 114 to the The second command slot 111).

When the decoded i-th control command has been sent to the second command slot 111, the status value of the second command slot completion signal is set to a value that characterizes completion, and at the same time, the first valid signal The state value is set to a value characterizing invalidity. The control module 11 can cache the decoded i+1th control instruction to the first instruction slot 114; when the decoded i+1th control instruction is cached to In the first instruction slot 114, the i+1th control instruction has not been cached by the control module 11 to the second instruction slot 111, and the state value of the first valid signal is set to indicate valid And the state value of the completion signal of the second command slot is set to a value that characterizes incompleteness.

Accordingly, the control instructions cached in the second instruction slot 111, the third instruction slot 112, and the fourth instruction slot 113 are decoded control instructions. For the execution process of the decoded control instructions cached in the second instruction slot 111, the third instruction slot 112, and the fourth instruction slot 113, please refer to the description in the embodiment shown in FIG. 4. I won't repeat them here.

In this embodiment, by setting the first instruction slot 114 in the control module 11, the second instruction slot 111 corresponding to the data loading module 12, the third instruction slot 112 corresponding to the processing module 13 and the corresponding In the fourth instruction slot 113 of the data writing back module 14, the control module 11 can load the data into the module 12 and the data through the second instruction slot 111, the third instruction slot 112, and the fourth instruction slot 113, respectively. The processing module 13 and the data write-back module 14 are controlled to ensure that the data loading module 12, the processing module 13 and the data write-back module 14 can execute different control commands at the same time, thereby reducing the corresponding The waiting time is conducive to improving the utilization rate of processing resources.

In an exemplary embodiment, the data processing device provided in this embodiment of the application can be applied to a convolutional neural network to target objects (the target objects include but are not limited to images, audio, video, text, etc.) In the process of processing, the convolution operation of the convolutional layer, the pooling operation of the pooling layer, or the activation operation of the activation layer can be performed to accelerate the calculation process of the deep neural network through hardware, reduce the calculation time of the deep neural network, and improve Operational efficiency.

Hereinafter, the convolutional neural network is applied to the field of image processing, and the data processing device is used to perform the convolution operation of the convolutional layer in the convolutional neural network as an example: the control module 11 receives a convolution operation control instruction And distributed to the data loading module 12, the processing module 13, and the data writing back module 14. Wherein, referring to FIG. 6, the distribution process of the control instruction by the control module 11 is divided into 4 stages, namely, a decoding stage, a loading stage, an execution stage, and a storage stage. This embodiment implements the control of the execution process of different control instructions through the above four stages.

The decoding stage corresponds to the first instruction slot 114, and the control module 11 caches the decoded control instruction into the first instruction slot 114.

The loading stage corresponds to the second instruction slot 111. The control module 11 caches the decoded control instruction in the first instruction slot 114 into the second instruction slot 111, and then transfers the second instruction slot 111 to the second instruction slot 111. The buffered decoded control instruction is sent to the data loading module 12.

The execution stage corresponds to the third instruction slot 112. The control module 11 caches the decoded control instruction in the second instruction slot 111 into the third instruction slot 112, and then caches the third instruction slot 112 The decoded control instruction is sent to the processing module 13.

The storage stage corresponds to the fourth instruction slot 113. The control module 11 caches the decoded control instruction in the third instruction slot 112 into the fourth instruction slot 114, and then caches the fourth instruction slot 114 The decoded control instruction is sent to the data writing back module 14.

Wherein, each instruction slot buffers different control instructions at different times, and the execution of the control instruction is controlled through the control instruction status signal and the control instruction completion signal in the instruction slot. For ease of understanding, please refer to Figure 7 to illustrate the buffering of the control instructions in different time periods in each instruction slot as an example:

In the T0 time period: the control module 11 decodes the control instruction a and caches the decoded control instruction a in the first instruction slot 114 corresponding to the decoding stage; at this time, in the first instruction slot 114, the The first valid signal indicates that the decoded control instruction a is valid, and the second instruction slot completion signal indicates that the second instruction slot 111 has not finished buffering the decoded control instruction a.

In the T1 time period: the control module 11 caches the decoded control instruction a in the first instruction slot 114 into the second instruction slot 111 corresponding to the loading stage, and then stores the decoded control instruction a in the second instruction slot 111 The control instruction a is sent to the data loading module 12; at this time, in the second instruction slot 111, the second valid signal indicates that the decoded control instruction a is valid, and the data loading module completion signal indicates that the data loading module 12 has completed receiving the decoded control instruction a, and the third instruction slot completion signal indicates that the third instruction slot 112 has not completed buffering the decoded control instruction a.

At the same time, since the decoded control instruction a in the first instruction slot 114 has been cached in the second instruction slot 111, the first valid signal in the first instruction slot 114 indicates the decoded control instruction a is invalid, the second instruction slot completion signal instructs the second instruction slot 111 to finish caching the decoded control instruction a, then the control control module 11 can decode the control instruction b and cache the decoded control instruction b to the first In the instruction slot 114, correspondingly, in the first instruction slot 114, the first valid signal indicates that the decoded control instruction b is valid, and the second instruction slot completion signal indicates that the second instruction slot 111 has not completed buffering the decoding After the control instruction b.

In the T2 time period: the control module 11 caches the decoded control instruction a in the second instruction slot 111 into the third instruction slot 112 corresponding to the execution stage, and then stores the decoded control instruction in the third instruction slot 112 a is sent to the processing module 13; at this time, in the third instruction slot, a third valid signal indicates that the decoded control instruction a is valid, and the processing module completion signal indicates that the processing module 13 has completed receiving the decoding The subsequent control instruction a and the fourth instruction slot completion signal indicate that the fourth instruction slot 113 has not completed buffering the decoded control instruction a.

At the same time, since the decoded control instruction a in the second instruction slot 111 has been cached in the third instruction slot 112, the second valid signal in the second instruction slot indicates the decoded control instruction a is invalid, the data loading module completion signal indicates that the data loading module 12 has completed receiving the decoded control instruction a, and the third instruction slot completion signal indicates that the third instruction slot 112 has completed buffering the decoded control instruction a. Control instruction a, the control module 11 can cache the decoded control instruction b in the first instruction slot 114 in the second instruction slot 111. In a possible situation, the load module 12 also When the decoded control instruction a is executed and the decoded control instruction b cannot be issued, then in the second instruction slot, the second valid signal indicates that the decoded control instruction b is valid, and the data loading module The completion signal indicates that the data loading module 12 has not completed receiving the decoded control instruction b, and the third instruction slot completion signal indicates that the third instruction slot 112 has not completed buffering the decoded control instruction b.

At the same time, since the decoded control instruction b in the first instruction slot 114 has been cached in the second instruction slot 111, the first valid signal in the first instruction slot indicates the decoded control instruction b The control instruction b is invalid, and the second instruction slot completion signal instructs the second instruction slot 111 to finish buffering the decoded control instruction b, then the control module 11 can decode the control instruction c and cache the decoded control instruction c to the first In an instruction slot 114, correspondingly, in the first instruction slot, the first valid signal indicates that the decoded control instruction c is valid, and the second instruction slot completion signal indicates that the second instruction slot 111 has not completed buffering the decoding After the control instruction c.

At time T3: the control module 11 caches the decoded control instruction a in the third instruction slot 112 into the fourth instruction slot 113 corresponding to the storage stage, and then stores the decoded control instruction a in the fourth instruction slot 113 Send to the data writing back module 14.

At the same time, because the data loading module 12 is still executing the decoded control instruction a, the decoded control instruction b cannot be issued to the data loading module 12, and it is still cached in the second instruction slot; In the second instruction slot, the second valid signal indicates that the decoded control instruction b is valid, the data loading module completion signal indicates that the data loading module 12 has not completed receiving the decoded control instruction b, and the third The instruction slot completion signal indicates that the third instruction slot 112 has not completed buffering the decoded control instruction b.

At time T4: the data loading module 12 has finished executing the decoded control instruction a, and the control module 11 sends the decoded control instruction b in the second instruction slot to the loading module 12; At this time, in the second instruction slot, the second valid signal indicates that the decoded control instruction b is valid, the data loading module completion signal indicates that the data loading module 12 has completed receiving the decoded control instruction b, and the third The instruction slot completion signal indicates that the third instruction slot 112 has not completed buffering the decoded control instruction b.

At time T5: the control module 11 caches the decoded control instruction b in the second instruction slot 111 into the third instruction slot 112, and then after the processing module 13 finishes executing the decoded control instruction b , Sending the decoded control instruction b buffered in the third instruction slot 112 to the processing module 13; at this time, in the third instruction slot, the third valid signal indicates that the decoded control instruction b is valid, The processing module completion signal indicates that the processing module 13 has completed receiving the decoded control instruction b, and the fourth instruction slot completion signal indicates that the fourth instruction slot 113 has not completed buffering the decoded control instruction b.

At the same time, since the decoded control instruction b in the second instruction slot 111 has been cached in the third instruction slot 112, the second valid signal in the second instruction slot 111 indicates the decoded control Instruction b is invalid, the data loading module completion signal indicates that the data loading module 12 has completed receiving the decoded control instruction b, and the third instruction slot completion signal indicates that the third instruction slot 112 has completed buffering the decoded control instruction b , The control module 11 may cache the decoded control instruction c in the first instruction slot 114 into the second instruction slot 111. In a possible situation, the loading module 12 is still performing the decoded control instruction c. If the decoded control instruction c cannot be issued to the data loading module 12, the second valid signal in the second instruction slot 111 indicates that the decoded control instruction c is valid, and the data The loading module completion signal indicates that the data loading module 12 has not completed receiving the decoded control instruction c, and the third instruction slot completion signal indicates that the third instruction slot 112 has not completed buffering the decoded control instruction c.

At the same time, since the decoded control instruction c in the first instruction slot 114 has been cached in the second instruction slot 111, the first valid signal in the first instruction slot indicates the decoded control The instruction c is invalid, and the second instruction slot completion signal instructs the second instruction slot 111 to finish caching the decoded control instruction c, then the control module 11 can decode the control instruction d and cache the decoded control instruction d to the first In the instruction slot 114, correspondingly, the first valid signal in the first instruction slot 114 indicates that the decoded control instruction d is valid, and the second instruction slot completion signal indicates that the second instruction slot 111 has not completed buffering the decoding After the control instruction d.

In the T6 time period: the control module 11 caches the decoded control instruction b in the third instruction slot 112 into the fourth instruction slot 113 corresponding to the storage stage, and then performs the data write-back module 14 After the decoded control instruction a is executed, the decoded control instruction b buffered in the fourth instruction slot 113 is sent to the data write-back module 14.

At the same time, because the load module 12 is still executing the decoded control instruction b, the decoded control instruction c cannot be issued and is still cached in the second instruction slot, then the second instruction slot 111 The second valid signal indicates that the decoded control instruction c is valid, the data loading module completion signal indicates that the data loading module 12 has not completed receiving the decoded control instruction c, and the third instruction slot completion signal indicates the third instruction slot 112 has not finished caching the decoded control instruction c.

In an exemplary embodiment, the data to be processed includes objects to be processed and operating parameters, where the objects to be processed include but are not limited to images, audio, or text; the operating parameters include but are not limited to convolution kernels, Pooling parameters or activation functions.

Wherein, the processing module 13 includes a systolic array; in response to the i-th control instruction, the processing module 13 writes the object to be processed and operating parameters corresponding to the i-th control instruction into the systolic array, respectively In this step, the object to be processed and the operating parameter are calculated through the systolic array to obtain the processing result.

In an example, the data processing device is used to perform the convolution operation of the image as an example: the object to be processed is the image, the operating parameter is the convolution kernel, and the processing module 13 converts the The image and the convolution kernel are written into the systolic array, and the image and the convolution kernel are multiplied, accumulated and added by the systolic array to obtain a convolved image.

In one embodiment, considering that in a convolutional neural network application scenario, the to-be-processed data that needs to be loaded includes at least two parts: the to-be-processed object and the operating parameters. Therefore, in order to further improve the efficiency of data loading, please refer to FIG. 8, which is The present application shows a schematic diagram of a fifth data processing device according to an exemplary embodiment. The data loading module 12 includes an object loading unit 121 and a parameter loading unit 122. In response to the completion of the execution of the i-th control instruction by the object loading unit 121, the control module 11 sends the (i+1)th control instruction to the object loading unit 121; After the i control instruction is executed, the i+1th control instruction is sent to the parameter loading unit 122. In this embodiment, the object loading unit 121 and the parameter loading unit 122 respectively load the object to be processed and the operating parameter, and perform the loading at the same time, which is beneficial to improving the loading efficiency.

It should be noted that the completion of the execution of the i-th control instruction by the object loading unit 121 means that the object loading unit 121 has finished loading the object to be processed corresponding to the i-th control instruction; The completion of the execution of the control instruction means that the parameter loading unit 122 has finished loading the operating parameter corresponding to the i-th control instruction.

In response to the i-th control instruction, the object loading unit 121 loads the object to be processed corresponding to the i-th control instruction; the parameter loading unit 122 loads the i-th control instruction in response to the i-th control instruction. The operating parameters corresponding to i control instructions.

Wherein, when the processing module 13 processes the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction, if the object loading unit 121 has finished loading the i-th control instruction The object to be processed corresponding to the instruction does not need to wait for the processing module 13 to complete the execution of the i-th control instruction, and the object loading unit 121 can directly receive the i+1-th control instruction sent by the control module 11. And load the object to be processed corresponding to the i+1th control instruction; similarly, if the parameter loading unit 122 has loaded the operating parameters corresponding to the i-th control instruction, there is no need to wait for the processing module 13 to respond to the After the i-th control instruction is executed, the parameter loading unit 122 can directly receive the i+1-th control instruction sent by the control module 11, and load the operating parameter corresponding to the i+1-th control instruction; this embodiment The waiting time of the object loading unit 121 and the parameter loading unit 122 for the i+1th control instruction is further reduced, and the waste of processing resources caused by the excessive waiting time is avoided.

In an embodiment, the control module 11 includes a second instruction slot 111 corresponding to the data loading module 12, a third instruction slot 112 corresponding to the data processing module 13, and a third instruction slot 112 corresponding to the data write-back module. 14 of the fourth instruction slot 113; when the data loading module 12 includes an object loading unit 121 and a parameter loading unit 122, correspondingly, for the second instruction slot 111, the control module 11 responds to the second The i-th control instruction in the instruction slot 111 is invalid, and the i+1-th control instruction is cached in the second instruction slot 111; and in response to the object loading unit 121 and the parameter loading unit 122, the i-th control instruction is After the control instruction is executed, the i+1th control instruction in the second instruction slot 111 is sent to the data loading module 12.

The second command slot 111 includes a second valid signal, an object loading unit completion signal, a parameter loading unit completion signal, and a third command slot completion signal; the second valid signal is used to instruct the second command slot 111 Whether the control instruction buffered in the central storage is valid; the object loading unit completion signal is used to indicate whether the object loading unit 121 has completed receiving the control instruction (that is, whether the control module 11 has sent the control instruction to the The object loading unit 121); the parameter loading unit completion signal is used to indicate whether the parameter loading unit 122 has completed receiving the control instruction (that is, whether the control module 11 has sent the control instruction to the parameter loading Unit 122); the third instruction slot completion signal is used to indicate whether the third instruction slot 112 has completed caching the control instruction (that is, whether the control module 11 has already stored the second instruction slot 111 The cache instruction is cached to the third instruction slot 112).

When the i-th control instruction has been sent to the object loading unit 121, the parameter loading unit 122, and has been cached in the third instruction slot 112, the object loading unit completion signal and the parameter loading unit completion signal And the state value of the completion signal of the third command slot is set to a value that characterizes completion. At the same time, the state value of the second valid signal is set to a value that characterizes invalidity. At this time, the control module 11 may The i+1th control instruction is cached in the second instruction slot 111; when the i+1th control instruction is cached in the second instruction slot 111, the i+1th control instruction It has not been sent to the object loading unit 121, the parameter loading unit 122, and cached in the third instruction slot 112. Therefore, the state value of the second valid signal is set to a value representing valid, and the The status values of the data loading module completion signal and the third instruction slot completion signal are set to values that represent incompleteness.

For the execution process of the third instruction slot 112 and the fourth instruction slot 113, please refer to the embodiment shown in FIG. 3, which will not be repeated in the embodiment of the present application.

In an embodiment, in order to further improve the efficiency of data processing, the target data to be processed may be divided into at least two parts. The data to be processed is a part of the target data to be processed. The control module 11 passes at least two parts. The control instruction performs control, and a control instruction instructs a part of the target data to be processed to realize the processing of the target data to be processed. Since the target data to be processed is divided into at least two parts, the data loading module 12 When loading the data to be processed based on one of the control instructions, only a part of the target data to be processed is loaded, which is beneficial to improve the loading efficiency, so that the processing module 13 does not need to wait for the data loading module 12 to load the complete data to be processed. By processing the target data, the loaded data to be processed can be processed faster, and the processing efficiency can be further improved.

Since the target data to be processed is divided into at least two parts, the control is controlled by at least two control instructions, and one control instruction instructs a part of the target data to be processed to realize the processing of the target data to be processed, Based on this, the control instruction includes a result write-back control instruction and a result non-write-back control instruction.

The result non-write-back control instruction is used to instruct the processing module 13 to not send the processing result to the data write-back module 14 after obtaining the processing result, but to cache the processing result. The result is not written back to the processing result corresponding to the control instruction is not the final processing result that is finally written into the external storage module by the data writing back module 14, but a part of the final processing result; the result is not written back to the control after receiving the result. After the instruction, the processing module 13 processes the corresponding data to be processed according to the result without writing back the control instruction, obtains the processing result and caches it, and then generates an end signal sent to the control signal after the cache is completed, so The control module 11 receives an end signal sent by the processing module 13 after buffering the processing result, and the end signal indicates that the result is not written back control instruction that has been executed in the data processing device.

The result write-back control instruction is used to instruct the processing module 13 to send all processing results related to the target data to be processed to the data write-back module 14. After receiving the result write-back control instruction, The processing module 13 processes the corresponding data to be processed according to the result write-back control instruction to obtain the processing result, and integrates all processing results related to the target data to be processed and sends it to the data write-back Module 14, the data write-back module 14 writes to the external storage module. After the write operation is completed, the data write-back module 14 generates an end signal and sends it to the control module 11, and the control module 11 receives The data write-back module 14 sends an end signal after writing the processing result, and the end signal indicates that the result write-back control instruction has been executed in the data processing device.

In this embodiment, since the target data to be processed is divided into at least two parts, each part is instructed by a control instruction, so that the data loading module 12 only needs to load all the data to be processed when loading the data to be processed corresponding to the control instruction. The part of the target data to be processed is beneficial to improve the loading efficiency, and reduces the time that the processing module 13 waits for the data loading module 12 to load the target data to be processed, so that the processing can more quickly perform the loading process. The processing of the data to be processed is conducive to improving processing efficiency.

Further, the control module 11 may return the end signal corresponding to the control instruction to the external control module to notify the external control module that the control instruction has been executed, so that the external control module can write to the external The final processing result of the storage module goes to the next processing step.

Wherein, since the control instruction includes a result write-back control instruction and a result-not-write-back control instruction, the two control instructions have different processing methods for the acquired processing results, and their ways of sending the end signal are also different. The results are written back. The control instruction is sent by the data write-back module 14 after the completion of the instruction, and the result is not written back control instruction is sent by the processing module 13 after the completion of the instruction; in an exemplary scenario, The data write-back module 14 is writing the final processing result A corresponding to the target data A to be processed into the external storage module. At this time, the data write-back module 14 has not been executed yet, so it is not generated and sent to the control module 11 At this time, the control module 11 may have processed a part of the to-be-processed data B _{1 of} the to-be-processed target data B and generated the end signal B ₁ to be sent to the control module 11. said control signal module 11 the end B is returned to the external control module _1, the external control module ₁ directly to the next step based on the end of the signal B, the signal a while ignoring the end has not been received, the external control module The processing steps based on the final processing result A may be skipped directly, which may lead to errors in the processing flow.

Therefore, in order to ensure the accuracy of the processing flow, the control module 11 returns the end signal corresponding to the control instruction to the external control module according to the order in which the control instruction is received and the first-in-first-out principle. The receiving order and the first-in-first-out principle, it is determined that the currently received end signal is not currently to be sent, and the currently received end signal is buffered first, and the end signal is not returned to the external control until it is the turn of the end signal to be sent Module. In this embodiment, the end signal is processed in order to ensure that the control signal received from the external control module first, and the corresponding end signal is first returned to the external control module, thereby ensuring the accuracy and orderly progress of the data processing process. .

In an example, please refer to FIG. 9. For example, the target data C to be processed is divided into two parts, including the data to be processed c1 and the data to be processed c2, and the control instructions include a result write-back control instruction and a result non-write-back control Instruction, the result not written back control instruction corresponds to the to-be-processed data c1, and the result write-back control instruction corresponds to the to-be-processed c2.

In the embodiment shown in FIG. 9, the data loading module 12 loads the data to be processed c1 based on the result not writing back control instruction, and the processing module 13 performs the processing on the data to be processed based on the result not writing back control instruction. c1 performs processing to obtain the processing result c1, and caches the processing result c1; the data loading module 12 loads the data to be processed c2 based on the data write-back control instruction, and the processing module 13 writes back the control instruction based on the result Process the to-be-processed data c2 to obtain the processing result c2, and then integrate the processing result c1 and the processing result c2 to obtain the final processing result (c1, c2) corresponding to the to-be-processed target data C, and then use the The data write-back module 14 writes the final processing result (c1, c2) into the external storage module.

In an exemplary embodiment, the data to be processed includes objects to be processed and operating parameters, where the objects to be processed include but are not limited to images, audio, or text; the operating parameters include but are not limited to convolution kernels, Pooling parameters or activation functions. In order to further improve the efficiency of data processing, the target object to be processed may be divided into at least two parts and the target operating parameter may be divided into at least two parts. The target object to be processed is a part of the target object to be processed, and the operating parameter Is part of the target operating parameters.

Correspondingly, the control instruction includes a result write-back control instruction and a result-not-write-back control instruction, and the result-not-write-back control instruction is used to instruct the processing module 13 not to perform the processing after obtaining the processing result. The result is sent to the data write-back module 14, but the processing result is cached. The processing result corresponding to the control instruction is not written back to the final processing of the data write-back module 14 to the external storage module. The result is part of the final processing result; the result write-back control instruction is used to instruct the processing module 13 to send all processing results related to the target object to be processed and the target operating parameter to the data The write-back module 14, the processing module 13 integrates all the processing results related to the target object to be processed and the target operating parameters according to the result write-back control instruction, and sends the result to the data write-back module 14. The data write-back module 14 writes to the external storage module.

In this embodiment, since the target object to be processed and the target operating parameter are divided into at least two parts, each part is instructed by a control instruction, so that the data loading module 12 loads all the corresponding control instructions. When the object to be processed and the operating parameters are described, only a part of the object to be processed and the target operating parameters need to be loaded, which is beneficial to improve the loading efficiency and reduces the processing module 13 waiting for the data loading module 12 to load. The time of the target data to be processed enables the processing to process the loaded data to be processed more quickly, which is beneficial to improve processing efficiency.

It should be noted that, in the embodiments shown in FIG. 4 and FIG. 5, the control module 11 includes a second instruction slot 111 corresponding to the data loading module 12, which corresponds to the data processing module 13 The third instruction slot 112 and the fourth instruction slot 113 corresponding to the data write-back module 14; the second instruction slot 111, the third instruction slot 112, and the fourth instruction slot 113 are used to cache control instructions. When the control instruction is a result non-write-back control instruction, there is no need to cache the data non-write-back control instruction to the fourth instruction slot 113, and there is no need to send the data non-write-back instruction to the data write-back. The module 14, that is, the data write-back module 14 does not need to execute the result non-write-back control instruction.

Hereinafter, the convolutional neural network is applied to the field of image processing, and the data processing device is used to perform the convolution operation of the convolutional layer in the convolutional neural network as an example: the control module 11 receives a convolution operation control instruction And distributed to the data loading module 12, the processing module 13 and the data write-back module 14; among them, referring to FIG. 10, the control module 11 is divided into 4 stages for the distribution process of the control instructions, namely Decoding phase, loading phase, execution phase and storage phase. The decoding stage corresponds to the first instruction slot 114, the loading stage corresponds to the second instruction slot 111, the execution stage corresponds to the third instruction slot 112, and the storage stage corresponds to the fourth instruction slot. 113; Wherein, when the control instruction is the result of non-write-back control instruction, there is no need to cache the data-non-write-back control instruction to the fourth instruction slot 113, and there is no need to send the data non-write-back instruction to The data write-back module 14, that is, the data write-back module 14 does not need to execute the result-not-write-back control instruction; when the control instruction is a result-write-back control instruction, it needs to be cached to the fourth The instruction slot 113 and sent to the data write-back module 14 are executed by the data write-back module 14.

The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Ordinary technicians in this neighborhood can understand and implement without creative work.

Correspondingly, referring to FIG. 11, an embodiment of the present application also provides a data processing method, which is applied to a data processing device, and the data processing device includes a data loading module and a processing module; the method includes:

In step S101, in response to a control instruction, data is loaded by the data loading module for the processing module to perform data processing.

In step S102, in response to the control instruction, data processing is performed by the processing module; wherein the data loading module and the processing module execute different control instructions at the same time.

In an embodiment, the method further includes:

In response to the control instruction, the processing result of the to-be-processed data is written into an external storage module; wherein the data loading module, the processing module, and the data write-back module execute different control instructions at the same time.

In an embodiment, the method further includes:

In response to the completion of the execution of the i-th control instruction by the data loading module, the i+1th control instruction is sent to the data loading module; and in response to the completion of the execution of the i-th control instruction by the processing module, the i+1 control instructions are sent to the processing module; where i is an integer.

The step S101 includes: in response to the i-th control instruction, loading the to-be-processed data corresponding to the i-th control instruction.

The step S102 includes: in response to the i-th control instruction, processing the to-be-processed data corresponding to the i-th control instruction to obtain a processing result.

In an embodiment, the device further includes a data write-back module.

The method also includes:

In response to the completion of the execution of the i-th control instruction by the data write-back module, the i+1-th control instruction is sent to the data write-back module.

In response to the i-th control instruction, the processing result corresponding to the i-th control instruction is written into the external storage module through the data write-back module.

In an embodiment, it further includes:

When the processing module processes the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction, there is no need to wait for the processing module to finish executing the i-th control instruction. The data loading module receives the i+1th control instruction.

In an embodiment, it further includes:

When the data write-back module writes the processing result corresponding to the i-th control instruction to the external storage module in response to the i-th control instruction, there is no need to wait for the data write-back module to control the i-th control instruction After the instruction is executed, the processing module receives the i+1th control instruction.

In an embodiment, it further includes:

The control instruction is cached through at least one instruction slot, and the execution process of the control instruction is controlled according to the state of the control instruction recorded in the at least one instruction slot.

In an embodiment, it further includes:

The control instruction is cached by at least one instruction slot; the at least one instruction slot includes at least one control instruction status signal, wherein the control instruction status signal is used to indicate whether the control instruction cached in the corresponding instruction slot is valid.

In an embodiment, it further includes:

The control instruction is cached by at least one instruction slot; the at least one instruction slot includes at least one control instruction completion signal; wherein, the at least one control instruction completion signal is used to indicate whether the module corresponding to the at least one instruction slot has completed the The operation of the control instruction corresponding to the at least one instruction slot, or the at least one control instruction completion signal is used to indicate whether the at least one instruction slot completes the operation of the corresponding control instruction.

In an embodiment, it further includes:

The control instruction is cached through at least one instruction slot; the instruction slot includes a control instruction status signal and a control instruction completion signal; wherein, when the control instruction completion signal instructs the corresponding module to complete the operation of the corresponding control instruction, the The control instruction status signal indicates that the control instruction buffered in the corresponding instruction slot is invalid.

In an embodiment, it further includes:

The control instruction is cached by at least one instruction slot; the instruction slot includes a control instruction status signal and a control instruction completion signal; wherein, when the control instruction completion signal instructs the at least one instruction slot to complete the operation of the corresponding control instruction , The control command status signal indicates that the control command buffered in the corresponding command slot is invalid.

In an embodiment, it further includes:

The control instruction is cached by at least one instruction slot; wherein the i+1th control instruction sent to the data loading module, the processing module, and the data write-back module is obtained from the instruction slot.

In an embodiment, it further includes:

Cache the control instructions corresponding to the data loading module through the second instruction slot, cache the control instructions corresponding to the processing module through the third instruction slot, and cache the control instructions corresponding to the data write-back module through the third instruction slot. Control instruction.

In an embodiment, it further includes:

In response to the i-th control instruction in the second instruction slot being invalid, the i+1-th control instruction is cached in the second instruction slot.

The sending the i+1th control instruction to the data loading module in response to the completion of the execution of the i-th control instruction by the data loading module includes:

In response to the completion of the execution of the i-th control instruction by the data loading module, the (i+1)th control instruction in the second instruction slot is sent to the data loading module.

In an embodiment, it further includes:

In response to the i-th control instruction in the third instruction slot being invalid, the i+1-th control instruction in the second instruction slot is cached to the third instruction slot.

The sending the i+1th control instruction to the processing module in response to the completion of the execution of the i-th control instruction by the processing module includes:

In response to the completion of the execution of the i-th control instruction by the processing module, the i+1-th control instruction in the third instruction slot is sent to the processing module.

In an embodiment, it further includes:

In response to the i-th control instruction in the fourth instruction slot being invalid, the i+1-th control instruction in the third instruction slot is cached to the fourth instruction slot.

The sending the i+1th control instruction to the data write-back module in response to the completion of the execution of the i-th control instruction by the data write-back module includes:

In response to the completion of the execution of the i-th control instruction by the data write-back module, the i+1-th control instruction in the fourth instruction slot is sent to the processing module.

In an embodiment, it further includes:

Decode the i-th control instruction; in response to the decoded i-th control instruction in the first instruction slot being invalid, buffer the decoded i+1-th control instruction to the first instruction slot; and,

In response to the invalidation of the i-th control instruction decoded in the second instruction slot, the i+1-th control instruction decoded in the first instruction slot is cached in the second instruction slot.

In an embodiment, the control instruction includes a result write-back control instruction and a result non-write-back control instruction.

The to-be-processed data is a part of the to-be-processed target data; the to-be-processed target data is divided into at least two parts.

The result not writing back control instruction is used to instruct the processing module to cache the processing result;

The result write-back control instruction is used to instruct the processing module to send all processing results related to the target data to be processed to the data write-back module.

In an embodiment, the step S102 includes:

According to the result, the control instruction is not written back, and the corresponding data to be processed is processed, and the processing result is obtained and cached; and,

According to the result write-back control instruction, the corresponding to-be-processed data is processed to obtain the processing result, and all the processing results related to the to-be-processed target data are integrated and sent to the data write-back module.

In an embodiment, it further includes:

If the control instruction is a control instruction not to write back the result, receiving an end signal sent by the processing module after buffering the processing result; and,

If the control instruction is the result write-back control instruction, receive the end signal sent by the data write-back module after the processing result is written; wherein, the end signal indicates that the result is not written back control The instruction or the result write-back control instruction is executed in the data processing device.

In an embodiment, it further includes:

According to the receiving order of the control instructions and the first-in-first-out principle, the end signal corresponding to the control instruction is returned to the external control module.

In an embodiment, it further includes:

If it is determined that the currently received end signal is not currently to be sent according to the receiving order of the control instructions and the first-in-first-out principle, the currently received end signal is buffered.

In an embodiment, the data to be processed includes the object to be processed and operating parameters.

In an embodiment, the object to be processed includes any one of the following: image, audio, or text; the operating parameter includes any one of the following: a convolution kernel, a pooling parameter, or an activation function.

In an embodiment, the data loading module includes an object loading unit and a parameter loading unit.

The step S101 includes:

In response to the i-th control instruction, load the object to be processed corresponding to the i-th control instruction through the object loading unit; and,

In response to the i-th control instruction, the operating parameter corresponding to the i-th control instruction is loaded by the parameter loading unit.

In an embodiment, the step S102 includes:

In response to the i-th control instruction, write the to-be-processed data corresponding to the i-th control instruction into a systolic array, and perform operations on the to-be-processed data through the systolic array to obtain a processing result.

In an embodiment, the step S102 includes:

In response to the i-th control instruction, the object to be processed and the operating parameters corresponding to the i-th control instruction are respectively written into the systolic array, and the object to be processed and the operating parameter are combined through the systolic array. The parameters are calculated and the processing result is obtained.

For the specific implementation of the method embodiment, please refer to the description of the device embodiment, which will not be repeated here.

Correspondingly, an embodiment of the present application also provides an accelerator, which includes any of the above-mentioned devices.

The accelerator can be applied to various neural networks, such as convolutional neural networks.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. The terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements that are not explicitly listed. Elements, or also include elements inherent to such processes, methods, articles, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

The methods and devices provided by the embodiments of the present invention are described in detail above. Specific examples are used in this article to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and methods of the present invention. The core idea; at the same time, for ordinary technicians in this neighborhood, according to the idea of the present invention, there will be changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as a reference to the present invention. limit.

Claims

A data processing device, characterized by comprising a control module, a data loading module, and a processing module;

The data loading module, in response to the control instruction of the control module, loads the data to be processed for processing by the processing module;

The processing module responds to the control instruction of the control module to process the data to be processed;

The control module controls the data loading module and the processing module to execute different control instructions at the same time.
The device according to claim 1, further comprising a data write-back module,

The data write-back module, in response to the control instruction of the control module, writes the processing result of the data to be processed into the external storage module;

The control module controls the data loading module, the processing module and the data writing back module to execute different control instructions at the same time.
The device of claim 1, wherein:

The control module is specifically configured to: in response to the completion of the execution of the i-th control instruction by the data loading module, send the i+1th control instruction to the data loading module; After executing the control instructions, send the i+1th control instruction to the processing module; where i is an integer;

The data loading module is specifically configured to: in response to the i-th control instruction, load the to-be-processed data corresponding to the i-th control instruction;

The processing module is specifically configured to: in response to the i-th control instruction, process the to-be-processed data corresponding to the i-th control instruction to obtain a processing result.
The device according to claim 3, wherein the device further comprises a data write-back module;

The control module is further configured to: in response to the completion of the execution of the i-th control instruction by the data write-back module, send the i+1-th control instruction to the data write-back module;

The data write-back module is configured to: in response to the i-th control instruction, write the processing result corresponding to the i-th control instruction into the external storage module.
The device according to claim 3, wherein:

When the processing module processes the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction, there is no need to wait for the processing module to finish executing the i-th control instruction. The data loading module receives the i+1th control instruction.
The device of claim 4, wherein:

When the data write-back module writes the processing result corresponding to the i-th control instruction to the external storage module in response to the i-th control instruction, there is no need to wait for the data write-back module to control the i-th control instruction After the instruction is executed, the processing module receives the i+1th control instruction.
The device according to claim 1, wherein the control module comprises at least one command slot for caching the control command; the control module controls the control command according to the state of the control command recorded in the at least one command slot The execution process of the control instruction.
The device according to claim 1, wherein the control module includes at least one command slot for caching control commands; the at least one command slot includes at least one control command status signal, wherein the control command status The signal is used to indicate whether the control command cached in the corresponding command slot is valid.
The device according to claim 1, wherein the control module comprises at least one command slot for caching control commands; the at least one command slot comprises at least one control command completion signal; wherein, the at least one control command The instruction completion signal is used to indicate whether the module corresponding to the at least one instruction slot has completed the operation of the control instruction corresponding to the at least one instruction slot, or the at least one control instruction completion signal is used to indicate whether the at least one instruction slot is completed The operation of the corresponding control instruction.
The device according to claim 1, wherein the control module includes an instruction slot for buffering the control instruction; the instruction slot includes a control instruction status signal and a control instruction completion signal; wherein, when the control When the instruction completion signal indicates that the corresponding module completes the operation of the corresponding control instruction, the control instruction status signal indicates that the control instruction buffered in the corresponding instruction slot is invalid.
The device according to claim 1, wherein the control module includes an instruction slot for buffering the control instruction; the instruction slot includes a control instruction status signal and a control instruction completion signal; wherein, when the control When the instruction completion signal indicates that the at least one instruction slot completes the operation of the corresponding control instruction, the control instruction status signal indicates that the control instruction buffered in the corresponding instruction slot is invalid.
The device according to claim 4, wherein the control module includes an instruction slot for caching the control instruction;

The control module is specifically configured to: in response to the completion of the execution of the i-th control instruction by the data loading module, send the i+1th control instruction buffered in the instruction slot to the data loading module; and in response to The processing module finishes executing the i-th control instruction, and sends the i+1-th control instruction cached in the instruction slot to the processing module; and responds to the data write-back module for the i-th control instruction After the execution is completed, the i+1th control instruction cached in the instruction slot is sent to the data write-back module.
The device according to claim 4, wherein the control module includes a second instruction slot corresponding to the data loading module, a third instruction slot corresponding to the processing module, and a third instruction slot corresponding to the data write-back The fourth command slot of the module;

The second instruction slot, the third instruction slot, and the fourth instruction slot are all used to: after the control module sends the i-th control instruction at different times, respectively, cache all corresponding commands. The control instruction of Article i+1;

The control module is specifically configured to: in response to the completion of the execution of the i-th control instruction by the data loading module, send the i+1th control instruction buffered in the second instruction slot to the data loading module; and In response to the completion of the execution of the i-th control instruction by the processing module, the i+1th control instruction buffered in the third instruction slot is sent to the processing module; and in response to the data write-back module, the first control instruction is sent to the processing module; After the i control instruction is executed, the i+1th control instruction buffered in the fourth instruction slot is sent to the data write-back module.
The device according to claim 4, wherein the control module comprises a second instruction slot corresponding to the data loading module;

The second instruction slot is used for buffering the control instruction sent to the data loading module;

The control module is further configured to: in response to the i-th control instruction in the second instruction slot being invalid, buffer the i+1-th control instruction to the second instruction slot; in response to the data loading After the module finishes executing the i-th control instruction, it sends the i+1-th control instruction in the second instruction slot to the data loading module.
The device according to claim 14, wherein the control module further comprises a third instruction slot corresponding to the processing module;

The third instruction slot is used for buffering the control instruction sent to the processing module;

The control module is further configured to: in response to the i-th control instruction in the third instruction slot being invalid, cache the i+1-th control instruction in the second instruction slot to the third instruction slot; And in response to the completion of the execution of the i-th control instruction by the processing module, the i+1-th control instruction in the third instruction slot is sent to the processing module.
The device according to claim 15, wherein the control module further comprises a fourth instruction slot corresponding to the data write-back module;

The fourth instruction slot is used for buffering the control instruction sent to the data write-back module;

The control module is further configured to: in response to the i-th control instruction in the fourth instruction slot being invalid, cache the i+1-th control instruction in the third instruction slot to the fourth instruction slot; And in response to the completion of the execution of the i-th control instruction by the data write-back module, the i+1-th control instruction in the fourth instruction slot is sent to the processing module.
The device according to claim 14, wherein the control module further comprises a first instruction slot;

The first instruction slot is used to cache decoded control instructions;

The control module is further configured to: decode the i-th control instruction; in response to the decoded i-th control instruction in the first instruction slot being invalid, buffer the decoded i+1-th control instruction to the first instruction slot. An instruction slot; and in response to the i-th control instruction decoded in the second instruction slot being invalid, buffering the i+1-th control instruction decoded in the first instruction slot to the second instruction slot .
The device according to claim 2, wherein the control instruction includes a result write-back control instruction and a result non-write-back control instruction;

The to-be-processed data is a part of the to-be-processed target data; the to-be-processed target data is divided into at least two parts;

The result not writing back control instruction is used to instruct the processing module to cache the processing result;

The result write-back control instruction is used to instruct the processing module to send all processing results related to the target data to be processed to the data write-back module.
The device of claim 18, wherein:

The processing module is specifically configured to: process the corresponding data to be processed according to the result not writing back control instruction to obtain the processing result and cache it; and write back the control instruction according to the result to perform processing on the corresponding data to be processed After processing, the processing result is obtained, and all processing results related to the target data to be processed are integrated and sent to the data write-back module.
The device of claim 18, wherein:

If the control instruction is a control instruction not to write back the result, the control module is further configured to: receive an end signal sent by the processing module after buffering the processing result; and

If the control instruction is the result write-back control instruction, the control module is further configured to: receive an end signal sent by the data write-back module after the processing result is written;

Wherein, the end signal indicates that the result is not written back control instruction or that the result write back control instruction has been executed in the data processing device.
The device of claim 20, wherein:

The control module is further configured to return the end signal corresponding to the control instruction to the external control module according to the receiving order of the control instruction and the first-in-first-out principle.
The device of claim 21, wherein:

The control module is further configured to: if it is determined that the currently received end signal is not currently to be sent according to the receiving order of the control instructions and the first-in-first-out principle, buffer the currently received end signal.
The device according to claim 1, wherein the data to be processed includes an object to be processed and operating parameters.
The device according to claim 23, wherein the object to be processed comprises any one of the following: image, audio or text;

The operating parameters include any one of the following: a convolution kernel, a pooling parameter, or an activation function.
The device according to claim 23, wherein the data loading module comprises an object loading unit and a parameter loading unit;

The control module is specifically configured to: in response to the completion of the execution of the i-th control instruction by the object loading unit, send the i+1-th control instruction to the object loading unit; After the execution of the i control instruction is completed, the i+1th control instruction is sent to the parameter loading unit;

The object loading unit is configured to: in response to the i-th control instruction, load the object to be processed corresponding to the i-th control instruction;

The parameter loading unit is configured to load the operating parameter corresponding to the i-th control instruction in response to the i-th control instruction.
The device according to claim 1, wherein the processing module comprises a systolic array;

The processing module is specifically configured to: in response to the i-th control instruction, write the data to be processed corresponding to the i-th control instruction into a systolic array, and perform operations on the data to be processed through the systolic array , To obtain the processing result.
The device according to claim 23, wherein the processing module comprises a systolic array;

The processing module is specifically configured to: in response to the i-th control instruction, write the to-be-processed objects and operating parameters corresponding to the i-th control instruction into the systolic array, respectively, and use the systolic array to write all The object to be processed is calculated with the operating parameter to obtain the processing result.
The device of claim 1, wherein:

The control module includes an instruction slot for caching the control instruction; the instruction slot includes a set of instruction cache flags and control state signals;

Wherein, the instruction cache flag is used to indicate whether the instruction cached in the instruction slot is valid; the set of control status signals is used to indicate the working status of the corresponding module, or indicate other instruction slots related to the instruction slot Working status.
A data processing method, characterized in that it is applied to a data processing device, the data processing device includes a data loading module and a processing module; the method includes:

In response to the control instruction, load data through the data loading module for the processing module to perform data processing; and,

In response to the control instruction, data processing is performed by the processing module; wherein the data loading module and the processing module execute different control instructions at the same time.
The method according to claim 29, wherein the method further comprises:

In response to the control instruction, the processing result of the to-be-processed data is written into an external storage module; wherein the data loading module, the processing module, and the data write-back module execute different control instructions at the same time.
The method according to claim 29, wherein the method further comprises:

In response to the completion of the execution of the i-th control instruction by the data loading module, the i+1th control instruction is sent to the data loading module; and in response to the completion of the execution of the i-th control instruction by the processing module, the i+1 control instructions are sent to the processing module; where i is an integer;

The loading data through the data loading module in response to the control instruction for the processing module to perform data processing includes:

In response to the i-th control instruction, load the to-be-processed data corresponding to the i-th control instruction;

The performing data processing by the processing module in response to the control instruction includes:

In response to the i-th control instruction, the to-be-processed data corresponding to the i-th control instruction is processed to obtain a processing result.
The method according to claim 31, wherein the device further comprises a data write-back module;

The method also includes:

In response to the completion of the execution of the i-th control instruction by the data write-back module, sending the i+1-th control instruction to the data write-back module;

In response to the i-th control instruction, the processing result corresponding to the i-th control instruction is written into the external storage module through the data write-back module.
The method according to claim 31, further comprising:

When the processing module processes the data to be processed corresponding to the i-th control instruction in response to the i-th control instruction, there is no need to wait for the processing module to finish executing the i-th control instruction. The data loading module receives the i+1th control instruction.
The method according to claim 32, further comprising:

When the data write-back module writes the processing result corresponding to the i-th control instruction to the external storage module in response to the i-th control instruction, there is no need to wait for the data write-back module to control the i-th control instruction After the instruction is executed, the processing module receives the i+1th control instruction.
The method according to claim 29, further comprising:

The control instruction is cached by at least one instruction slot, and the execution process of the control instruction is controlled according to the state of the control instruction recorded in the at least one instruction slot.
The method according to claim 29, further comprising:

The control instruction is cached by at least one instruction slot; the at least one instruction slot includes at least one control instruction status signal, wherein the control instruction status signal is used to indicate whether the control instruction cached in the corresponding instruction slot is valid.
The method according to claim 29, further comprising:

The control instruction is cached by at least one instruction slot; the at least one instruction slot includes at least one control instruction completion signal; wherein, the at least one control instruction completion signal is used to indicate whether the module corresponding to the at least one instruction slot has completed the The operation of the control instruction corresponding to the at least one instruction slot, or the at least one control instruction completion signal is used to indicate whether the at least one instruction slot completes the operation of the corresponding control instruction.
The method according to claim 29, further comprising:

The control instruction is cached by at least one instruction slot; the instruction slot includes a control instruction status signal and a control instruction completion signal; wherein, when the control instruction completion signal instructs the corresponding module to complete the operation of the corresponding control instruction, the The control instruction status signal indicates that the control instruction buffered in the corresponding instruction slot is invalid.
The method according to claim 29, further comprising:

The control instruction is cached by at least one instruction slot; the instruction slot includes a control instruction status signal and a control instruction completion signal; wherein, when the control instruction completion signal instructs the at least one instruction slot to complete the operation of the corresponding control instruction , The control command status signal indicates that the control command buffered in the corresponding command slot is invalid.
The method according to claim 31, further comprising:

The control instruction is cached by at least one instruction slot; wherein the i+1th control instruction sent to the data loading module, the processing module, and the data write-back module is obtained from the instruction slot.
The method according to claim 31, further comprising:

Cache the control instructions corresponding to the data loading module through the second instruction slot, cache the control instructions corresponding to the processing module through the third instruction slot, and cache the control instructions corresponding to the data write-back module through the third instruction slot. Control instruction.
The method according to claim 32, further comprising:

In response to the i-th control instruction in the second instruction slot being invalid, buffering the i+1-th control instruction to the second instruction slot;

The sending the i+1th control instruction to the data loading module in response to the completion of the execution of the i-th control instruction by the data loading module includes:

In response to the completion of the execution of the i-th control instruction by the data loading module, the (i+1)th control instruction in the second instruction slot is sent to the data loading module.
The method according to claim 42, further comprising:

In response to the i-th control instruction in the third instruction slot being invalid, buffering the i+1-th control instruction in the second instruction slot to the third instruction slot;

The sending the i+1th control instruction to the processing module in response to the completion of the execution of the i-th control instruction by the processing module includes:

In response to the completion of the execution of the i-th control instruction by the processing module, the i+1-th control instruction in the third instruction slot is sent to the processing module.
The method according to claim 43, further comprising:

In response to the i-th control instruction in the fourth instruction slot being invalid, buffering the i+1-th control instruction in the third instruction slot to the fourth instruction slot;

The sending the i+1th control instruction to the data write-back module in response to the completion of the execution of the i-th control instruction by the data write-back module includes:

In response to the completion of the execution of the i-th control instruction by the data write-back module, the i+1-th control instruction in the fourth instruction slot is sent to the processing module.
The method according to claim 42, further comprising:

Decode the i-th control instruction; in response to the decoded i-th control instruction in the first instruction slot being invalid, buffer the decoded i+1-th control instruction to the first instruction slot; and,

In response to the invalidation of the i-th control instruction decoded in the second instruction slot, the i+1-th control instruction decoded in the first instruction slot is cached in the second instruction slot.
The method according to claim 30, wherein the control instruction includes a result write-back control instruction and a result non-write-back control instruction;

The to-be-processed data is a part of the to-be-processed target data; the to-be-processed target data is divided into at least two parts;

The result not writing back control instruction is used to instruct the processing module to cache the processing result;

The result write-back control instruction is used to instruct the processing module to send all processing results related to the target data to be processed to the data write-back module.
The method of claim 46, wherein the processing of data by the processing module in response to the control instruction comprises:

According to the result, the control instruction is not written back, and the corresponding data to be processed is processed, and the processing result is obtained and cached; and,

According to the result write-back control instruction, the corresponding to-be-processed data is processed to obtain the processing result, and all the processing results related to the to-be-processed target data are integrated and sent to the data write-back module.
The method according to claim 47, further comprising:

If the control instruction is a control instruction not to write back the result, receiving an end signal sent by the processing module after buffering the processing result; and,

If the control instruction is the result write-back control instruction, receive the end signal sent by the data write-back module after the processing result is written; wherein, the end signal indicates that the result is not written back control The instruction or the result write-back control instruction is executed in the data processing device.
The method according to claim 48, further comprising:

According to the receiving order of the control instructions and the first-in-first-out principle, the end signal corresponding to the control instruction is returned to the external control module.
The method of claim 49, further comprising:

If it is determined that the currently received end signal is not currently to be sent according to the receiving order of the control instructions and the first-in-first-out principle, the currently received end signal is buffered.
The method according to claim 29, wherein the data to be processed includes the object to be processed and operating parameters.
The method according to claim 51, wherein the object to be processed comprises any one of the following: image, audio or text;

The operating parameters include any one of the following: a convolution kernel, a pooling parameter, or an activation function.
The method according to claim 51, wherein the data loading module comprises an object loading unit and a parameter loading unit;

The loading data through the data loading module in response to the control instruction for the processing module to perform data processing includes:

In response to the i-th control instruction, load the object to be processed corresponding to the i-th control instruction through the object loading unit; and,

In response to the i-th control instruction, the operating parameter corresponding to the i-th control instruction is loaded by the parameter loading unit.
The method according to claim 29, wherein the performing data processing by the processing module in response to the control instruction comprises:

In response to the i-th control instruction, write the to-be-processed data corresponding to the i-th control instruction into a systolic array, and perform operations on the to-be-processed data through the systolic array to obtain a processing result.
The method according to claim 51, wherein the performing data processing by the processing module in response to the control instruction comprises:

In response to the i-th control instruction, the object to be processed and the operating parameters corresponding to the i-th control instruction are respectively written into the systolic array, and the object to be processed and the operating parameter are combined through the systolic array. The parameters are calculated and the processing result is obtained.
The device of claim 29, wherein:

The control module includes an instruction slot for caching the control instruction; the instruction slot includes a set of instruction cache flags and control state signals;

Wherein, the instruction cache flag is used to indicate whether the instruction cached in the instruction slot is valid; the set of control status signals is used to indicate the working status of the corresponding module, or indicate other instruction slots related to the instruction slot Working status.
An accelerator, characterized by comprising the data processing device according to any one of claims 1 to 28.