WO2023093335A1 - Data processing circuit, artificial intelligence chip, and data processing method and apparatus - Google Patents

Abstract

Provided in the embodiments of the present disclosure are a data processing circuit, an artificial intelligence chip, and a data processing method and apparatus. Instructions are processed by means of a plurality of processing units, and each processing unit independently determines, on the basis of information of a first instruction received by the processing unit and information of a second instruction sent by the processing unit, whether there are data hazards between the instructions processed by the processing units. When there are data hazards between the instructions processed by some of the processing units, the instructions can still be sent by means of other processing units, such that instruction congestion is reduced.

Description

Data processing circuit and artificial intelligence chip, data processing method and device

Related Publication Cross-References

This disclosure claims priority to the Chinese patent publication with application number 202111435830.3 filed on November 29, 2021, the entire content of which is incorporated herein by reference.

technical field

The present disclosure relates to the technical field of integrated circuit design, in particular to a data processing circuit, an artificial intelligence chip, a data processing method and a device.

Background technique

With the continuous development of artificial intelligence and high-performance computing, the amount of data that the corresponding processing system needs to process is increasing. During processing, a large amount of data needs to be moved between internal storage space and external storage space. Data hazards may occur during the transfer process, that is, at least two instructions indicate to read and write data to the same storage address, which may lead to data read and write errors. Therefore, it is necessary to control the read and write commands to prevent data read and write errors. The related technology generally only judges whether there is a data hazard between the preceding and following two instructions. Once a data hazard occurs, subsequent commands cannot be issued, which easily leads to command congestion.

Contents of the invention

The disclosure provides a data processing circuit, an artificial intelligence chip, a data processing method and a device.

According to the first aspect of the embodiments of the present disclosure, there is provided a data processing circuit, the circuit includes a plurality of processing units, and each of the plurality of processing units includes: an instruction queue for processing received The first instruction is cached; the sent instruction queue is used to cache the information of each second instruction in at least one second instruction that has been sent; the detection unit is used for based on the information of the first instruction and each second instruction information to detect whether the first instruction satisfies the instruction sending condition, and if the instruction sending condition is met, take the first instruction from the instruction queue and send it; the information includes read-write type information and at least one of the address information.

Optionally, the detection unit is specifically configured to: when the sent instruction queue is not full, based on the read/write type information of the first instruction, the read/write type information of the second instruction, the The address information of the first instruction and the address information of the second instruction detect whether the first instruction satisfies the instruction sending condition; when the sent instruction queue is full, read and write based on the first instruction The type information and the read/write type information of the second instruction detect whether the first instruction satisfies the instruction sending condition.

Optionally, when the sent instruction queue is not full, the detection unit is specifically configured to: address information of each second instruction in the sent instruction queue is different from that of the first instruction In the case of address information, it is determined that the first instruction satisfies the instruction sending condition; there is at least one potential conflicting instruction whose address information is the same as the address information of the first instruction in the sent instruction queue, but each In the case where the read/write type of the potential conflicting instruction is the same as the read/write type of the first instruction, it is determined that the first instruction satisfies the instruction sending condition; there is at least one of the potential conflicting instructions in the sent instruction queue In the case of conflicting instructions, and there is at least one conflicting instruction whose read/write type is different from that of the first instruction among the at least one potential conflicting instruction, it is determined that the first instruction does not satisfy the instruction sending condition.

Optionally, in the case that the sent instruction queue is full, the detecting unit is specifically configured to: there is at least one second instruction in the sent instruction queue that has the same read/write type information as the first instruction In the case of different read-write type information, it is determined that the first instruction does not meet the instruction sending condition; the read-write type information of each second instruction in the sent instruction queue is the same as the read-write type information of the first instruction If the information is the same, it is determined that the first instruction satisfies the instruction sending condition.

Optionally, different processing units are used to process instructions sent by different thread groups.

Optionally, the circuit further includes a first instruction distribution unit, configured to: receive first instructions sent by each thread group, each first instruction sent by each thread group carries identification information of the corresponding thread group; The identification information carried in the first instruction sent by the thread group distributes the first instruction sent by each thread group to the corresponding processing unit.

Optionally, instructions sent by different processing units have different priorities; the circuit further includes: an instruction arbitration unit, configured to receive the first instructions sent by each processing unit, and based on the first instruction sent by each processing unit The priority is to send the first instruction received in sequence.

Optionally, the first instruction includes a storage address of the bypass information, and the bypass information corresponding to the first instruction is stored under the storage address of the bypass information; the circuit further includes: a second instruction distribution A unit, configured to decouple the original instruction carrying the bypass information, obtain the decoupled original instruction and the bypass information, store the bypass information under the storage address of the bypass information, and based on the solution generating the first instruction from the coupled original instruction and the storage address of the bypass information, and sending the first instruction to the instruction queue in the processing unit; If the first instruction satisfies the instruction sending condition, generate a target instruction based on the first instruction and the storage address of the bypass information, and send the target instruction.

Optionally, the bus control unit is further configured to: clear the information of the processed second instruction from the sent instruction queue.

Optionally, the first instruction includes a write instruction, and the bypass information includes first bypass information corresponding to the write instruction; the circuit further includes: a first storage unit, configured to store the first bypass information road information.

Optionally, the bus control unit is configured to: extract the storage address of the first bypass information from the write instruction, and acquire the first bypass information from the storage address of the first bypass information , generating the target instruction based on the first bypass information and the write instruction, and sending the target instruction.

Optionally, the first instruction includes a read instruction, and the bypass information includes second bypass information corresponding to the read instruction; the circuit further includes: a second storage unit, configured to store the second bypass information road information.

Optionally, the bus control unit is further configured to: use the read instruction as the target instruction.

Optionally, the bus control unit is further configured to: receive the target data read by the read instruction, the target data carrying the storage address of the second bypass information; write the target data into the In the storage address of the second bypass information in the second storage unit.

Optionally, the bus control unit is further configured to: clear the bypass information corresponding to the first instruction from the storage address of the bypass information when the processing of the first instruction is completed.

Optionally, when the total number of the second instructions exceeds the length of the sent instruction queue, all items of information in the sent instruction queue are invalidated.

Optionally, the circuit further includes a statistical unit for counting the following information: the total number of the second instructions; the number of processed instructions in each second instruction; and the read/write type information of each second instruction; The detection unit is configured to: in the case that all items of information in the sent instruction queue are invalid, and there is a second instruction whose read/write type information is different from the first instruction, based on the statistics of the statistical unit The information detects whether the first instruction satisfies the instruction sending condition.

Optionally, when all items of information in the sent command queue are invalid, and there are second commands whose read-write type information is different from the first command, if all the second commands are processed, It is determined that the first instruction satisfies the instruction sending condition.

Optionally, the information of the first instruction is written into the sent instruction queue when the first instruction is sent successfully.

According to the second aspect of the embodiments of the present disclosure, there is provided an artificial intelligence chip, including: the data processing circuit described in any embodiment of the present disclosure; and a control unit, configured to send instructions to the data processing circuit.

According to a third aspect of the embodiments of the present disclosure, there is provided a data processing method, which is applied to the detection unit included in each processing unit in the data processing circuit described in any embodiment of the present disclosure, the method includes: based on the The information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit detect whether the first instruction satisfies the instruction sending condition; When the instruction sending condition is satisfied, the first instruction is taken out from the instruction queue and sent; the information includes at least one of read-write type information and address information.

According to the fourth aspect of the embodiments of the present disclosure, there is provided a data processing device, which is applied to the detection unit included in each processing unit in the data processing circuit according to any embodiment of the present disclosure, the device includes: a detection module, It is used to detect whether the first instruction satisfies the instruction sending condition based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit; A sending module, configured to take out the first instruction from the instruction queue and send it when the first instruction satisfies the instruction sending condition; the information includes at least one of read-write type information and address information .

In the embodiment of the present disclosure, a plurality of processing units are used to process instructions, and each processing unit judges independently based on the information of the first instruction received by the processing unit and the information of the second instruction sent by the processing unit. Whether there is a data hazard between the instructions processed. In the case of data hazards between instructions processed by some of the processing units, instructions can still be sent through other processing units, thereby reducing instruction congestion.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Description of drawings

The drawings here show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solutions of the present disclosure.

FIG. 1 is a schematic diagram of a data processing circuit of an embodiment of the present disclosure.

Fig. 2 is a schematic diagram of the detection principle of the detection unit of the embodiment of the present disclosure.

FIG. 3 is a schematic diagram of a data processing circuit according to another embodiment of the present disclosure.

FIG. 4 is a schematic diagram of instruction over-issuance according to an embodiment of the present disclosure.

5A and 5B are schematic diagrams of decoupling and merging of instructions in an embodiment of the present disclosure, respectively.

FIG. 6 is a schematic diagram of an instruction sending process of an embodiment of the present disclosure.

FIG. 7 is an overall flowchart of an embodiment of the present disclosure.

Fig. 8 is a block diagram of an artificial intelligence chip according to an embodiment of the present disclosure.

FIG. 9 is a flowchart of a data processing method according to an embodiment of the present disclosure.

FIG. 10 is a block diagram of a data processing device of an embodiment of the present disclosure.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only, and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality.

It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present disclosure, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure, and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure are described below in conjunction with the accompanying drawings The program is described in further detail.

When data is moved between the internal storage space and the external storage space, data hazards may occur, that is, at least two instructions indicate to read and write data to the same storage address, which may lead to data read and write errors. Therefore, it is necessary to control the read and write commands to prevent data read and write errors. For example, address 1 in the external storage space stores data d1, and the control unit for data transfer generates two instructions, one instruction is a read instruction indicating to read data from address 1 and write it into the register, and the other One command is a write command indicating to write data d2 to address 1. The external storage space may include, but is not limited to, double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR) or high bandwidth memory (High Bandwidth Memory, HBM). The control unit may include, but is not limited to, a central processing unit (Central Processing Unit, CPU) or a graphics processing unit (Graphics Processing Unit, GPU). These two instructions are executed in different orders, which will produce different execution results. If the read command is executed first and then the write command is executed, the data written to the register is the original data d1 in address 1. If the write command is executed first and then the read command is executed, the data written to the register is the data d2 in address 1 modified by the write command. This kind of data read and write errors that may be caused by executing instructions of different read and write types on the same address is called data hazard. In order to reduce data hazards, it is necessary to control the sending process of read commands and write commands. However, the related technology generally only judges whether there is a data hazard between two instructions before and after. Once a data hazard occurs, subsequent commands cannot be issued, which easily leads to command congestion.

Based on this, an embodiment of the present disclosure provides a data processing circuit 100. Referring to FIG. 1 and FIG. 3, the circuit 100 includes a plurality of processing units 101, and each processing unit 101 in the plurality of processing units includes:

an instruction queue 1011, configured to cache the received first instruction;

Sent instruction queue 1012, configured to cache information of each second instruction in at least one second instruction that has been sent;

The detection unit 1013 is configured to detect whether the first instruction satisfies the instruction sending condition based on the information of the first instruction and the information of each second instruction, and if the instruction sending condition is satisfied, the taking the first command out of the queue and sending it;

The information includes at least one of read-write type information and address information.

In the related art, the instruction is generally stopped when there is a data hazard, and the instruction cannot be resent until the processing of the sent instruction is completed. The embodiment of the present disclosure adopts multiple independent processing units 101 , even if there is a data hazard among the instructions processed by one or some of the processing units, the instructions can still be sent through other processing units. It is only necessary to stop issuing instructions if there is a data hazard between instructions processed by all processing units. Therefore, the embodiments of the present disclosure can effectively improve instruction sending efficiency and reduce instruction congestion.

Since each processing unit independently judges whether there is a data hazard between the instructions processed by the processing unit based on the information of the first instruction received by the processing unit and the information of the second instruction sent by the processing unit. In the case of data hazards between instructions processed by some of the processing units, instructions can still be sent through other processing units, thereby reducing instruction congestion. It should be noted that when two instructions indicate to write data to the same address, a write conflict problem may occur, and the data processing circuit in the embodiment of the present disclosure may further include a processing module for avoiding the write conflict problem.

The above instruction queue 1011 may be a First In First Out (FIFO) queue, and each time the instruction queue 1011 receives the first instruction sent by the superior control unit, it may cache the first instruction. In the embodiment of the present disclosure, the upper-level control unit is, for example, the instruction distribution unit 102 shown in FIG. 3 , or the control unit 802 shown in FIG. 8 . The order in which the first instructions are cached in the instruction queue 1011 may be the same as the order in which the instruction queue 1011 receives the first instructions. The first instruction may include a read instruction for reading data from a certain address in the external storage space (for example, a hard disk). The first instruction may also include a write instruction for writing data to a certain address in the external storage space. The first instruction may include information corresponding to the first instruction, where the information includes address information and read/write type information. Wherein, the address information is used to indicate the address requested to be accessed by the first instruction. For a read command, the address information indicates the address where the data to be read is located (ie, the data source); for a write command, the address information indicates the address where the data needs to be written (ie, the data destination). The read/write type information is used to indicate whether the first instruction is a read instruction or a write instruction.

The information of each second instruction may be cached in the above-mentioned sent instruction queue 1012 . The second instruction refers to an instruction that has been successfully sent but not yet processed. It may be determined that the instruction has been processed after receiving a notification message returned for a second instruction indicating that the instruction has been processed, and the information of the second instruction is cleared in the sent instruction queue 1012 . The return condition of the notification message can be set based on actual conditions. For example, for a read command, the notification message may be returned by the data requester when the data requested by the read command is returned to the data requester. For another example, for a write command, when the data carried in the write command is written to the specified data receiver, the data receiver may return the notification message. Of course, the actual situation is not limited to the methods listed above.

The information of the second command may also include address information and read/write type information. For the meaning of the address information and the read/write type information included in the second instruction, please refer to the meaning of the address information and the read/write type information included in the first instruction, which will not be repeated here. The address information and read/write type information in the sent instruction queue 1012 may be cached correspondingly, that is, the address information and read/write type information of the same second instruction are cached as one piece of information.

The detecting unit 1013 may respectively compare the information of the first instruction with each piece of information cached in the sent instruction queue 1012, so as to determine whether there is a data hazard between the first instruction and the sent second instruction. If there is a data hazard between the first instruction and any second instruction, the first instruction is not sent. The first instruction is sent only when there is no data hazard between the first instruction and any second instruction. In the case of a data hazard, the detection unit 1013 may detect whether there is a data hazard again after a certain time interval until there is no data hazard and take the first instruction out of the instruction queue 1011 for transmission. Wherein, the time interval may be one clock cycle, or other durations.

The manner of judging whether there is a data hazard will be described below with reference to FIG. 2 . The detection unit 1013 may determine whether there is a data hazard based on at least one of the read/write type information and the address information. Specifically, it may first be determined whether the sent instruction queue 1012 is full (step 201). When the sent instruction queue is not full, it may be based on the read/write type information of the first instruction, the read/write type information of the second instruction, the address information of the first instruction, and the second instruction. The address information of the second instruction detects whether the first instruction satisfies the instruction sending condition.

Specifically, in the case where the address information of each second instruction in the sent instruction queue 1012 is different from the address information of the first instruction, it may be determined that the first instruction satisfies the instruction sending condition (step 202) , and send the first instruction (step 207). For example, assume that the information included in the sent instruction queue 1012 is as shown in Table 1:

Table 1 Sent command queue

information belongs to the instruction	Address information	Read and write type information
instruction 1	A1	read
command 2	A2	Write
command 3	A2	read
...	...	...

Wherein, instruction 1, instruction 2 and instruction 3 are the second instructions, the information {A1, read} included in the sent instruction queue 1012 is the information of instruction 1, {A2, write} is the information of instruction 2, {A3, Read} is the information of instruction 3. Assume further that the address information in the first instruction is A3, A3 is different from A1, and A3 is different from A2. Since the address information in the first command is different from the address information in the sent second commands, and the data read and write between different addresses are independent of each other, no matter the read and write data type of the first command and the What is the read-write data type of each second instruction? There is no data hazard between the first instruction and any second instruction, that is, the first instruction satisfies the instruction sending condition, so the first instruction can be sent.

Still assume that the information included in the sent instruction queue 1012 is as shown in Table 1, and assume that the address information in the first instruction is A1, since the first instruction and instruction 1 are directed to the same address, if the first instruction If the instruction is sent, if the read/write type information of the first instruction is different from that of instruction 1, a data read/write error may occur, that is, there is a data hazard. Therefore, in this case, it is necessary to combine the read-write type information and the address information to determine whether the first instruction satisfies the instruction sending condition.

In the case that the sent instruction queue 1012 has the same target address information as the address information of the first instruction, if the sent instruction queue 1012 has the read/write type information different from the first instruction target read/write type information, and the target read/write type information and the target address information belong to the same second instruction, it can be determined that the first instruction does not meet the instruction sending condition (step 203), so that the first instruction is not sent instruction (step 206).

In the case that there is the same target address information as the address information of the first instruction in the sent instruction queue 1012, if there is no read/write type information of the first instruction in the sent instruction queue 1012 different target read-write type information, wherein the target read-write type information and the target address information belong to the same second instruction, it can be determined that the first instruction satisfies the instruction sending condition (step 202), and send the first instruction (step 207).

For example, in the above embodiment, it is assumed that the read/write type information of the first instruction is write, since the address information of the first instruction is the same as the address information of instruction 1, and the read/write type information of the first instruction is the same as the read/write type information of instruction 1. The type information is different, so there is a data hazard, and the first command does not satisfy the command sending condition. Assuming that the read-write type information of the first instruction is read, there is no second instruction that satisfies the same address information of the second instruction as the address information of the first instruction and satisfies the read-write type information of the second instruction. Different from the read/write type information of the first instruction, therefore, there is no data hazard, and the first instruction satisfies the instruction sending condition.

In addition, when the read/write type information of each second instruction in the sent instruction queue 1012 is the same as the read/write type information of the first instruction, it may be determined that the first instruction satisfies the instruction sending condition. In a feasible implementation, when the sent instruction queue is not full, the detection unit may also use the following method to determine whether the first instruction satisfies the instruction sending condition, including: When the address information of the second instruction is different from the address information of the first instruction, it is determined that the first instruction satisfies the instruction sending condition; the address information and the first instruction exist in the sent instruction queue. In the case of at least one potentially conflicting instruction with the same address information of the instruction, but the read/write type of each potential conflicting instruction is the same as the read/write type of the first instruction, it is determined that the first instruction satisfies the instruction sending condition ; In the case where there is at least one potentially conflicting instruction in the sent instruction queue, and there is at least one conflicting instruction whose read/write type is different from the read/write type of the first instruction in the at least one potentially conflicting instruction , determining that the first instruction does not satisfy the instruction sending condition.

When the sent instruction queue 1012 is full, it may be detected whether the first instruction satisfies the instruction sending condition based on the read/write type information of the first instruction and the read/write type information of the second instruction. Specifically, in the case where the read/write type information of each second instruction in the sent instruction queue 1012 is the same as the read/write type information of the first instruction, it is determined that the first instruction satisfies the instruction sending condition ( Step 204), and send the first instruction (step 207).

For example, referring to (1) at the top of FIG. 4 , it is assumed that each item of read/write type information in the sent instruction queue 1012 is write, and the read/write type information of the first instruction is also write, regardless of whether each write instruction Whether the address information (address 1, address 2, . . . , address n) is the same, there is no data hazard between the various write instructions. Therefore, in this case, the first instruction can be sent directly. In the same way, referring to (2) at the top of FIG. 4 , assuming that all the read-write type information in the sent instruction queue 1012 is read, and the read-write type information of the first instruction is also read, it is also possible to directly write to the first instruction A command is sent. If the first instruction continues to be sent when the sent instruction queue 1012 is full, the number of sent instructions exceeds the length of the sent instruction queue 1012, and this situation may be called instruction overissue. Through instruction oversending, it is possible to improve the sending efficiency of multiple consecutive instructions of the same read and write type, and further reduce instruction congestion.

When the read/write type information of at least one second instruction (for example, the jth second instruction) is different from the read/write type information of the first instruction in the sent instruction queue 1012, it is determined that the An instruction does not satisfy the instruction sending condition (step 205), so the first instruction is not sent (step 206). For example, each item of read-write type information in the sent instruction queue 1012 includes both read and write; or, each item of read-write type information in the sent instruction queue 1012 is read, but the read-write type information of the first instruction is write; or, all the read-write type information in the sent instruction queue 1012 is write, but the read-write type information of the first instruction is read. In the above three cases, regardless of the address information in the first command and the second command, the first command is not sent.

It should be noted that, when over-issuing instructions, the number of instructions that have been sent but not yet processed has exceeded the length of the sent instruction queue 1012 . For example, the number of instructions that have been sent and have not been processed is 5, and the information of a total of 4 instructions can be cached in the sent instruction queue 1012 (that is, the length of the sent instruction queue 1012 is 4). In this case, the information of the super-issued command cannot be stored in the sent command queue 1012 after the super-issued command is sent, and whether there is a data hazard cannot be determined based on the information in the sent command queue 1012 . For example, assume that the super-issue instruction S0 is a read instruction for address A0, and the information {A0, read} of instruction S0 is not cached in the sent instruction queue 1012, and assume that the information in the sent instruction queue 1012 is as shown in the table 2 shows:

Table 2

Address information	Read and write type information
A1	read
A2	read
A3	read

When the processing of the second instruction corresponding to the information {A1, read} is completed, the information {A1, read} is cleared from the sent instruction queue 1012, and the sent instruction queue 1012 is not full at this time. If the instruction queue 1011 receives an instruction Sk whose information is {A0, write}, it can be known from the above-mentioned method of judging data hazard that the same address information as the address information of the instruction Sk does not exist in the sent instruction queue 1012, therefore, If it is determined based on the information in the sent command queue 1012 that the command Sk satisfies the command sending condition. But in fact, because there is an instruction S0 with different read and write type information and the same address information as the instruction Sk among the instructions that have been sent and have not been processed, there is a data hazard between the instruction S0 and the instruction Sk, so the actual instruction Sk is The conditions for sending the command are not met. That is to say, in the case of command oversending, if it is still judged based on the information in the sent command queue 1012 whether the first command satisfies the command sending condition, a wrong judgment result may be obtained.

In order to improve the judgment accuracy in the case of command oversending, when the total number of the second commands exceeds the length of the sent command queue, all items of information in the sent command queue can be set to invalid . In the case that all items of information in the sent instruction queue are invalid, and there is a second instruction whose read/write type information is different from that of the first instruction, the first instruction may be detected based on the statistical information of the sent instruction. Whether the instruction satisfies the sending condition of the instruction. In this case, the first instruction is sent only when all the sent second instructions have been processed; as long as there are unprocessed second instructions, the first instruction is not sent. In the embodiment of the present disclosure, after all the second instructions sent are processed, or after all the super-issued instructions are processed, the invalid setting for each item of information in the sent instruction queue can be canceled.

In some embodiments, the circuit further includes a statistical unit 1014, configured to count the following information: the total number of the second instructions; the number of instructions that have been processed in each second instruction; and the reading and writing of each second instruction type information. The detection unit 1013 may base on the statistics of the statistical unit 1014 in the case that all items of information in the sent command queue 1012 are invalid and there is a second command whose read/write type information is different from the first command. The information detects whether the first instruction satisfies an instruction sending condition.

The detection unit 1013 may first determine whether the read/write type information of the first instruction is the same as that of the sent second instructions. In the case that the read and write type information is different, the total number of sent second instructions and the number of instructions processed in each second instruction are obtained from the statistics unit 1014 . Only when the total number of sent second instructions is equal to the number of processed instructions, it is determined that the first instruction satisfies the instruction sending condition; otherwise, it is judged that the first instruction does not meet the instruction sending condition.

In some embodiments, different processing units 101 are used to process instructions sent by different thread groups. Wherein, each processing unit 101 may be used to process instructions sent by one or more thread groups, and the thread groups responsible for different processing units may be the same or different. For example, processing unit 0 is used to process instructions sent by thread group 0, processing unit 1 is used to process instructions sent by thread group 1 and thread group 2, and processing unit 2 is used to process instructions sent by thread group 3 and thread group 4. In order to distribute the instructions sent by different thread groups to corresponding processing units, the circuit further includes a first instruction distribution unit, configured to receive the first instructions sent by each thread group, and the first instructions sent by each thread group carry Corresponding to the identification information of the thread group; distributing the first instruction sent by each thread group to the corresponding processing unit 101 based on the identification information carried in the first instruction sent by each thread group respectively. Identification information such as thread group number.

In some embodiments, instructions sent by different processing units 101 have different priorities; the circuit further includes: an instruction arbitration unit 103, configured to receive the first instruction sent by each processing unit 101, and based on each processing unit 101 The priority of the first instruction to be sent is to send the first instructions sent by each processing unit 101 in sequence.

In some embodiments, the first instruction includes a storage address of the bypass information, and the bypass information corresponding to the first instruction is stored under the storage address of the bypass information; the circuit further includes: a second an instruction distribution unit, configured to decouple the original instruction carrying the bypass information, obtain the decoupled original instruction and the bypass information, and store the bypass information under the storage address of the bypass information, generating the first instruction based on the decoupled original instruction and the storage address of the bypass information, and sending the first instruction to the instruction queue; a bus control unit, configured to based on the first instruction Generate a target instruction, and send the target instruction.

It should be noted that, in the above embodiments, the first instruction distribution unit for distributing instructions to each processing unit 101 and the second instruction distribution unit for decoupling original instructions may be the same instruction distribution unit 102, However, in practical applications, instruction distribution and instruction decoupling may also be performed separately through different instruction distribution units. In one possible implementation, raw instructions may be issued by thread groups.

The original instruction may carry some bypass information, which has nothing to do with the process of judging whether the first instruction satisfies the instruction sending condition. For example, for a write instruction, the bypass information may include, but not limited to, data to be written, identification information for identifying valid bits of the data to be written, and the like. For a read instruction, the bypass information may include, but not limited to, a target address for data reading, an address of a register storing the target address, and the like. If the bypass information is always carried in the instruction, each processing unit needs additional storage space to store the bypass information, which will increase the area and power consumption of the data processing circuit. Therefore, in this embodiment, the bypass information is decoupled from the instruction, the bypass information is stored separately, and the data hazard detection is performed based on the part of the instruction except the bypass information, thereby reducing the area and power consumption of the circuit, and at the same time Reduce crossbar (crossbar matrix) complexity. Only when it is determined that the first instruction satisfies the instruction sending condition, the bypass information is combined with the first instruction again to obtain the target instruction, and the target instruction is sent.

Referring to FIG. 5A and FIG. 5B, first, after receiving the original instruction including bypass information and instruction information, the instruction distribution unit 102 can extract the bypass information from the original instruction, and send the bypass information to the bypass information storage unit. Here, the bypass information corresponding to the read command and the bypass information corresponding to the write command may be stored separately. Wherein, referring to FIG. 3 , the bypass information corresponding to the write command may be stored in the first storage unit 105 , and the bypass information corresponding to the read command may be stored in the second storage unit 106 . The bypass information storage unit may return the storage address of the bypass information to the instruction dispatch unit 102 . For a write instruction, the storage address of the bypass information is the storage address of the bypass information corresponding to the write instruction in the first storage unit 105; for a read instruction, the storage address of the bypass information is the corresponding bypass information of the read instruction. The storage address of the information in the second storage unit 106 . The instruction distribution unit 102 may combine the storage addresses of the instruction information and the bypass information to generate a first instruction, and send the first instruction to the bus control unit 104 through the instruction arbitration unit 103 .

The bus control unit 104 can generate the final target instruction and send it. Wherein, for the write command and the read command, the bus control unit 104 may generate the target command in different ways. Specifically, for the write command, since the bypass information such as the data to be written needs to be sent to the target address together with the command, so as to write the data to be written into the target address, the bus control unit 104 can obtain the data from the write command Extracting the storage address of the first bypass information corresponding to the write instruction, obtaining the first bypass information from the storage address of the first bypass information, based on the first bypass information and the instruction of the write instruction The information generates the target instruction and sends the target instruction, as shown in FIG. 5A . For the read command, the data storage unit that stores the data to be read does not need to know where the data to be read will be read. Therefore, the bus control unit 104 can directly use the storage address of the bypass information and the instruction information of the read command as The target command is sent, as shown in Figure 5B.

Further, for the read command, the bus control unit 104 may also receive target data read by the read command, where the target data may carry a storage address of the second bypass information corresponding to the read command. The bus control unit 104 may write the target data into the storage address of the second bypass information corresponding to the read command, so that the data requester (for example, a register) can read the target data from the storage address of the second bypass information.

In some embodiments, the bus control unit 104 is further configured to clear the bypass information corresponding to the first instruction from the storage address of the bypass information when the processing of the first instruction is completed, so that The storage unit for storing bypass information can free up storage space to store bypass information of other instructions. Wherein, when the processing of the read instruction is completed, the bypass information corresponding to the read instruction can be cleared from the storage address of the bypass information corresponding to the read instruction; Clear the bypass information corresponding to the write command in the storage address of the information.

In some embodiments, if the first instruction is sent successfully, write the information of the first instruction into the sent instruction queue. Referring to Fig. 6, in the instruction sending process, assume that in the clock cycle T1, the sent instruction queue includes the information of instruction 1, the information of instruction 2 and the information of instruction 3 and the sent instruction queue is not full; the instruction queue includes the instruction 4. Instruction 5 and Instruction 6. Then it is possible to extract the information of the command at the forefront in the command queue (that is, command 4), and based on the information of command 4 and the information of command 1, command 2 and command 3, check whether command 4 satisfies the command sending condition. If satisfied, send instruction 4. If the command 4 is sent successfully, the information of the command 4 is stored in the sent command queue in the clock cycle T2. In the clock cycle T3, the information of the processed instruction (assumed to be the information of instruction 1) can also be cleared from the sent instruction queue. It should be noted that the clock period T2 may be before the clock period T3 or after the clock period T3, which is not limited in the present disclosure. In addition, the processing completion time of the instruction sent earlier may be earlier or later than the processing completion time of the instruction sent later, that is, the order in which the information of each instruction is stored in the sent instruction queue is the same as the information of each instruction from the sent instruction The order in which the queues are cleared is not necessarily the same.

In some embodiments, the bus control unit 104 is further configured to clear the information of the processed second instruction from the sent instruction queue. The sent instruction queue 1012 may send the cached information of the second instruction and/or the cached address of the information to the bus control unit 104 . In this way, when the processing of the second instruction to which the information belongs is completed, the bus control unit 104 may send an enable signal to the sent instruction queue 1012, and the enable signal may carry information such as the second instruction whose processing is completed cache address in the sent instruction queue 1012, so that the sent instruction queue 1012 can clear the information in the corresponding cache address in response to the enable signal.

Referring to FIG. 7 , it is an overall flowchart of an embodiment of the present disclosure. This process can be realized by the circuit shown in Fig. 3 . First, the instruction distribution unit 102 may distribute instructions to the instruction queues 1011 in each processing unit 101 according to the thread group number (S1). The instruction distribution unit 102 may also decouple the instruction information and the bypass information in the instruction, and store the decoupled bypass information into the storage address of the bypass information (S2). The storage address and instruction information are combined to generate a first instruction. The detecting unit 1013 can judge whether there is a data hazard (that is, whether the first instruction satisfies the instruction sending condition) based on the information of the first instruction and the information of the second instructions stored in the sent instruction queue, or based on the statistics of the statistics unit 1014 The information judges whether there is a data risk (S3). If there is a data hazard, the first instruction is still cached in the instruction queue 1011, and periodically re-judged whether there is a data hazard (S4). If there is no data hazard, the first instruction is taken from the instruction queue 1011 and sent to the instruction arbitration unit 103 (S5). The processing flow of each processing unit 101 is the same, and will not be described one by one here.

The instruction arbitration unit 103 may sequentially send each first instruction to the bus control unit 104 according to the priorities of the first instructions sent by each processing unit 101 ( S6 ). The bus control unit 104 may generate a target command based on the first command received from the command arbitration unit 103, and send the target command to a corresponding target address (S7).

In addition, the bus control unit 104 may also clear the information corresponding to the second command in the sent command queue 1012 (S8), and clear the bypass information (S9) when receiving the processing completion information for a certain second command. ). The detection unit 1013 may also write the information of the first instruction into the sent instruction queue 1012 when the first instruction is successfully sent through the bus control unit 104 ( S10 ).

The execution order of the above steps is not limited to the order shown in the figure, for example, the order of steps S8 and S9 can be interchanged, the order of step S10 and step S8 or S9 can be interchanged, etc.

In the embodiment of the present disclosure, multiple processing units process data hazards in parallel. When some processing units have data hazards, other processing units without data hazards can still send instructions. In addition, the present disclosure allows consecutive super-issuance of instructions of the same read and write type. Instruction over-issuance can effectively improve system memory access performance. The disclosed method can be used to realize efficient multi-processing unit processing data adventure, improve the memory access performance of the system, and use the scalability and deformation of the disclosed method to reduce power consumption and reduce crossbar complexity.

As shown in FIG. 8 , the present disclosure also provides an artificial intelligence chip, including: a data processing circuit 801 ; and a control unit 802 configured to send instructions to the data processing circuit 801 .

The data processing circuit 801 may adopt the data processing circuit described in any embodiment of the present disclosure. For details of the data processing circuit 801 in this embodiment, refer to the foregoing embodiments for details, and details are not repeated here.

Referring to Fig. 9, an embodiment of the present disclosure also provides a data processing method, which is applied to the detection unit included in each processing unit in the data processing circuit described in any embodiment of the present disclosure, the method comprising:

Step 901: Based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit, detect whether the first instruction satisfies instruction sending condition;

Step 902: If the first instruction satisfies the instruction sending condition, take the first instruction from the instruction queue and send it;

The information includes at least one of read-write type information and address information.

Optionally, the detecting based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit whether the first instruction satisfies Instruction sending conditions, including: when the sent instruction queue is not full, based on the read/write type information of the first instruction, the read/write type information of the second instruction, and the address of the first instruction information and the address information of the second instruction to detect whether the first instruction satisfies the instruction sending condition; when the sent instruction queue is full, based on the read/write type information of the first instruction and the second instruction The read/write type information of the second instruction detects whether the first instruction satisfies the instruction sending condition.

Optionally, when the sent instruction queue is not full, the read/write type information based on the first instruction, the read/write type information of the second instruction, the address of the first instruction information and the address information of the second instruction to detect whether the first instruction satisfies the instruction sending condition, including: the address information of each second instruction in the sent instruction queue is different from the address information of the first instruction In the case of , it is determined that the first instruction satisfies the instruction sending condition; there is at least one potential conflicting instruction whose address information is the same as the address information of the first instruction in the sent instruction queue, but each of the potential conflicting instructions When the read/write type of the first instruction is the same as the read/write type of the first instruction, it is determined that the first instruction satisfies the instruction sending condition; there is at least one of the potential conflict instructions in the sent instruction queue, and If there is at least one conflicting instruction whose read/write type is different from that of the first instruction among the at least one potential conflicting instruction, it is determined that the first instruction does not satisfy the instruction sending condition.

Optionally, when the sent instruction queue is full, the detection based on the read/write type information of the first instruction and the read/write type information of the second instruction whether the first instruction satisfies the instruction The sending condition includes: in the case that there is at least one second instruction in the sent instruction queue whose read/write type information is different from the read/write type information of the first instruction, determining that the first instruction does not satisfy instruction sending Condition: when the read/write type information of each second instruction in the sent instruction queue is the same as the read/write type information of the first instruction, it is determined that the first instruction satisfies the instruction sending condition.

Optionally, different processing units are used to process instructions sent by different thread groups.

Optionally, the circuit further includes a statistical unit for counting the following information: the total number of the second instructions; the number of processed instructions in each second instruction; and the read/write type information of each second instruction; Detecting whether the first instruction satisfies the instruction sending condition based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit, It includes: in the case that all items of information in the sent instruction queue are invalid, and there is a second instruction whose read-write type information is different from that of the first instruction, detecting the Whether the first instruction satisfies the instruction sending condition.

Optionally, the detecting based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit whether the first instruction satisfies Instruction sending conditions include: when all items of information in the sent instruction queue are invalid, and there are second instructions whose read-write type information is different from the first instruction, if each second instruction is processed Completed, it is determined that the first instruction satisfies the instruction sending condition.

Optionally, the information of the first instruction is written into the sent instruction queue when the first instruction is sent successfully.

For details of the foregoing method embodiments, refer to the aforementioned embodiments of the data processing circuit, and details are not repeated here.

Referring to FIG. 10 , an embodiment of the present disclosure also provides a data processing device, which is applied to the detection unit included in each processing unit in the data processing circuit described in any embodiment of the present disclosure, and the device includes:

A detection module 1001, configured to detect whether the first instruction satisfies the requirements based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit. Instruction sending condition;

A sending module 1002, configured to take out the first instruction from the instruction queue and send it when the instruction sending condition is satisfied;

The information includes at least one of read-write type information and address information.

In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the method embodiments above, and its specific implementation can refer to the description of the method embodiments above. For brevity, here No longer.

It can be known from the above description of the implementation manners that those skilled in the art can clearly understand that the embodiments of this specification can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the essence of the technical solutions of the embodiments of this specification or the part that contributes to related technologies can be embodied in the form of software products, and the computer software products can be stored in storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this specification.

The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer, which may take the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, e-mail device, game control device, etc. desktops, tablets, wearables, or any combination of these.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the functions of each module may be integrated in the same or multiple software and/or hardware implementations. Part or all of the modules can also be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

The above is only the specific implementation of the embodiment of this specification. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the embodiment of this specification, some improvements and modifications can also be made. These Improvements and modifications should also be regarded as the scope of protection of the embodiments of this specification.

Claims (17)

1. A data processing circuit, characterized in that the circuit includes a plurality of processing units, and each processing unit in the plurality of processing units includes:

   an instruction queue, configured to cache the received first instruction;

   A sent instruction queue, configured to cache information of each of the at least one second instruction that has been sent, wherein the information includes at least one of read-write type information and address information;

   A detection unit, configured to detect whether the first instruction satisfies the instruction sending condition based on the information of the first instruction and the information of each second instruction, and if the instruction sending condition is satisfied, from the The first instruction is taken out from the instruction queue and sent.
2. The circuit according to claim 1, wherein the detection unit is specifically used for:

   If the sent instruction queue is not full, based on the read/write type information of the first instruction, the read/write type information of the second instruction, the address information of the first instruction, and the second instruction The address information of the instruction detects whether the first instruction satisfies the instruction sending condition;

   When the sent instruction queue is full, it is detected whether the first instruction satisfies the instruction sending condition based on the read/write type information of the first instruction and the read/write type information of the second instruction.
3. The circuit according to claim 2, wherein when the sent instruction queue is not full, the detection unit is specifically used for:

   When the address information of each of the second instructions in the sent instruction queue is different from the address information of the first instruction, determining that the first instruction satisfies the instruction sending condition;

   There is at least one potential conflict instruction whose address information is the same as that of the first instruction in the sent instruction queue, but the read/write type of each of the potential conflict instructions is the same as the read/write type of the first instruction In the case of , it is determined that the first instruction satisfies the instruction sending condition;

   In the case where there is at least one potentially conflicting instruction in the sent instruction queue, and there is at least one conflicting instruction whose read/write type is different from the read/write type of the first instruction in the at least one potentially conflicting instruction, It is determined that the first instruction does not satisfy the instruction sending condition.
4. The circuit according to claim 2 or 3, wherein when the sent instruction queue is full, the detection unit is specifically configured to:

   If there is at least one second instruction in the sent instruction queue whose read/write type information is different from that of the first instruction, it is determined that the first instruction does not satisfy the instruction sending condition ;

   If the read/write type information of each second instruction in the sent instruction queue is the same as the read/write type information of the first instruction, it is determined that the first instruction satisfies the instruction sending condition.
5. The circuit according to any one of claims 1 to 4, wherein different processing units are used to process instructions sent by different thread groups, and the circuit also includes a first instruction distribution unit, configured to:

   receiving the first instruction sent by each thread group, the first instruction sent by each thread group carries the identification information of the corresponding thread group;

   The first instructions sent by each thread group are distributed to corresponding processing units based on the identification information carried in the first instructions sent by each thread group respectively.
6. The circuit according to any one of claims 1 to 5, wherein the circuit further comprises:

   An instruction arbitration unit, configured to receive the first instructions sent by each of the processing units, and based on the priority of the first instructions sent by each of the processing units, sequentially send the received first instructions .
7. The circuit according to any one of claims 1 to 6, wherein the circuit further comprises:

   The second instruction distribution unit is configured to decouple the original instruction carrying the bypass information, obtain the decoupled original instruction and the bypass information, and store the bypass information under the storage address of the bypass information, generating the first instruction based on the decoupled original instruction and the storage address of the bypass information, and sending the first instruction to the instruction queue in the processing unit;

   A bus control unit, configured to generate a target command based on the first command and the storage address of the bypass information when the first command satisfies the command sending condition, and send the target command.
8. The circuit according to claim 7, wherein the bus control unit is also used for:

   clearing the information of the processed second instruction from the sent instruction queue;

   and / or

   When the processing of the first instruction is completed, the bypass information corresponding to the first instruction is cleared from the storage address of the bypass information.
9. The circuit according to claim 7 or 8, wherein the first instruction includes a write instruction, and the bypass information includes first bypass information corresponding to the write instruction; a first storage unit of the first bypass information;

   and / or

   The first instruction includes a read instruction, and the bypass information includes second bypass information corresponding to the read instruction; the circuit further includes a second storage unit for storing the second bypass information.
10. The circuit according to claim 9, wherein the bus control unit is also used for:

    receiving the target data read by the read instruction, the target data carrying the storage address of the second bypass information;

    writing the target data into a storage address of the second bypass information in the second storage unit.
11. The circuit according to any one of claims 7 to 10, wherein when the total number of the second instructions exceeds the length of the sent instruction queue, the items in the sent instruction queue Information is invalidated.
12. The circuit according to claim 11, characterized in that the circuit also includes a statistical unit for counting the following information:

    the total number of said second instructions;

    the number of completed orders in each second order; and

    Read and write type information of each second instruction;

    The detection unit is configured to: in the case that all items of information in the sent instruction queue are invalid, and there is a second instruction whose read/write type information is different from the first instruction, based on the statistics of the statistical unit The information is used to detect whether the first instruction satisfies the instruction sending condition.
13. The circuit according to claim 12, wherein the detection unit is also used for:

    In the case that all items of information in the sent instruction queue are invalid, and there is a second instruction whose read-write type information is different from the first instruction, if all the second instructions are processed, it is determined that the The first instruction satisfies the instruction sending condition.
14. The circuit according to any one of claims 1 to 13, wherein the information of the first instruction is written into the sent instruction queue when the first instruction is sent successfully.
15. An artificial intelligence chip is characterized in that, comprising:

    The data processing circuit according to any one of claims 1 to 14; and

    A control unit, configured to send instructions to the data processing circuit.
16. A data processing method, characterized in that it is applied to the detection unit included in each processing unit in the data processing circuit according to any one of claims 1 to 14, the method comprising:

    Detecting whether the first instruction satisfies the instruction based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit Sending conditions, wherein the information includes at least one of read-write type information and address information;

    If the first instruction satisfies the instruction sending condition, the first instruction is taken out from the instruction queue and sent.
17. A data processing device, characterized in that it is applied to the detection unit included in each processing unit in the data processing circuit according to any one of claims 1 to 14, the device comprising:

    A detection unit, configured to detect the first instruction based on the information of the first instruction in the instruction queue corresponding to the processing unit and the information of each second instruction in the sent instruction queue corresponding to the processing unit Whether the instruction sending condition is satisfied, wherein the information includes at least one of read-write type information and address information;

    A sending unit, configured to take out the first instruction from the instruction queue and send it if the first instruction satisfies the instruction sending condition.

Images