KR20240064470A - Task Manager, Neural processing device and Method for Processing task dependency thereof - Google Patents

Abstract

The present invention discloses a task manager, a neural processing device, and a task dependency processing method thereof. The task manager includes a task buffer for receiving a first task of a different first type and a second task of a second type, a 1_1 queue for receiving a first task descriptor for the first task from the task buffer, and the task A 1_2 queue for receiving a second task descriptor for the second task from a buffer, a dependency checker for checking the dependencies of the first and second task descriptors, and the first task from the dependency checker. A 2_1 queue for receiving a descriptor and a 2_2 queue for receiving the second task descriptor from the dependency checker, wherein the dependency checker includes a plurality of dependency matrices, and the plurality of dependencies Each of the time matrices includes first and second rows corresponding to the first and second types, respectively, and first and second columns corresponding to the first and second types, respectively.

Description

Task Manager, Neural processing device and Method for Processing task dependency thereof}

The present invention relates to a task manager, a neural processing device, and a task dependency processing method thereof. Specifically, the present invention relates to a task manager, a neural processing device, and a task dependency processing method that improves the efficiency of overall task dependency setting and dependency checking by using a dummy descriptor.

Over the past few years, artificial intelligence (AI) technology has been mentioned as the most promising technology worldwide as a core technology of the 4th Industrial Revolution. The biggest problem with these artificial intelligence technologies is computing performance. For artificial intelligence technology that realizes human learning, reasoning, perception, and natural language translation abilities, the most important thing is to quickly process a lot of data.

The central processing unit (CPU) or graphics processing unit (GPU) of existing computers were used for deep learning learning and inference in early artificial intelligence, but deep learning learning and inference tasks with high workloads were used. Due to its limitations, the Neural Processing Unit (NPU), which is structurally specialized for deep learning tasks, is gaining attention.

This neural network processing device has a plurality of calculation units inside, and each calculation unit operates in parallel to increase calculation efficiency. At this time, the method of distributing and managing tasks to each computing device may be important. Multiple tasks may have dependencies on each other, and setting and checking these dependencies is an essential part of task performance.

In particular, there is a need for a method of performing dependency setting and checking for many-to-one or one-to-many dependencies rather than one-to-one.

Registered Patent Publication No. 10-2258566

The object of the present invention is to provide a task manager that efficiently processes task dependencies.

Another object of the present invention is to provide a neural processing device that efficiently processes task dependency.

Another object of the present invention is to provide a task dependency processing method for a neural processing device that efficiently processes task dependencies.

The objects of the present invention are not limited to the objects mentioned above, and other objects and advantages of the present invention that are not mentioned can be understood by the following description and will be more clearly understood by the examples of the present invention. Additionally, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof indicated in the patent claims.

A task manager according to some embodiments of the present invention for solving the above problem includes a task buffer that receives a first task of a different first type and a second task of a second type, and a task buffer that receives a first task of a different first type and a second task of a second type from the task buffer to the first task. A 1_1 queue that receives the first task descriptor for the second task, a 1_2 queue that receives the second task descriptor for the second task from the task buffer, and a dependency that checks the dependencies of the first and second task descriptors. It includes a dependency checker, a 2_1 queue that receives the first task descriptor from the dependency checker, and a 2_2 queue that receives the second task descriptor from the dependency checker, and the dependency checker includes a plurality of and a dependency matrix, wherein each of the plurality of dependency matrices includes first and second rows corresponding to the first and second types, respectively, and first and second rows corresponding to the first and second types, respectively. Includes 1st and 2nd rows.

In addition, the plurality of dependency matrices include a first group and a second group, the first group processes the first task descriptor, the second group processes the third task descriptor, The third task descriptor may be a descriptor for the third task of the first type.

In addition, the first and second tasks are each dependent on a third task, the dependency checker receives a third task descriptor and a third dummy set descriptor for the third task, and the third task descriptor And the third dummy set descriptor may be allocated to the dependency matrices of different groups.

In addition, a task passage including the task buffer, the 1_1 queue, the 1_2 queue, the dependency checker, the 2_1 queue, and the 2_2 queue, the first to third task descriptors, and the third It may contain a throw passage that checks out a dummy set descriptor.

Additionally, the task passage may transmit the first to third task descriptors to the core global.

In addition, it may further include a void core that receives the third dummy set descriptor from the task passage and transmits the dummy set descriptor to the dune passage.

Additionally, the void core may transmit the dummy set descriptor to the dark passage.

In addition, the fourth task is dependent on the fifth and sixth tasks, respectively, and the dependency checker receives a fourth task descriptor and a fourth dummy check descriptor for the fourth task, and the fourth task descriptor and The fourth dummy check descriptor may be assigned to dependency matrices of different groups.

Additionally, the fourth dummy check descriptor may be burned within the dependency checker.

A neural processing device according to some embodiments of the present invention for solving the above other problems includes at least one neural processor, a shared memory shared by the at least one neural processor, and the at least one neural processor and the shared memory store data. Exchange global interconnection, each of the at least one neural processor includes a task manager, the task manager receives a plurality of descriptors, checks dependencies between the plurality of descriptors, and transmits them to the neural core, The plurality of descriptors include a task descriptor, a dummy check descriptor, and a dummy set descriptor, and the dummy check descriptor may be burned within the task manager.

Additionally, if there are two or more subsequent task descriptors dependent on the task descriptor, the dummy set descriptor may be received one less than the number of subsequent task descriptors.

Additionally, when two or more task descriptors depend on one subsequent task descriptor, one less dummy check descriptor may be received than the number of task descriptors.

In addition, the task manager receives a task passage that receives the plurality of descriptors and delivers them to the neural core according to dependency, the task descriptor and the dummy set descriptor from the neural core, and determines whether or not the command processor is complete. It may contain a dun passage reporting as .

Additionally, the completion status may include completion of the task descriptor and may not include completion of the dummy set descriptor.

Additionally, the dawn passage may transmit a dependency field update request to the task passage when the task descriptor and the dummy set descriptor are checked out.

Additionally, the task passage may include a dependency checker that receives the dependency field update request and checks dependencies between the plurality of descriptors.

Additionally, the dependency checker may include a plurality of groups of dependency matrices.

A task dependency processing method according to some embodiments of the present invention to solve the above problem includes receiving a dependency set code, storing a dependency matrix according to the dependency set code, and storing the dependency matrix of the task descriptor. Comparing a dependency check code with a corresponding row of the dependency matrix, clearing the corresponding row if the dependency check code and the corresponding row are the same, and transmitting the task descriptor.

Additionally, receiving the dependency set code may include checking out the task descriptor and a dummy set descriptor and generating the dependency set code for the task descriptor.

Additionally, transmitting the task descriptor may include identifying a dummy check descriptor and a task descriptor, and incinerating the dummy check descriptor.

The task manager, neural processing device, and task dependency processing method of the present invention can be configured so that the task manager can check and process dependencies between tasks on its own without reporting to the command processor, and can handle different dependencies. It is configured so that dependencies can be set for a plurality of task descriptors provided in the time matrix, so that dependencies of the overall task can be configured.

Additionally, the case where multiple dependencies are set through a dummy descriptor can be effectively handled.

Accordingly, the processing speed and efficiency of the entire device can be maximized by minimizing traffic between the command processor and the task manager.

In addition to the above-described content, specific effects of the present invention are described below while explaining specific details for carrying out the invention.

1 is a block diagram for explaining a neural processing system according to some embodiments of the present invention.
FIG. 2 is a block diagram for explaining the neural processing device of FIG. 1 in detail.
FIG. 3 is a block diagram for explaining the neural core SoC of FIG. 2 in detail.
FIG. 4 is a structural diagram for explaining the global interconnection of FIG. 3 in detail.
FIG. 5 is a block diagram for explaining the flow of control signals of the neural processing device of FIG. 1.
FIG. 6 is a block diagram for explaining the neural processor of FIG. 3 in detail.
FIG. 7 is a diagram illustrating the hierarchical structure of a neural processing device according to some embodiments of the present invention.
FIG. 8 is a block diagram for explaining the neural core of FIG. 6 in detail.
FIG. 9 is a block diagram for explaining the LSU of FIG. 8 in detail.
FIG. 10 is a block diagram for explaining the processing unit of FIG. 8 in detail.
FIG. 11 is a block diagram for explaining the L0 memory of FIG. 8 in detail.
FIG. 12 is a block diagram for explaining the local memory bank of FIG. 11 in detail.
FIG. 13 is a block diagram for explaining the flow of data and control signals of the neural processing device of FIG. 1.
FIG. 14 is a block diagram for explaining the relationship between the command processor and task manager of FIG. 13.
FIG. 15 is a block diagram to explain in detail the structure of the task manager of FIG. 8.
FIG. 16 is a block diagram for explaining the table passage of FIG. 15 in detail.
FIG. 17 is a block diagram for explaining the task passage of FIG. 15 in detail.
FIG. 18 is a diagram for explaining the first dependency matrix stored inside the dependency checker of FIG. 17.
FIG. 19 is a block diagram for explaining the Dawn passage of FIG. 15 in detail.
FIG. 20 is a block diagram for explaining the report management module of FIG. 19 in detail.
FIG. 21 is a diagram illustrating data exchanged between the core global and the task processing unit of FIG. 15.
Figure 22 is a diagram for explaining the types of task descriptors stored in the first queue, second queue, and check-in buffer.
FIG. 23 is a diagram illustrating an exemplary dependency matrix for checking the dependency of the task descriptor according to FIG. 22.
Figures 24 to 27 are exemplary diagrams for explaining the operation of the second dependency matrix.
Figure 28 is an example diagram of a dependency matrix for checking dependencies between a plurality of task descriptors.
Figure 29 is a conceptual diagram to explain the latency reduction effect for each group.
Figure 30 exemplarily shows a plurality of task descriptors and one task descriptor whose dependency is checked in different dependency matrices.
Figure 31 is a diagram for explaining a dummy check descriptor.
Figure 32 is a diagram to explain that a dummy check descriptor is burned in the dependency checker.
Figure 33 exemplarily shows one task descriptor and a plurality of task descriptors whose dependencies are checked in different dependency matrices.
Figure 34 is a diagram for explaining a dummy set descriptor.
FIG. 35 is a diagram for explaining the data exchange process performed in the void core of FIG. 15.
FIG. 36 is a diagram illustrating a first queue, a second queue, and a check-in buffer of a neural processing device according to some embodiments of the present invention.
FIG. 37 is a diagram illustrating a first queue, a second queue, and a check-in buffer of a neural processing device according to some embodiments of the present invention.
Figure 38 is a block diagram for explaining in detail the structure of a neural processing device according to some embodiments of the present invention.
FIG. 39 is a diagram illustrating the hierarchical structure of a command processor and a task manager of a neural processing device according to some embodiments of the present invention.
FIG. 40 is a diagram illustrating the hierarchical structure of command processors and task managers of a neural processing device according to some embodiments of the present invention.
Figure 41 is a block diagram for explaining memory reorganization of a neural processing system according to some embodiments of the present invention.
Figure 42 is a block diagram showing an example of memory reorganization of a neural processing system according to some embodiments of the present invention.
Figure 43 is an enlarged block diagram of part A of Figure 41.
FIG. 44 is a diagram for explaining the first memory bank of FIG. 43 in detail.
Figure 45 is a block diagram to explain the software hierarchy of a neural processing device according to some embodiments of the present invention.
Figure 46 is a conceptual diagram to explain a deep learning operation performed by a neural processing device according to some embodiments of the present invention.
Figure 47 is a conceptual diagram for explaining learning and inference operations of a neural network of a neural processing device according to some embodiments of the present invention.
Figure 48 is a flowchart illustrating a method of setting task dependency of a neural processing device according to some embodiments of the present invention.
FIG. 49 is a flowchart illustrating in detail the step of receiving the dependency set code of FIG. 48.
FIG. 50 is a flowchart illustrating in detail the step of transmitting the task descriptor of FIG. 48.

Terms or words used in this specification and patent claims should not be construed as limited to their general or dictionary meaning. According to the principle that the inventor can define the term or word concept in order to explain his or her invention in the best way, it should be interpreted with a meaning and concept consistent with the technical idea of the present invention. In addition, the embodiments described in this specification and the configurations shown in the drawings are only one embodiment of the present invention and do not completely represent the technical idea of the present invention, so they cannot be replaced at the time of filing the present application. It should be understood that there may be various equivalents, variations, and applicable examples.

Terms such as first, second, A, and B used in the present specification and claims may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component without departing from the scope of the present invention, and similarly, the second component may also be named a first component. The term 'and/or' includes any of a plurality of related stated items or a combination of a plurality of related stated items.

The terms used in the specification and claims are merely used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "include" or "have" should be understood as not precluding the existence or addition possibility of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. .

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains.

Terms defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Additionally, each configuration, process, process, or method included in each embodiment of the present invention may be shared within the scope of not being technically contradictory to each other.

Hereinafter, a neural processing device according to some embodiments of the present invention will be described with reference to FIGS. 1 to 39.

1 is a block diagram for explaining a neural processing system according to some embodiments of the present invention.

Referring to FIG. 1, a neural processing system (NPS) according to some embodiments of the present invention may include a first neural processing device 1, a second neural processing device 2, and an external interface 3. .

The first neural processing device 1 may be a device that performs calculations using an artificial neural network. For example, the first neural processing device 1 may be a device specialized for performing deep learning calculation tasks. However, this embodiment is not limited to this.

The second neural processing device 2 may be a device that has the same or similar configuration as the first neural processing device 1. The first neural processing device 1 and the second neural processing device 2 may be connected to each other through an external interface 3 and share data and control signals.

Although two neural processing devices are shown in FIG. 1, the neural processing system (NPS) according to some embodiments of the present invention is not limited thereto. That is, in the neural processing system (NPS) according to some embodiments of the present invention, three or more neural processing devices may be connected to each other through the external interface 3. Also, conversely, a neural processing system (NPS) according to some embodiments of the present invention may include only one neural processing device.

At this time, the first neural processing device 1 and the second neural processing device 2 may each be a processing device other than a neural processing device. That is, the first neural processing device 1 and the second neural processing device 2 are respectively a graphics processing unit (GPU, graphics processing unit), a central processing unit (CPU), and other types of processing devices. It may be. Hereinafter, for convenience, the first neural processing device 1 and the second neural processing device 2 will be described as neural processing devices.

FIG. 2 is a block diagram for explaining the neural processing device of FIG. 1 in detail.

Referring to FIG. 2, the first neural processing device 1 includes a neural core SoC 10, a CPU 20, an off-chip memory 30, a first non-volatile memory interface 40, and a first volatile memory interface ( 50), a second non-volatile memory interface 60, a second volatile memory interface 70, and a control interface (CIF) 80.

The Neural Core SoC 10 may be a System on Chip device. The neural core SoC (10) is an artificial intelligence calculation unit and may be an accelerator. The neural core SoC 10 may be, for example, one of a graphics processing unit (GPU), a field programmable gate array (FPGA), and an application-specific integrated circuit (ASIC). However, this embodiment is not limited to this.

The neural core SoC (10) can exchange data with other external computational units through the external interface (3). Additionally, the neural core SoC 10 may be connected to the non-volatile memory 31 and the volatile memory 32 through the first non-volatile memory interface 40 and the first volatile memory interface 50, respectively.

The CPU 20 may be a control device that controls the system of the first neural processing device 1 and executes program operations. The CPU 20 is a general-purpose calculation unit and may be inefficient in performing parallel simple calculations commonly used in deep learning. Accordingly, the neural core SoC 10 can achieve high efficiency by performing calculations on deep learning inference and learning tasks.

The CPU 20 can exchange data with other external computational units through the external interface 3. Additionally, the CPU 20 may be connected to the non-volatile memory 31 and the volatile memory 32 through the second non-volatile memory interface 60 and the second volatile memory interface 70, respectively.

The CPU 20 may also transfer a task to the neural core SoC 10 through a command. At this time, the CPU 20 may be a type of host that gives instructions to the neural core SoC 10. In other words, the neural core SoC 10 can efficiently perform parallel computing tasks such as deep learning tasks according to instructions from the CPU 20.

The off-chip memory 30 may be memory placed outside the chip of the neural core SoC 10. Off-chip memory 30 may include non-volatile memory 31 and volatile memory 32.

The non-volatile memory 31 may be a memory that continues to retain stored information even when power is not supplied. The non-volatile memory 31 includes, for example, Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Alterable ROM (EAROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Memory (EEPROM). Read-Only Memory (e.g., NAND Flash memory, NOR Flash memory), UVEPROM (Ultra-Violet Erasable Programmable Read-Only Memory), FeRAM (Ferroelectric Random Access Memory), Magnetoresistive Random Access Memory (MRAM), Phase-change Random Access Memory (PRAM), silicon-oxide-nitride-oxide-silicon (SONOS), Resistive Random Access Memory (RRAM), Nanotube Random Access Memory (NRAM), magnetic computer memory It may include at least one of a device (eg, hard disk, diskette drive, magnetic tape), optical disk drive, and 3D XPoint memory. However, this embodiment is not limited to this.

Unlike the non-volatile memory 31, the volatile memory 32 may be a memory that continuously requires power to maintain stored information. The volatile memory 32 may include, for example, at least one of Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Synchronous Dynamic Random Access Memory (SDRAM), and Double Data Rate SDRAM (DDR SDRAM). there is. However, this embodiment is not limited to this.

The first non-volatile memory interface 40 and the second non-volatile memory interface 60 are, for example, Parallel Advanced Technology Attachment (PATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), and SATA ( It may include at least one of Serial Advanced Technology Attachment) and PCIe (PCI Express). However, this embodiment is not limited to this.

The first volatile memory interface 50 and the second volatile memory interface 70 may be configured to perform, for example, single data rate (SDR), double data rate (DDR), quad data rate (QDR), and eXtreme data rate (XDR), respectively. , Octal Data Rate). However, this embodiment is not limited to this.

The control interface 80 may be an interface for transmitting control signals between the CPU 20 and the neural core SoC 10. The control interface 80 may transmit a command from the CPU 20 and a response from the neural core SoC 10 thereto. The control interface 80 may be, for example, PCIe (PCI Express), but is not limited thereto.

FIG. 3 is a block diagram for explaining the neural core SoC of FIG. 2 in detail.

2 and 3, the neural core SoC 10 includes at least one neural processor 1000, a shared memory 2000, a direct memory access (DMA) 3000, a non-volatile memory controller 4000, and a volatile memory controller 4000. It may include a memory controller 5000, a command processor 7000, and a global interconnection 6000.

The neural processor 1000 may be a computational unit that directly performs computational tasks. When there are multiple neural processors 1000, computational tasks may be assigned to each neural processor 1000. Each neural processor 1000 may be connected to each other through a global interconnection 6000.

The shared memory 2000 may be memory shared by several neural processors 1000. The shared memory 2000 can store data of each neural processor 1000. Additionally, the shared memory 2000 can receive data from the off-chip memory 30, temporarily store it, and transmit it to each neural processor 1000. Conversely, the shared memory 2000 may receive data from the neural processor 1000, temporarily store it, and transfer it to the off-chip memory 30 of FIG. 2.

Shared memory 2000 may require relatively fast memory. Accordingly, the shared memory 2000 may include, for example, SRAM. However, this embodiment is not limited to this. That is, the shared memory 2000 may include DRAM.

The shared memory 2000 may be a memory corresponding to the SoC level, that is, level 2 (L2). Accordingly, the shared memory 2000 may be defined as L2 shared memory.

The DMA 3000 can directly control the movement of data without the need for the CPU 20 or the neural processor 1000 to control input and output of data. Accordingly, the DMA 3000 can control data movement between memories to minimize the number of interrupts of the CPU 20 or the neural processor 1000.

The DMA (3000) can control data movement between the shared memory (2000) and the off-chip memory (30). The non-volatile memory controller 4000 and volatile memory controller 5000 can move data through the authority of the DMA (3000).

The non-volatile memory controller 4000 can control read or write operations on the non-volatile memory 31. The non-volatile memory controller 4000 can control the non-volatile memory 31 through the first non-volatile memory interface 40.

The volatile memory controller 5000 can control read or write operations on the volatile memory 32. Additionally, the volatile memory controller 5000 may perform a refresh operation on the volatile memory 32. The volatile memory controller 5000 can control the volatile memory 32 through the first volatile memory interface 50.

The command processor 7000 may be connected to the control interface 80. The command processor 7000 may receive a control signal from the CPU 20 through the control interface 80. The command processor 7000 can create a task through a control signal received from the CPU 20 and transmit it to each neural processor 1000. Additionally, the command processor 7000 may receive a completion report for the task from each neural processor 1000.

The global interconnection 6000 connects at least one neural processor 1000, shared memory 2000, DMA 3000, non-volatile memory controller 4000, command processor 7000, and volatile memory controller 5000 to each other. You can. Additionally, the external interface 3 may also be connected to the global interconnection 6000. The global interconnection 6000 includes at least one neural processor 1000, a shared memory 2000, a DMA 3000, a non-volatile memory controller 4000, a volatile memory controller 5000, a command processor 7000, and an external interface. (3) It may be a path through which data moves.

The global interconnection 6000 can transmit not only data but also control signals and signals for synchronization. In the neural processing device according to some embodiments of the present invention, each neural processor 1000 can directly transmit and receive a synchronization signal. Accordingly, latency due to transmission of the synchronization signal generated by the command processor 7000 can be minimized.

That is, when there are multiple neural processors 1000, there may be a dependency of individual tasks in which the task of one neural processor 1000 must be completed before the next neural processor 1000 can start a new task. The end and start of these individual tasks can be confirmed through synchronization signals. In the existing technology, the command processor 7000 or the host, that is, the CPU 20, is in charge of both receiving these synchronization signals and instructing the start of new tasks. did.

However, as the number of neural processors 1000 increases and the dependency of the task is designed more complexly, the number of these synchronization signals increases exponentially, and the latency according to each synchronization signal can significantly reduce the efficiency of the task. .

Therefore, in the neural processing device according to some embodiments of the present invention, instead of the command processor 7000, each neural processor 1000 may directly transmit a part of the synchronization signal to another neural processor 1000 according to the dependency of the task. You can. In this case, compared to the method managed by the command processor 7000, multiple neural processors 1000 can perform synchronization tasks in parallel, thereby minimizing latency due to synchronization.

In addition, the command processor 7000 must perform task scheduling of the neural processors 1000 according to task dependency, and the overhead of such scheduling may increase significantly as the number of neural processors 1000 increases. Accordingly, in the neural processing device according to some embodiments of the present invention, the scheduling task is partially performed by the individual neural processor 1000, thereby reducing the scheduling burden, thereby improving the performance of the device.

FIG. 4 is a structural diagram for explaining the global interconnection of FIG. 3 in detail.

Referring to FIG. 4, the global interconnection 6000 may include a data channel 6100, a control channel 6200, and an L2 sync channel 6300.

The data channel 6100 may be a dedicated channel for transmitting data. Through the data channel 6100, at least one neural processor 1000, shared memory 2000, DMA 3000, non-volatile memory controller 4000, volatile memory controller 5000, and external interface 3 exchange data with each other. can be exchanged.

The control channel 6200 may be a dedicated channel that transmits control signals. At least one neural processor 1000, shared memory 2000, DMA 3000, non-volatile memory controller 4000, volatile memory controller 5000, command processor 7000, and external interface through the control channel 6200. (3) can exchange control signals with each other. In particular, the command processor 7000 can transmit various control signals to each neural processor 1000.

The L2 sync channel 6300 may be a dedicated channel that transmits a synchronization signal. Through the L2 sync channel 6300, at least one neural processor 1000, shared memory 2000, DMA 3000, non-volatile memory controller 4000, volatile memory controller 5000, command processor 7000, and external The interface 3 can exchange synchronization signals with each other.

The L2 sync channel 6300 is set as a dedicated channel within the global interconnection 6000 and can quickly transmit synchronization signals without overlapping with other channels. Accordingly, the neural processing device according to some embodiments of the present invention does not require new wiring work and can smoothly perform synchronization work using the existing global interconnection 6000.

FIG. 5 is a block diagram for explaining the flow of control signals of the neural processing device of FIG. 1.

Referring to FIG. 5, the CPU 20 may transmit a control signal to the command processor 7000 through the control interface 80. At this time, the control signal may be a signal instructing to perform each operation, such as a computation task or a data load/store task.

The command processor 7000 may receive a control signal and transmit the control signal to at least one neural processor 1000 through the control channel 6200. Each control signal may be stored in the neural processor 1000 as each task.

FIG. 6 is a block diagram for explaining the neural processor of FIG. 3 in detail.

3 to 6, the neural processor 1000 includes at least one neural core 100, a local interconnection 200, an L1 sync path 300, an L1 shared memory 400, and a core global 500. , may include a task manager 600 and an L1 LSU 700.

At least one neural core 100 may share and perform the tasks of the neural processor 1000. For example, there may be eight neural cores 100. However, this embodiment is not limited to this. 3 and 5 show that several neural cores 100 are included in the neural processor 1000, but the present embodiment is not limited thereto. In other words, the neural processor 1000 can be configured with only one neural core 100.

The neural core 100 may receive task information from the core global 500 and perform a task according to the task information. At this time, the task may be defined by a control signal, and the task may be any one of memory operations. The memory operation may be, for example, any one of micro DMA (μDMA), low priority μDMA (LP micro DMA), store μDMA (STμDMA), and pre-processing operations.

The L1 shared memory 400 may be a memory shared by each neural core 100 within the neural processor 1000. The L1 shared memory 400 can store data of each neural core 100. Additionally, the L1 shared memory 400 can receive data from the shared memory 2000 of FIG. 4, temporarily store it, and transmit it to each neural core 100. Conversely, the L1 shared memory 400 may receive data from the neural core 100, temporarily store it, and transfer it to the shared memory 2000 of FIG. 3.

The L1 shared memory 400 may be a memory corresponding to the neural processor level, that is, level 1 (L1). L2 shared memory, that is, shared memory 2000, may be shared by the neural processor 1000, and L1 shared memory 400 may be shared by the neural core 100.

The L1 LSU 700 may receive at least one of data, control signals, and synchronization signals from the outside through the global interconnection 6000. The L1 LSU 700 may transmit at least one of the received data, control signal, and synchronization signal to the L1 shared memory 400. Similarly, the L1 LSU 700 can transmit at least one of data, control signals, and synchronization signals to the outside through the global interconnection 6000. Additionally, the L1 LSU 700 may transmit and receive at least one of data, control signals, and synchronization signals to each of the neural cores 100.

The neural core 100 may receive task information from the core global 500 and perform a task according to the task information. At this time, the task may be a computation task (operation task) or a task related to memory operation. Tasks can be defined by control signals. Task information is information about a task and may be information about the type of task, the form of the task, additional information about the task, etc.

The neural core 100 may transmit a completion signal indicating completion of the task to the core global 500.

The task manager 600 may receive a task from a control interconnection (CI). At this time, control interconnection (CI) may be a general term for a transmission interface that transmits tasks from the command processor 7000. That is, the control interconnection (CI) may include a control channel 6200 and a local interconnection 200.

The task manager 600 may receive a task, generate task information, and transmit it to the core global 500. Additionally, the task manager 600 may receive a completion signal through the core global 500, generate a completion report accordingly, and transmit it to the command processor 7000 through a control interconnection (CI).

The core global 500 may be a wire structure connected in hardware to the neural core 100. Although not shown, the core global 500 may be a structure that connects the neural core 100, L1 shared memory 400, L1 LSU 700, and task manager 600. Accordingly, the local interconnection 200 and the L1 sync path 300 may also be included in the core global 500. However, this embodiment is not limited to this.

The core global 500 may receive task information from the task manager 600, transmit it to the neural core 100, and receive a completion signal for the task from the neural core 100. Subsequently, Core Global 500 may transmit a completion signal to the task manager 600.

The local interconnection 200 may connect at least one neural core 100, L1 shared memory 400, core global 500, task manager 600, and L1 LSU 700 to each other. The local interconnection 200 may be a path through which data moves between at least one neural core 100, L1 shared memory 400, core global 500, task manager 600, and L1 LSU 700. . The local interconnection 200 is connected to the global interconnection 6000 of FIG. 3 and can transmit data.

The L1 sync path 300 may connect at least one neural core 100, L1 shared memory 400, core global 500, task manager 600, and L1 LSU 700 to each other. The L1 sync path 300 may be a path along which synchronization signals of at least one neural core 100, L1 shared memory 400, core global 500, task manager 600, and L1 LSU 700 travel. .

The L1 sync path 300 may be formed physically separately from the local interconnection 200. In the case of the local interconnection 200, unlike the global interconnection 6000, sufficient channels may not be formed internally. In this case, the L1 sync path 300 is formed separately so that transmission of the synchronization signal can be performed quickly and without delay. The L1 sync path 300 can be used for synchronization performed at a level one level lower than the L2 sync channel 6300 of the global interconnection 6000.

FIG. 7 is a diagram illustrating the hierarchical structure of a neural processing device according to some embodiments of the present invention.

Referring to FIG. 7, the neural core SoC 10 may include at least one neural processor 1000. Each neural processor 1000 can transmit data to each other through the global interconnection 6000.

Each neural processor 1000 may include at least one neural core 100. The neural core 100 may be a processing unit optimized for deep learning calculation tasks. The neural core 100 may be a processing unit corresponding to one operation of a deep learning calculation task. In other words, deep learning computational tasks can be expressed as a sequential or parallel combination of multiple operations. The neural core 100 is a processing unit that can each process one operation, and may be the minimum computational unit that can be considered for scheduling from a compiler's perspective.

The neural processing device according to this embodiment can achieve fast and efficient scheduling and performance of calculation tasks by configuring the scale of the minimum calculation unit and hardware processing unit considered from a compiler scheduling perspective to be the same.

In other words, if the processing unit that can be divided into hardware is too large compared to the computational task, inefficiency in the computational task may occur when driving the processing unit. Conversely, it is not appropriate to always schedule a processing unit smaller than the operation, which is the compiler's minimum scheduling unit, because scheduling inefficiencies may occur and hardware design costs may increase.

Therefore, in this embodiment, the scale of the compiler's scheduling unit and the hardware processing unit can be similarly adjusted to simultaneously satisfy fast computational task scheduling and efficient computational task performance without wasting hardware resources.

FIG. 8 is a block diagram for explaining the neural core of FIG. 6 in detail.

Referring to FIG. 8, the neural core 100 includes a Load/Store Unit (LSU) 110, an L0 memory 120, a weight buffer 130, an activation LSU 140, an activation buffer 150, and a processing unit ( 160) may be included.

The LSU 110 may receive at least one of data, control signals, and synchronization signals from the outside through the local interconnection 200 and the L1 sync path 300. The LSU 110 may transmit at least one of the received data, control signal, and synchronization signal to the L0 memory 120. Similarly, the LSU 110 may transmit at least one of data, a control signal, and a synchronization signal to the outside through the local interconnection 200 and the L1 sync path 300.

Specifically, the micro DMA task may be a task in which the neural core 100 loads a program or data from the shared memory 2000 or the off-chip memory 30 to the L0 memory 120. Unlike a typical micro DMA operation, the LP micro DMA operation may be a load operation for a program or data to be used later rather than the current program or data. Because these tasks have a low priority, they can be identified differently from micro DMA tasks. The ST Micro DMA operation may be a store operation that stores data from the L0 memory 120 of the neural core 100 to the shared memory 2000 or the off-chip memory 30. The preprocessing task may include preloading data, such as a large amount of lookup tables, in the CPU 20.

FIG. 9 is a block diagram for explaining the LSU of FIG. 8 in detail.

Referring to FIG. 9, the LSU 110 includes a local memory load unit 111a, a local memory store unit 111b, a neural core load unit 112a, a neural core store unit 112b, a load buffer (LB), and a store. It may include a buffer (SB), a load engine 113a, a store engine 113b, and a conversion index buffer 114.

The local memory load unit 111a may fetch a load instruction for the L0 memory 120 and issue the load instruction. When the local memory load unit 111a provides an issue load instruction to the load buffer LB, a memory access request can be sequentially transmitted to the load engine 113a according to the order in which the load buffer LB is input.

Additionally, the local memory store unit 111b may fetch a store instruction for the L0 memory 120 and issue the store instruction. When the local memory store unit 111b provides the issue store instruction to the store buffer SB, a memory access request can be sequentially transmitted to the store engine 113b according to the order in which the store buffer SB was input.

The neural core load unit 112a may fetch a load instruction for the neural core 100 and issue the load instruction. When the neural core load unit 112a provides an issue load instruction to the load buffer LB, a memory access request can be sequentially transmitted to the load engine 113a according to the order in which the load buffer LB is input.

Additionally, the neural core store unit 112b may fetch a store instruction for the neural core 100 and issue the store instruction. When the neural core store unit 112b provides the issue store instruction to the store buffer SB, a memory access request can be sequentially transmitted to the store engine 113b according to the order in which the store buffer SB was input.

The load engine 113a may receive a memory access request and load data through the local interconnection 200. At this time, the load engine 113a can quickly find data using the conversion table of recently used logical addresses and physical addresses in the conversion index buffer 114. If the logical address of the load engine 113a is not in the translation index buffer 114, address translation information can be found in another memory.

The store engine 113b may receive a memory access request and load data through the local interconnection 200. At this time, the store engine 113b can quickly find data using the conversion table of recently used logical addresses and physical addresses in the conversion index buffer 114. If the logical address of the store engine 113b is not in the translation index buffer 114, address translation information can be found in another memory.

The load engine 113a and the store engine 113b may send a synchronization signal to the L1 sync path 300. At this time, the synchronization signal may mean that the task has ended.

Referring again to FIG. 8, the L0 memory 120 is a memory located inside the neural core 100, and allows the neural core 100 to receive all input data required for work from the outside and temporarily store them. Additionally, the L0 memory 120 can temporarily store output data calculated by the neural core 100 in order to transmit it to the outside.

The L0 memory 120 may transmit input activation (Act_In) to the activation buffer 150 and receive output activation (Act_Out) by the activation LSU 140. In addition to the activation LSU 140, the L0 memory 120 can directly transmit and receive data with the processing unit 160. That is, the L0 memory 120 can exchange data with each of the PE array 163 and the vector unit 164. The L0 memory 120 may be a memory corresponding to the neural core level. At this time, the L0 memory 120 may be a private memory of the neural core.

The L0 memory 120 can transmit data such as activation or weight through a data path. The L0 memory 120 can send and receive synchronization signals through the L0 Sync Path, which is a separate dedicated path. For example, the L0 memory 120 may exchange a synchronization signal with the LSU 110, the weight buffer 130, the activation LSU 140, and the processing unit 160 through an L0 Sync Path. .

The weight buffer 130 may receive weight from the L0 memory 120. The weight buffer 130 may transmit weight to the processing unit 160. The weight buffer 130 may temporarily store the weight before transmitting the weight.

Input activation (Act_In) and output activation (Act_Out) may refer to the input and output values of the layer of the neural network network. At this time, when the neural network has multiple layers, the output value of the previous layer becomes the input value of the next layer, so the output activation (Act_Out) of the previous layer can be used as the input activation (Act_In) of the next layer.

Weight may refer to a parameter multiplied by the input activation (Act_In) input from each layer. Weight is adjusted and confirmed in the deep learning learning stage, and can be used to derive output activation (Act_Out) through a fixed value in the inference stage.

The activation LSU 140 may transfer the input activation (Act_In) from the L0 memory 120 to the activation buffer 150 and the output activation (Act_Out) from the activation buffer 150 to the on-chip buffer. That is, the activation LSU 140 can perform both activation load and store operations.

The activation buffer 150 may provide input activation (Act_In) to the processing unit 160 and receive output activation (Act_Out) from the processing unit 160. The activation buffer 150 can temporarily store input activation (Act_In) and output activation (Act_Out).

The activation buffer 150 can quickly provide activation to the processing unit 160 with a large computational load, especially the PE array 163, and quickly receive activation to increase the computation speed of the neural core 100.

The processing unit 160 may be a module that performs calculations. The processing unit 160 can perform not only one-dimensional operations but also two-dimensional matrix operations, that is, convolution operations. The processing unit 160 may receive the input activation (Act_In), multiply it by the weight, and add it to generate the output activation (Act_Out).

FIG. 10 is a block diagram for explaining the processing unit of FIG. 8 in detail.

Referring to FIGS. 8 and 10 , the processing unit 160 may include a PE array 163, a vector unit 164, a column register 161, and a row register 162.

The PE array 163 can perform multiplication by receiving input activation (Act_In) and weight (Weight). At this time, input activation (Act_In) and weight (Weight) can each be calculated through convolution in matrix form. Through this, the PE array 163 can generate output activation (Act_Out). However, this embodiment is not limited to this. The PE array 163 can generate any number of types of output other than output activation (Act_Out).

The PE array 163 may include at least one processing element 163_1. The processing elements 163_1 are aligned with each other and can perform multiplication of one input activation (Act_In) and one weight (Weight), respectively.

The PE array 163 can generate a subtotal that sums the values for each multiplication. These partial sums can be used as output activation (Act_Out). Since the PE array 163 performs two-dimensional matrix multiplication, it may also be referred to as a two-dimensional matrix compute unit.

The vector unit 164 can perform one-dimensional operations. The vector unit 164 can perform deep learning calculations together with the PE array 163. Through this, the processing unit 160 can be specialized for necessary operations. In other words, the neural core 100 has calculation modules that perform large amounts of two-dimensional matrix multiplication and one-dimensional calculations, so it can efficiently perform deep learning tasks.

The column register 161 may receive the first input (I1). The column register 161 may receive the first input (I1), divide it, and provide it to each column of the PE array 163.

Low register 162 may receive a second input (I2). The row register 162 may receive the second input I2, divide it, and provide it to each row of the PE array 163.

The first input (I1) may be input activation (Act_In) or weight (Weight). The second input (I2) may be a value other than the first input (I1) among input activation (Act_In) or weight (Weight). Alternatively, the first input (I1) and the second input (I2) may be values other than input activation (Act_In) and weight (Weight).

FIG. 11 is a block diagram for explaining the L0 memory of FIG. 8 in detail.

Referring to FIG. 11, the L0 memory 120 may include a scheduler 121 and at least one local memory bank 122.

When data is stored in the L0 memory 120, the scheduler 121 may receive the data from the load engine 113a. At this time, data may be allocated to the local memory bank 122 in a round robin manner. Accordingly, data may be stored in any one of at least one local memory bank 122.

Conversely, when data is loaded from the L0 memory 120, the scheduler 121 may receive data from the local memory bank 122 and transfer it to the store engine 113b. The store engine 113b can store data externally through the local interconnection 200.

FIG. 12 is a block diagram for explaining the local memory bank of FIG. 11 in detail.

Referring to FIG. 12, the local memory bank 122 may include a local memory bank controller 122_1 and a local memory bank cell array 122_2.

The local memory bank controller 122_1 can manage read and write operations through the address of data stored in the local memory bank 122. That is, the local memory bank controller 122_1 can manage the overall input and output of data.

The local memory bank cell array 122_2 may have a structure in which cells in which data is directly stored are aligned in rows and columns. The local memory bank cell array 122_2 may be controlled by the local memory bank controller 122_1.

FIG. 13 is a block diagram for explaining the flow of data and control signals of the neural processing device of FIG. 1, and FIG. 14 is a block diagram for explaining the relationship between the command processor and task manager of FIG. 13.

Referring to FIGS. 13 and 14 , each neural processor 1000 may include a task manager 600 and an L1 LSU 700 therein, respectively. Task managers 600 can exchange control signals and responses with the command processor 7000 through a control interconnection (CI).

In contrast, the L1 LSU 700 can exchange data through data interconnection and memory (DIM). Data interconnection and memory (DIM) may include an interconnection for transmitting data and a memory in which the data is shared. Specifically, data interconnection and memory (DIM) may include a local interconnection 200 and a data channel 6100. Additionally, data interconnection and memory (DIM) may include L1 shared memory 400, shared memory 2000, and volatile memory 32. However, this embodiment is not limited to this.

The task manager 600 may be controlled by the command processor 7000. That is, the command processor 7000 can transmit a task to the task manager 600 through a control signal, and the task manager 600 can transmit a task completion report to the command processor 7000. The task manager 600 may be included in the neural core 100 and at least one task manager 600 may be included in the neural processor 1000. Additionally, if there are multiple neural processors 1000, the number of task managers 600 may increase. All of these plurality of task managers 600 can be controlled by the command processor 7000.

FIG. 15 is a block diagram to explain in detail the structure of the task manager of FIG. 8.

Referring to FIGS. 8, 9, and 15, the task manager 600 may include a table passage 610, a task passage 620, a dark passage 630, and a void core 640.

The table passage 610 may receive a table update request for updating a matching table of physical addresses and logical addresses from the control channel 6200 and transmit it to the core global 500. At this time, the table update request may be transmitted from the command processor 7000 through the control channel 6200.

The task passage 620 may receive a task from the control channel 6200, generate task information accordingly, and transmit it to the core global 500. At this time, the task may be transmitted from the command processor 7000 through the control channel 6200. Core Global 500 may transmit task information to Neural Core 100. The neural core 100 can perform a task according to the delivered task information and transmit a completion signal to the core global 500.

Core Global 500 may transmit a completion signal to Dawn Passage 630. The Dawn Passage 630 may receive a completion signal and generate a task completion report (DNrp). The Dawn Passage 630 may transmit a completion report (DNrp) to the command processor 7000 through the control channel 6200.

Additionally, the table update request (TURQ) of the table passage 610 may be transmitted to the LSU 110 through the core global 500. At this time, the table of the conversion index buffer 114 within the LSU 110 may be updated.

The void core 640 may process the task without transmitting it to the neural core 100 and generate a completion signal for the task. That is, the void core 640 can virtually process the task and generate a completion signal. In some embodiments, void core 640 may virtually process a dummy set task for setting dependencies between dependency matrices. The dummy set task is a task for dependency setting, does not include information for computation or memory operation, and may be configured to include information for dependency setting. The void core 640 may generate a completion signal in response to a dummy set task for the dependency set, and enable dependency setting to be performed in response to the completion signal.

FIG. 16 is a block diagram for explaining the table passage of FIG. 15 in detail.

Referring to FIG. 16, the table passage 610 may include a table buffer 611 and first to mth update request queues 611a1 to 611am.

The table buffer 611 may store a table update request (TURQ) in which a physical address and a logical address are matched, transmitted from the command processor 7000. When the core global 500 fetches these table update requests (TURQ), each table update request (TURQ) may be stored in the first to mth update request queues 611a1 to 611am.

Each of the first to mth update request queues 611a1 to 611am may store different types of table update requests (TURQ). For example, different types of table update requests (TURQ) may include at least one of a neural core TLB update request, a micro DMA TLB update request, an LP micro DMA TLB update request, and an ST micro DMA TLB update request. However, this embodiment is not limited to this. Alternatively, the same type of table update request (TURQ) may be stored in each of the first to mth update request queues 611a1 to 611am.

Additionally, the first to mth update request queues 611a1 to 611am may each be general queues, that is, queues that accommodate various types of requests. Accordingly, each of the first to mth update request queues 611a1 to 611am can accept requests regardless of type.

Each of the first to mth update request queues 611a1 to 611am may transmit a table update request (TURQ) to the core global 500.

FIG. 17 is a block diagram for explaining the task passage of FIG. 15 in detail.

Referring to FIG. 17, the task passage 620 may include a task buffer 621, a first queue (Q1), a dependency checker (DPc), a second queue (Q2), and a runtime handle (RH). .

The task buffer 621 may store a task according to a control signal transmitted from the command processor 7000. The task buffer 621 may store a task in the first queue Q1 in the form of a task descriptor by the task fetching operation of the core global 500. That is, the task passage 620 may create a task descriptor corresponding to the task and store it in the first queue Q1.

The task buffer 621 may transmit the task descriptor to the first queue (Q1) and generate a transfer report (TRrp). The task buffer 621 may transmit the transfer dawn report (TRrp) to the dawn passage 630. The transfer report (TRrp) may be a report on the task transmitted to the first queue (Q1).

The first queue Q1 may store task descriptors divided according to the type of task descriptor. In Figure 17, n first queues Q1 are shown. At this time, n may be a natural number. That is, there may be at least one first queue Q1.

At this time, the first queue (Q1) may include the 1_1th to 1_nth queues (Q1_1 to Q1_n). The 1_1 queue (Q1_1) may store the first task descriptor (Tsk_d1), and the 1_2 queue (Q1_2) may store the second task descriptor (Tsk_d2). The 1_nth queue (Q1_n) can store the nth task descriptor (Tsk_dn).

The first to nth task descriptors (Tsk_d1 to Tsk_dn) may be of different types or may be of the same type. Alternatively, some of the first to nth task descriptors (Tsk_d1 to Tsk_dn) may be of the same type, and some may be of different types.

The dependency checker (DPc) may include a dependency matrix (DPmx). The dependency for the task descriptor can be checked by comparing the dependency set code defined in the dependency matrix (DPmx) with the check code extracted from the task descriptor. The dependency set code set in the dependency matrix (DPmx) can be updated through a dependency update request (DFURQ). A dependency update request (DFURQ) can notify changes in dependencies as completed tasks occur according to defined dependencies between specific tasks.

In an embodiment, each task descriptor may include dependency information about which task it has a dependency on. That is, a specific task descriptor may include dependency information, which is information about other task descriptors dependent on it. As a specific task descriptor is completed, a dependency update request (DFURQ) is generated considering the dependency information included in the task descriptor. Accordingly, the dependency update request (DFURQ) may include a request to update the dependency matrix of the task descriptor.

In some embodiments, the dependency matrix (DPmx) included in the dependency checker (DPc) may be composed of multiple dependency matrices (DPmx). Referring to FIG. 17, the dependency matrix DPmx may include a first dependency matrix DPmx_1 to an mth dependency matrix DPmx_m. In FIG. 17 , the 1_1st to 1_nth queues (Q1_1 to Q1_n) and the 1st to mth dependency matrices (DPmx_1 to DPmx_m) are shown as corresponding, but the embodiment of the present invention is not limited thereto. In some embodiments, m, which is the number of dependency matrices (DPmx), may be different from n, which is the number of first queues (Q1), and the first to mth dependency matrices (DPmx_1) to ( DPmx_m) and the first queue (Q1) do not correspond to each other. In an embodiment, the dependency matrix (DPmx) may be composed of a pre-calculated number (m) for efficient operation of the neural processing system. At this time, each dependency matrix (DPmx) may be expressed as belonging to a separate group.

Additionally, each of the first to mth dependency matrices (DPmx_1 to DPmx_m) may operate independently. In an embodiment, each of the first to mth dependency matrices (DPmx_1 to DPmx_m) may operate independently, and updating of the dependency matrices may also be performed individually. That is, the dependency update request (DFURQ) may include the first to mth dependency field update requests (DFURQ_1 to DFURQ_m) corresponding to the first to mth dependency matrices (DPmx_1 to DPmx_m), respectively. . Hereinafter, the structure of one dependency matrix and the process by which multiple dependency matrices operate independently will be described in more detail.

FIG. 18 is a diagram for explaining the first dependency matrix stored inside the dependency checker of FIG. 17.

Referring to FIG. 18, the first dependency checker DPc1 may be an example of the dependency checker DPc of FIG. 17. The first dependency checker (DPc1) may include a first dependency matrix (DPmx1) therein. The first dependency matrix (DPmx1) may be a square matrix. The first dependency matrix DPmx1 may include a plurality of rows and a plurality of columns, and may exemplarily include a 4 However, this embodiment is not limited to this. That is, the number of rows and columns of the first dependency matrix DPmx1 may be 5 or more or 3 or less. At this time, the number of rows and columns of the first dependency matrix DPmx1 may be n each. That is, the number of check-in buffers of the first queue (Q1), the second queue (Q2), and the dun passage 630 and the number of rows and columns of the first dependency matrix (DPmx1) may all be the same.

Each column of the first dependency matrix (DPmx1) can be updated through the first to fourth dependency set codes (DFURQ1 to DFURQ4) generated by the dependency update request (DFURQ). That is, the first dependency checker (DPc1) can generate dependency set codes (DFURQ1 to DFURQ4) through a dependency update request (DFURQ).

That is, the first to fourth columns of the first dependency matrix DPmx1 may be updated to be the same as the values of the first to fourth dependency set codes DFURQ1 to DFURQ4, respectively.

For example, if the dependency set code of the first dependency matrix (DPmx1) is “1001”, e11 may be 1, e12 may be 0, e13 may be 0, and e14 may be 1. Similarly, other elements may also be updated by the second to fourth dependency set codes (DFURQ2 to DFURQ4).

After the first dependency matrix (DPmx1) is updated in this way, it can be compared with the first to fourth dependency check codes (chk1 to chk4). That is, the first dependency checker DPc1 may receive the first to nth task descriptors Tsk_d1 to Tsk_dn, and generate a dependency check code according to each task descriptor. The dependency check code may correspond to each row of the first dependency matrix (DPmx1).

That is, the first to fourth dependency check codes (chk1 to chk4) may respectively correspond to the first to fourth rows of the first dependency matrix (DPmx1). For example, the first dependency check code (chk1) may have a first row as a corresponding row, and the fourth dependency check code (chk4) may have a fourth row as a corresponding row. Accordingly, the first to fourth dependency check codes (chk1 to chk4) may be compared with the first to fourth rows of the first dependency matrix (DPmx1), respectively. For example, the first dependency checker (DPc1) is operated when the first dependency check code (chk1) and the first rows (e11, e21, e31, e41) of the first dependency matrix (DPmx1) are the same. The first task descriptor (Tsk_d1) can be transmitted to check in, and the first row of the first dependency matrix (DPmx1) can be cleared. At this time, clear may mean removing all data in the first row of the first dependency matrix (DPmx1). The checked-in task descriptor may be transmitted to the second queue (Q2).

At this time, data in the first row of the first dependency matrix (DPmx1) may be a value updated by the first to fourth dependency set codes (DFURQ1 to DFURQ4). The first to fourth dependency set codes (DFURQ1 to DFURQ4) may be transmitted to the first dependency checker (DPc1) when each descriptor of the first to nth task descriptors (Tsk_d1 to Tsk_dn) is checked out. there is. Accordingly, the first to fourth dependency set codes (DFURQ1 to DFURQ4) can set the first dependency matrix (DPmx1) to determine the next task descriptor with a dependency relationship.

If the first dependency check code (chk1) and the first rows (e11, e21, e31, e41) of the first dependency matrix (DPmx1) are not the same, the first dependency checker (DPc1) The first task descriptor (Tsk_d1) until the first dependency check code (chk1) and the first row (e11, e21, e31, e41) of the first dependency matrix (DPmx1) are the same by additional update. may not be delivered to the second queue (Q2).

Referring again to FIG. 17, the dependency checker (DPc) may sequentially transmit the first to nth task descriptors (Tsk_d1 to Tsk_dn) to the second queue (Q2) according to the above method.

At this time, the second queue Q2 may include the 2_1st to 2_nth queues (Q2_1 to Q2_n). The 2_1 queue (Q2_1) may store the first task descriptor (Tsk_d1), and the 2_2 queue (Q2_2) may store the second task descriptor (Tsk_d2). The 2_nth queue (Q2_n) can store the nth task descriptor (Tsk_dn). The number of second queues Q2 may be the same as the number of first queues Q1.

The first to nth task descriptors (Tsk_d1 to Tsk_dn) may also include a wait item. The waiting item may be an item specified by software. A task descriptor for which a wait item is set may wait depending on whether a specific condition is satisfied and then be delivered sequentially instead of being delivered immediately according to a pre-given instruction.

The runtime handle (RH) can generate first to nth task information (Tsk_d1' to Tsk_dn') by extracting necessary information from each of the first to nth task descriptors (Tsk_d1 to Tsk_dn). The runtime handle (RH) may transmit first to nth task information (Tsk_d1' to Tsk_dn') to the core global 500. At this time, the first to nth task information (Tsk_d1' to Tsk_dn') may correspond to the first to nth task descriptors (Tsk_d1 to Tsk_dn), respectively. At this time, each of the first to nth task information (Tsk_d1' to Tsk_dn') may be the same as the first to nth task descriptors (Tsk_d1 to Tsk_dn). However, this embodiment is not limited to this.

The runtime handle (RH) can transmit check-in data (ChI) to the Dawn passage 630. Check-in data (ChI) may include first to nth task descriptors (Tsk_d1 to Tsk_dn). The check-in data (ChI) is the first to nth task information (Tsk_d1' to Tsk_dn') corresponding to the first to nth task descriptors (Tsk_d1 to Tsk_dn) that leaves the task passage 620 and returns to the core global (500) for operation. ) can be informed by the Dungeon Passage (630).

FIG. 18 is a diagram for explaining the first dependency matrix and the second dependency matrix of FIG. 17.

Referring to FIG. 18, the dependency checker DPc may include at least a first dependency matrix DPmx_1 and a second dependency matrix DPmx_2. Each of the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) may be a square matrix. Each of the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) may include a plurality of rows and a plurality of columns, and in FIG. 18, may exemplarily include a 4 x 4 matrix. However, this embodiment is not limited to this. That is, the number of rows and columns of each of the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) may be 5 or more or 3 or less. At this time, the number of rows and columns of each of the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) may be n. That is, the number of check-in buffers of the first queue Q1, the second queue Q2, and the dun passage 630 and the number of rows and columns of the first dependency matrix DPmx_1 may all be the same.

Each column of the first dependency matrix (DPmx_1) can be updated through the first to fourth dependency set codes (DFURQ1_1 to DFURQ1_4) generated by the first dependency field update request (DFURQ1). That is, the dependency checker (DPc) can generate dependency set codes (DFURQ1_1 to DFURQ1_4) through the first dependency field update request (DFURQ1). That is, the first to fourth columns of the first dependency matrix (DPmx_1) may be updated to be the same as the values of the first to fourth dependency set codes (DFURQ1_1 to DFURQ1_4), respectively. For example, if the first dependency matrix (DPmx_1) is “1001”, e11 may be 1, e12 may be 0, e13 may be 0, and e14 may be 1. Similarly, other elements may also be updated by the second to fourth dependency set codes (DFURQ1_2 to DFURQ1_4).

Each column of the second dependency matrix (DPmx_2) can be updated through the first to fourth dependency set codes (DFURQ2_1 to DFURQ2_4) generated by the second dependency field update request (DFURQ2). That is, the dependency checker (DPc) can generate dependency set codes (DFURQ2_1 to DFURQ2_1) through the second dependency field update request (DFURQ2). That is, the first to fourth columns of the second dependency matrix (DPmx_2) may be updated to be the same as the values of the first to fourth dependency set codes (DFURQ2_1 to DFURQ2_4), respectively.

After the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) are updated in this way, they can be compared with the first to fourth dependency check codes (chk1 to chk4), respectively. The dependency checker (DPc) may receive the first to nth task descriptors (Tsk_d1 to Tsk_dn) and generate a dependency check code according to each task descriptor. The dependency check code may correspond to each row of the input dependency matrix.

Exemplarily, in FIG. 18, the first to fourth dependency check codes (chk1 to chk4) generated in response to the first task descriptor (Tsk_d1) may be input to the first dependency matrix (DPmx_1). , the first to fourth dependency check codes (chk1 to chk4) generated in response to the third task descriptor (Tsk_d3) may be input to the second dependency matrix (DPmx_2). In an embodiment, the task descriptor may have a dependency matrix input for dependency check predetermined in the command processor 7000. The dependency checker (DPc) can input a task descriptor into a predetermined dependency matrix to perform a dependency check. However, the embodiment of the present invention is not limited to this, and the dependency checker (DPc) checks the dependency by randomly or sequentially assigning the task descriptor to one of the 1st to mth dependency matrices (DPmx_1 to DPmx_m). can be performed.

As an example, a process in which a dependency check is performed in the first dependency matrix DPmx_1 will be described. The first to fourth dependency check codes (chk1 to chk4) may respectively correspond to the first to fourth rows of the first dependency matrix (DPmx_1). For example, the first dependency check code (chk1) may have a first row as a corresponding row, and the fourth dependency check code (chk4) may have a fourth row as a corresponding row. Accordingly, the first to fourth dependency check codes (chk1 to chk4) may be compared with the first to fourth rows of the first dependency matrix (DPmx_1), respectively. For example, the dependency checker (DPc) is the first dependency check code (chk1) when the first rows (e11, e21, e31, e41) of the first dependency matrix (DPmx_1) are the same. The task descriptor (Tsk_d1) can be transmitted to check in, and the first row of the first dependency matrix (DPmx_1) can be cleared. At this time, clear may mean removing all data in the first row of the first dependency matrix (DPmx_1). The checked-in first task descriptor may be transmitted to the second queue (Q2).

At this time, data in the first row of the first dependency matrix (DPmx_1) may be a value updated by the first to fourth dependency set codes (DFURQ1 to DFURQ4). The first to fourth dependency set codes (DFURQ1 to DFURQ4) may be transmitted to the dependency checker (DPc) when each descriptor of the first to nth task descriptors (Tsk_d1 to Tsk_dn) is checked out. Accordingly, the first to fourth dependency set codes (DFURQ1 to DFURQ4) can set the first dependency matrix (DPmx_1) to determine the next task descriptor with a dependency relationship.

If the first dependency check code (chk1) and the first row (e11, e21, e31, e41) of the first dependency matrix (DPmx_1) are not the same, the dependency checker (DPc) is added. By updating, the first task descriptor (Tsk_d1) is maintained until the first dependency check code (chk1) and the first rows (e11, e21, e31, e41) of the first dependency matrix (DPmx_1) are the same. 2 It may not be delivered to queue (Q2).

FIG. 19 is a block diagram for explaining the Dawn passage of FIG. 15 in detail.

Referring to FIG. 19, the dawn passage 630 may include a check-in buffer (Cib), dependency setters (DPs), timeout monitor (ToM), event monitor (EM), and report management module 631. there is.

The check-in buffer (Cib) can receive check-in data (ChI). The check-in buffer (Cib) may include first to n-th check-in buffers (Cib_1 to Cib_n). The check-in buffer (Cib) may store the first to n-th task descriptors (Tsk_d1 to Tsk_dn) stored in the check-in data (ChI). The first to nth check-in buffers (Cib_1 to Cib_n) may store the first to nth task descriptors (Tsk_d1 to Tsk_dn), respectively. The check-in buffer (Cib) can perform check-in of the first to n-th task descriptors (Tsk_d1 to Tsk_dn) through this.

That is, the first check-in buffer (Cib_1) can store the first task descriptor (Tsk_d1), and the second check-in buffer (Cib_2) can store the second task descriptor (Tsk_d2). The nth check-in buffer (Cib_n) may store the nth task descriptor (Tsk_dn). The number of check-in buffers (Cib) may be equal to the number of first queues (Q1) and the number of second queues (Q2).

The check-in buffer (Cib) may receive a completion signal from the core global 500. At this time, the completion signal may include the first to nth completion signals (Tsk_d1d to Tsk_dnd). The first to nth completion signals (Tsk_d1d to Tsk_dnd) may be completion signals for the first to nth task descriptors (Tsk_d1 to Tsk_dn), respectively. The first to nth completion signals (Tsk_d1d to Tsk_dnd) may be received into the first to nth check-in buffers (Cib_1 to Cib_n), respectively. That is, the first check-in buffer (Cib_1) may receive the first completion signal (Tsk_d1d), and the second check-in buffer (Cib_2) may receive the second completion signal (Tsk_d2d). The nth check-in buffer (Cib_n) may receive the nth completion signal (Tsk_dnd).

Dependency setters (DPs) may receive a completion signal from the check-in buffer (Cib) and generate a dependency update request (DFURQ). Depending on which task descriptor the task corresponding to has been completed, dependency setters (DPs) can generate a dependency update request (DFURQ).

Dependency setters (DPs) may transmit a dependency update request (DFURQ) to the task passage 620.

At this time, the dependency update request (DFURQ) may include content about the dependency set codes (DFURQ1 to DFURQ4) of FIG. 18 described above. That is, the dependency update request (DFURQ) may include dependency information of the task descriptor that is currently checked out. That is, the dependency update request (DFURQ) may include information about which task descriptor can be executed as the dependency limit is removed by the task descriptor currently checked out.

In an embodiment, the dependency information included in the task descriptor may include information about a dependent task descriptor that has a dependency relationship with the task descriptor and dependency matrix information in which the dependent task descriptor is checked. Accordingly, the dependency update request (DFURQ) may include dependent task descriptor information and dependency matrix information in which the dependency of the dependent task descriptor is checked. As an example, the second task descriptor (Tsk_d2) is dependent on the first task descriptor (Tsk_d1) to be checked out, and the second task descriptor (Tsk_d2) checks dependency on the third dependency matrix (DPmx_3). When input for, the dependency setter (DPs) provides a dependency set code in the third dependency matrix to check the dependency of the second task descriptor (Tsk_d2) dependent on the first task descriptor (Tsk_d1). A third dependency field update request (DFURQ3) that delivers can be configured. The third dependency field update request (DFURQ3) may be provided to the third dependency matrix (DPmx_3) to set the dependency on the second task descriptor (Tsk_d2).

Dependency setters (DPs) may also check out each of the first to nth task descriptors (Tsk_d1 to Tsk_dn) according to the completion signal. Accordingly, dependency setters (DPs) can generate a checkout report (COrp) about which tasks have been completed and checked out. Dependency setters (DPs) can transmit the checkout report (COrp) to the report management module 631.

As the dependency setters (DPs) transmit a dependency update request (DFURQ) to the task passage 620, the dependency checker (DPc) of the task passage 620 updates the dependency checker (DPc) according to the dependency of each task descriptor. Sequential transmission may be possible.

In this embodiment, the command processor 7000 does not take full charge of processing according to dependencies, but allows the task manager 600 to directly perform dependency checking and setting, thereby improving communication with the command processor 7000. Overhead can be minimized. Accordingly, the performance and speed of the neural processing device 1 according to this embodiment can be dramatically improved.

The timeout monitor (ToM) can receive a timeout detection signal (TOdec) from the check-in buffer (Cib). The timeout detection signal (TOdec) may be a signal indicating whether the time from check-in to check-out exceeds a preset time. The timeout monitor (ToM) can generate a timeout report (TOrp) according to the timeout detection signal (TOdec). The timeout monitor (ToM) can transmit a timeout report (TOrp) to the report management module 631.

The timeout monitor (ToM) may determine the check-in and check-out of each task descriptor and enable the command processor 7000 to identify an error in which a checked-in task descriptor is not checked out. Through this, data loss can be prevented and prolonged delays due to dependency can be prevented.

In this embodiment, an event according to at least one of a hardware or software error, a log-related event, and a task performed without a descriptor may occur. The embodiments of the present invention are not limited thereto. An event monitor (EM) can detect whether such an event occurs and generate an event report (Erp) according to the event detection signal. The generated event report (Erp) may be provided to the report management module 631.

The report management module 631 may generate a completion report (DNrp) by receiving at least one of a transfer report (TRrp), an event report (Erp), a checkout report (COrp), and a timeout report (TOrp). . At this time, an event report (Erp) may be generated by detecting an event inside the neural processor 1000. The event may include, for example, at least one of a hardware or software error, a log-related event, and a task performed without a descriptor. However, this embodiment is not limited to this.

A completion report (DNrp) can be written to comprehensively report whether a task has been performed, whether a timeout has occurred, etc. The report management module 631 may report a completion report (DNrp) to the command processor 7000.

FIG. 20 is a block diagram for explaining the report management module of FIG. 19 in detail.

Referring to FIG. 20, the report management module 631 includes a transfer report queue (TQ), an event report queue (EQ), a checkout report queue (CQ), a timeout report queue (TOQ), and a reporting manager (RM). may include.

The Transfer Dawn Report Queue (TQ) can receive the Transfer Dawn Report (TRrp) and deliver it to the Reporting Manager (RM). The event report queue (EQ) can receive the event report (Erp) and deliver it to the reporting manager (RM). The checkout report queue (CQ) can receive the checkout report (Corp) and deliver it to the reporting manager (RM). Additionally, the timeout report queue (TOQ) can receive the timeout report (Torp) and deliver it to the reporting manager (RM).

The reporting manager (RM) can receive at least one of the transfer report (TRrp), event report (Erp), checkout report (Corp), and timeout report (Torp), and generate a completion report (DNrp) through this. there is. The reporting manager (RM) may transmit the completion report (DNrp) to the command processor 7000.

FIG. 21 is a diagram illustrating data exchanged between the core global and the task processing unit of FIG. 15.

Referring to FIG. 21, the core global 500 may receive a table update request (TURQ) and transmit it to the LSU 110 of the neural core 100. Additionally, the core global 500 may receive task information (Tsk_d') and transmit it to the neural core 100.

The neural core 100 may perform a task and generate a completion signal. The neural core 100 may transmit a completion signal to the core global 500. Core global 500 may include a signal scheduler (sgn_sch). The signal scheduler (sgn_sch) may receive a completion signal, schedule transmission of the completion signal, and transmit it to the Dawn passage 630.

Figure 22 is a diagram for explaining the types of task descriptors stored in the first queue, second queue, and check-in buffer.

Referring to FIG. 22, the 1_1st to 1_4th queues (Q1_1 to Q1_4) of the first queue (Q1), the 2_1st to 2_4th queues (Q2_1 to Q2_4) of the second queue (Q2), and the check-in buffer (Cib) The first to fourth check-in buffers (Cib_1 to Cib_4) may each store a specific type of task descriptor. The 1_1st to 1_4th queues (Q1_1 to Q1_4), the 2_1st to 2_4th queues (Q2_1 to Q2_4), and the 1st to 4th check-in buffers (Cib_1 to Cib_4) may each store different types of task descriptors.

For example, the 1_1 queue (Q1_1), the 2_1 queue (Q2_1), and the first check-in buffer (Cib_1) store the task descriptor for computation, and the 1_2 queue (Q1_2), the 2_2 queue (Q2_2), and The second check-in buffer (Cib_2) may store a task descriptor for micro DMA. In addition, the 1_3 queue (Q1_3), the 2_3 queue (Q2_3), and the third check-in buffer (Cib_3) store the task descriptor for LP micro DMA, and the 1_4 queue (Q1_4), the 2_4 queue (Q2_4), and the 4 Check-in buffer (Cib_4) can store a task descriptor for ST micro DMA. However, this embodiment is not limited to this.

FIG. 23 is a diagram illustrating an exemplary dependency matrix for checking the dependency of the task descriptor according to FIG. 22.

Referring to FIG. 23, the first to fourth columns of the dependency matrix (DPmx) stored inside the dependency checker (DPc) may each correspond to the type of the task descriptor. For example, in the dependency matrix (DPmx), the first to fourth columns may correspond to computation, micro DMA, LP micro DMA, and ST micro DMA, respectively. Additionally, the first to fourth rows of the dependency matrix DPmx may correspond to computation, micro DMA, LP micro DMA, and ST micro DMA, respectively. However, this embodiment is not limited to this. The type of task descriptor may further include preprocessing, and computation may be divided into detailed computation and defined as the type of task descriptor.

The first column of the dependency matrix (DPmx) may be updated by the computation set code (COMP_s). The second column of the dependency matrix (DPmx) can be updated by the micro DMA set code (μDMA_s). The third column of the dependency matrix (DPmx) can be updated by the micro LP micro DMA set code (LPμDMA_s). The fourth column of the dependency matrix (DPmx) can be updated by the micro ST micro DMA set code (STμDMA_s).

Additionally, the first row of the dependency matrix (DPmx) is compared with the computation check code (COMP_c), the second row is compared with the micro DMA set code (μDMA_c), and the third row is compared with the LP micro DMA set code ( LPμDMA_c), and the fourth row may be compared with the ST micro DMA set code (STμDMA_c).

Accordingly, the dependency matrix (DPmx) may include CC, CU, CL, and CS in the first column. At this time, CC may mean dependency from computation to computation, and CU may mean dependency from computation to micro DMA. Similarly, CL may mean a dependency from computation to LP micro DMA, and CS may mean a dependency from computation to ST micro DMA. In other words, the dependency from the task of the type corresponding to each column to the task of the type corresponding to each column may refer to each element of the dependency matrix (DPmx).

Figures 27 to 30 are exemplary diagrams for explaining the operation of the second dependency matrix.

Referring to FIGS. 23 and 24, when all other elements of the second dependency matrix (DPmx2) are “0” and “1” is recorded in UC (when “1” is defined as a display signal), This may mean that the dependency is set from the micro DMA task descriptor (Tsk_d_u) to the computation task descriptor (Tsk_d_c).

Referring to FIGS. 23 and 25, when all other elements of the second dependency matrix (DPmx2) are “0” and “1” is recorded in UC and LC (when “1” is defined as the display signal) ), this may mean that the dependency is set from the micro DMA task descriptor (Tsk_d_u) and LP micro DMA task descriptor (TSK_D_L) to the computation task descriptor (Tsk_d_c). This may mean that in order for the computation task descriptor (Tsk_d_c) to be delivered, both the micro DMA task descriptor (Tsk_d_u) and the LP micro DMA task descriptor (TSK_D_L) must be checked out.

Referring to FIGS. 23 and 26, when all other elements of the second dependency matrix (DPmx2) are “0” and “1” is recorded in CS, CL, and CU (“1” is defined as the display signal This may mean that the dependency is set from the computation task descriptor (Tsk_d_c) to the micro DMA task descriptor (Tsk_d_u), LP micro DMA task descriptor (Tsk_d_l), and ST micro DMA task descriptor (Tsk_d_s). This may mean that the computation task descriptor (Tsk_d_c) must be checked out in order for the micro DMA task descriptor (Tsk_d_u), LP micro DMA task descriptor (Tsk_d_l), and ST micro DMA task descriptor (Tsk_d_s) to be transmitted, respectively.

Referring to FIGS. 23 and 27, when all elements of the second dependency matrix (DPmx2) are “1” (when “1” is defined as a display signal), the primary task including all four types This may mean that dependencies are set from all of the descriptors (Tsk_d_1) to each of the secondary task descriptors (Tsk_d_2) that include all four types. In other words, this may mean that in order for each secondary task descriptor (Tsk_d_2) to be transmitted, all primary task descriptors (Tsk_d_1) must be checked out.

In this embodiment, the task manager 600 can quickly check and process the dependency of each task without additional data exchange with the command processor 7000. Through this, the speed of task processing is improved, and sequential processing between tasks can be performed smoothly.

FIG. 28 is a diagram for explaining the first dependency matrix and the second dependency matrix of FIG. 17.

Referring to FIG. 18, the dependency checker DPc may include at least a first dependency matrix DPmx_1 and a second dependency matrix DPmx_2. Each of the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) may be a square matrix. Each of the first dependency matrix DPmx_1 and the second dependency matrix DPmx_2 may include a plurality of rows and a plurality of columns, and in FIG. 28, may exemplarily include a 4 x 4 matrix. However, this embodiment is not limited to this. That is, the number of rows and columns of each of the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) may be 5 or more or 3 or less. At this time, the number of rows and columns of each of the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) may be n. That is, the number of check-in buffers of the first queue Q1, the second queue Q2, and the dun passage 630 and the number of rows and columns of the first dependency matrix DPmx_1 may all be the same.

Each column of the first dependency matrix (DPmx_1) can be updated through the first to fourth dependency set codes (DFURQ1_1 to DFURQ1_4) generated by the first dependency field update request (DFURQ1). That is, the dependency checker (DPc) can generate dependency set codes (DFURQ1_1 to DFURQ1_4) through the first dependency field update request (DFURQ1). That is, the first to fourth columns of the first dependency matrix (DPmx_1) may be updated to be the same as the values of the first to fourth dependency set codes (DFURQ1_1 to DFURQ1_4), respectively. For example, if the first column of the first dependency matrix (DPmx_1) is “1001”, e11 may be 1, e12 may be 0, e13 may be 0, and e14 may be 1. Similarly, other elements may also be updated by the second to fourth dependency set codes (DFURQ1_2 to DFURQ1_4).

Each column of the second dependency matrix (DPmx_2) can be updated through the first to fourth dependency set codes (DFURQ2_1 to DFURQ2_4) generated by the second dependency field update request (DFURQ2). That is, the dependency checker (DPc) can generate dependency set codes (DFURQ2_1 to DFURQ2_1) through the second dependency field update request (DFURQ2). That is, the first to fourth columns of the second dependency matrix (DPmx_2) may be updated to be the same as the values of the first to fourth dependency set codes (DFURQ2_1 to DFURQ2_4), respectively.

After the first dependency matrix (DPmx_1) and the second dependency matrix (DPmx_2) are updated in this way, they can be compared with the first to fourth dependency check codes (chk1 to chk4), respectively. The dependency checker (DPc) may receive the first to nth task descriptors (Tsk_d1 to Tsk_dn) and generate a dependency check code according to each task descriptor. The dependency check code may correspond to each row of the input dependency matrix.

Exemplarily, in FIG. 28, the first to fourth dependency check codes (chk1 to chk4) generated in response to the first task descriptor (Tsk_d1) may be input to the first dependency matrix (DPmx_1). , the first to fourth dependency check codes (chk1 to chk4) generated in response to the third task descriptor (Tsk_d3) may be input to the second dependency matrix (DPmx_2). In an embodiment, the task descriptor may have a dependency matrix input for dependency check predetermined in the command processor (CP). The dependency checker (DPc) can input a task descriptor into a predetermined dependency matrix to perform a dependency check. However, the embodiment of the present invention is not limited to this, and the dependency checker (DPc) checks the dependency by randomly or sequentially assigning the task descriptor to one of the 1st to mth dependency matrices (DPmx_1 to DPmx_m). can be performed.

Figure 29 is a conceptual diagram to explain the latency reduction effect for each group.

Referring to FIG. 29, when there is only one dependency matrix, additional latency may occur for the same type of dependency. That is, for example, when there is a dependency from LPuDMA to COMP, when there is one dependency matrix, the first LPuDMA is completed and set, and after the first COMP is delivered and the dependency matrix is cleared, the second LPuDMA is set. It can be set.

On the other hand, if there are multiple groups of dependency matrices, LPuDMA can be completed and set in the second group immediately without having to wait for the first COMP to be delivered. Additionally, there is no need to wait for the first LPuDMA to complete and be set. Therefore, latency due to check and set can be minimized, dramatically increasing the speed of the entire device.

FIG. 30 exemplarily illustrates a plurality of task descriptors and one task descriptor whose dependency is checked in different dependency matrices.

Referring to FIG. 30, a dependency is formed from LPuDMA1 of the first dependency matrix (DPmx1) to COMP9 of the third dependency matrix (DPmx3), and the third dependency matrix (DPmx2) Dependency can be formed with COMP9 of the dependency matrix (DPmx3). Accordingly, a dependency is formed between two task descriptors of different groups and one task descriptor of another group.

Figure 31 is a diagram for explaining a dummy check descriptor.

Referring to FIG. 31, since COMP9 of the third dependency matrix DPmx3 has a dependency with COMP6 and LPuDMA1, COMP6 and LPuDMA1 must be checked out respectively for COMP to be transmitted and performed.

Accordingly, if the dependency check code of COMP9 is performed for LPuDMA1, it is difficult to perform the dependency check code for COMP6 again, so the dependency dummy check code generated by the dummy check descriptor is additionally added to the third dependency. It can be created in the time matrix (DPmx3).

At this time, the dummy check descriptor may be checked before the corresponding task descriptor. Through this, the task descriptor must be finally checked before the task corresponding to the task descriptor can be transmitted and performed.

Specifically, the dependency formation of COMP9 with respect to COMP6 may be processed in the second dependency matrix (DPmx2) before the third dependency matrix (DPmx3). At this time, the dependency of COMP9 can be processed through a dummy check code. At this time, there may be one or more dummy check descriptors. That is, when multiple dependencies are set, all dependencies other than the subsequent task descriptor itself can be treated as dummy check descriptors. For example, 3 dependencies may require 2 dummy check descriptors. In this embodiment, many-to-one dependencies between different groups can be freely processed through a plurality of groups.

Figure 32 is a diagram to explain that a dummy check descriptor is burned in the dependency checker.

Referring to FIG. 32, for example, the second dummy check descriptor (DC_Tsk_d2) may be received in the dependency checker (DPc). Since the dummy check descriptor can be generated and received by the command processor 7000 only when necessary, it can be received by the dependency checker (DPc) in response to some task descriptors, not all task descriptors.

The dependency checker (DPc) can identify a task descriptor and a dummy check descriptor among the received descriptors. Additionally, dummy set descriptors and dummy check descriptors can also be identified. At this time, the task descriptor and the dummy set descriptor must be checked out by passing the dependency checker (DPc), but the dummy check descriptor does not need to be checked out and can be identified.

The second dummy check descriptor (DC_Tsk_d2) may be burned out within the dependency checker (DPc) when the dependency check is completed. That is, since the second dummy check descriptor (DC_Tsk_d2) does not need to be transmitted to the core global 500 or checked out, it can be burned by the dependency checker (DPc).

Figure 33 exemplarily shows one task descriptor and a plurality of task descriptors whose dependencies are checked in different dependency matrices.

In an embodiment, one task descriptor may be configured to have dependencies between a plurality of task descriptors, and the plurality of task descriptors may be configured to have dependencies checked in different dependency matrices.

Referring to FIG. 33, the third computation task descriptor (COMP3) whose dependency is checked in the first dependency matrix (DPmx_1) is the second computation task descriptor (COMP3) whose dependency is checked in the second dependency matrix (DPmx_2). A dependency relationship may be established in the micro DMA task descriptor (uDMA2) and the third LP micro DMA task descriptor (LPuDMA3) whose dependency is checked in the third dependency matrix (DPmx_3).

However, a dependency update request (DFURQ) generated as one task descriptor is checked out may be generated in response to one dependency matrix. In other words, one dependency update request (DFURQ) cannot update two dependency matrices simultaneously.

As an example, as the third computation task descriptor (COMP3) is checked out, only the third dependency update request (DFURQ) is generated and only the third dependency matrix (DPmx_3) is updated to update the third micro DMA task descriptor ( Dependency setting for uDMA3) can be performed. That is, an update to the second dependency matrix (DPmx_2) and a dependency setting to the second LP micro DMA task descriptor (LPuDMA2) are required, but this is done in response to the checkout of the third computation task descriptor (COMP3). not performed

In an embodiment of the present invention, when a specific task descriptor forms a dependency between a plurality of task descriptors whose dependencies are checked in different dependency matrices, the command processor (CP) A dummy set descriptor corresponding to a specific task descriptor can be configured so that dependency setting can be performed. In other words, dependency settings for the remaining dependency matrices are performed through the dummy set descriptor.

Figure 34 is a diagram for explaining a dummy set descriptor.

Referring to FIG. 34, as the third computation task descriptor (COMP3) of FIG. 33 is checked out, a third dependency update request (DFURQ) is generated, and only the third dependency matrix (DPmx_3) is updated to update the third dependency update request (DFURQ). Dependency setting for the micro DMA task descriptor (uDMA3) may be performed. According to the third dependency update request (DFURQ) generated in response to the third computation task descriptor (COMP3), all other elements of the third dependency matrix (DPmx_3) are “0”, and CU is “1”. This can be updated to record.

The update to the second dependency matrix (DPmx_2), that is, the dependency setting for the second LP micro DMA task descriptor (LPuDMA2), is performed in the dummy set descriptor (Dummy set COMP3) of the third computation task descriptor (COMP3). It can be performed by . According to the second dependency field update request (DFURQ2) generated in response to the dummy set descriptor (Dummy set COMP3), all other elements of the second dependency matrix (DPmx_2) are “0”, and CL is “1”. This can be updated to record.

This dummy set descriptor may be configured to include only information for dependency setting for the task descriptor whose dependency is checked in a specific dependency matrix. That is, the dummy set descriptor may be configured so that no actual computation or operation is performed, and only a checkout process for generating a dependency field update request may be performed. That is, the dummy set descriptor is not provided to the task processing unit 190, a completion signal may be generated in the void core 640, and the dependency setters (DPs) of the dawn passage 630 are generated in the void core 640. A dependency field update request can be generated in response to the completion signal generated in .

FIG. 35 is a diagram for explaining the data exchange process performed in the void core of FIG. 15.

In FIG. 35, the task descriptor DS_Tsk_d output from the task passage 620 is a task descriptor on which a dependency check has been performed and corresponds to a dummy set descriptor of a specific task descriptor. However, the dummy set descriptor (DS_Tsk_d) is not provided to the neural core 100, but to the void core 640. That is, the dummy set descriptor (DS_Tsk_d) is not provided to the core global 500, and the completion signal (DS_tsk_dd) is generated through the void core 640.

In an embodiment, a task descriptor on which an actual operation or operation is performed is defined as a real task descriptor, and a task descriptor on which an actual operation or operation is not performed is defined as a dummy task descriptor. Additionally, the task descriptor configured to set the update of a specific dependency matrix in the dummy task descriptor is defined as a dummy set descriptor.

In some embodiments, the task passage 620 may identify the type of task descriptor, and if the task descriptor is a real task descriptor, that is, if it contains information related to an actual operation or operation, the task descriptor is a core global ( 500). Additionally, when the task descriptor is a dummy set descriptor, the task passage 620 may process the task descriptor to be provided to the void core 640. In some embodiments, determination of the type of task descriptor may be performed in the runtime handle (RH), but is not limited to this, and may also be performed in the dependency checker (DPc).

In some embodiments, the dummy set descriptor (DS_Tsk_d) on which the dependency check was performed may be transmitted to the second queue. The runtime handle (RH) may identify the dummy set descriptor (DS_Tsk_d) waiting in the second queue and provide it to the void core 640. In some embodiments, the dependency checker (DPc) may also identify the type of task descriptor along with the dependency check. The dependency checker (DPc) may transmit the dummy set descriptor (DS_Tsk_d) to the void core 640 without going through the second queue and runtime handle (RH).

The void core 640 may receive a dummy set descriptor (DS_Tsk_d) on which a dependency check has been performed from the task passage 620. The task passage 620 may provide a dummy set descriptor (DS_Tsk_d) to the void core 640 and transmit check-in data (ChI) corresponding to the dummy set descriptor (DS_Tsk_d) to the dune passage 630. Specifically, the dependency checker (DPc) of the task passage 620 may transmit check-in data (ChI) corresponding to the dummy set descriptor (DS_Tsk_d) to the check-in buffer (Cib) of the dummy passage 630. The check-in buffer (Cib) waits for a task completion signal according to the provided check-in data (ChI).

The void core 640 may generate a completion signal (DS_tsk_dd) in response to the provided dummy set descriptor (DS_Tsk_d). The void core 640 may transmit the generated completion signal DS_tsk_dd to the Dawn passage 172. In an embodiment, void core 640 may include a signal scheduler (sgn_sch). The signal scheduler (sgn_sch) may receive the completion signal (DS_tsk_dd), schedule transmission of the completion signal (DS_tsk_dd), and transmit it to the dawn passage 630. That is, the signal scheduler (sgn_sch) can control the timing at which the generated completion signal is delivered to the Dawn passage 630.

Additionally, since the void core 640 does not perform actual calculations or operations, the table update request (TURQ) can be deleted without being transmitted to the LSU 110 of the neural core 100.

The check-in buffer (Cib) of the Dawn passage 172 may receive a completion signal from the void core 640, and the dependency setters (DPs) may receive a completion signal from the check-in buffer (Cib) and request a dependency update. (DFURQ) can be created. The dummy set descriptor (DS_Tsk_d) may include information about a dependent task descriptor that has a dependency relationship with the dummy set descriptor (DS_Tsk_d) and dependency matrix information in which the dependent task descriptor is checked. Dependency setters (DPs) may generate a dependency update request (DFURQ) containing the dependency information of the dummy set descriptor (DS_Tsk_d) in response to the completion signal, and the generated dependency update request ( DFURQ) is passed to the corresponding dependency matrix so that the dependency information in the dependency field is updated.

That is, through the dummy set descriptor, dependency settings for a plurality of dependency matrices can be performed without causing duplication of task calculations and/or operations.

In this embodiment, the task manager 600 can quickly check and process the dependency of each task without additional data exchange with the command processor 7000. In addition, this embodiment is configured so that dependencies can be set for a plurality of task descriptors provided in different dependency matrices, so that the dependency of the entire task can be configured. Through this, the speed of task processing can be further improved, parallel processing between tasks can be performed more smoothly, and traffic between the command processor and task manager can be minimized to maximize the processing speed and efficiency of the entire device.

FIG. 36 is a diagram illustrating a first queue, a second queue, and a check-in buffer of a neural processing device according to some embodiments of the present invention.

Referring to FIG. 36, the 1_1st to 1_4th queues (Q1_1 to Q1_4), the 2_1st to 2_4th queues (Q2_1 to Q2_4), and the 1st to 4th check-in buffers (Cib_1 to Cib_4) each have a specific type of task descriptor. You can save it. The 1_1st to 1_4th queues (Q1_1 to Q1_4), the 2_1st to 2_4th queues (Q2_1 to Q2_4), and the 1st to 4th check-in buffers (Cib_1 to Cib_4) may store task descriptors of the same type.

For example, the 1_1 queue (Q1_1), the 2_1 queue (Q2_1), and the 1st check-in buffer (Cib_1) store the task descriptor for the first computation, and the 1_2 queue (Q1_2) and the 2_2 queue (Q2_2) ) and the second check-in buffer (Cib_2) may store a task descriptor for the second computation. In addition, the 1_3 queue (Q1_3), the 2_3 queue (Q2_3), and the third check-in buffer (Cib_3) store the task descriptor for the third computation, and the 1_4 queue (Q1_4), the 2_4 queue (Q2_4), and The fourth check-in buffer (Cib_4) may store a task descriptor for the fourth computation. However, this embodiment is not limited to this.

At this time, the first to fourth computations may be completely the same computation, or may be computations of the same type but different in detail.

FIG. 37 is a diagram illustrating a first queue, a second queue, and a check-in buffer of a neural processing device according to some embodiments of the present invention.

Referring to FIG. 37, the 1_1st to 1_4th queues (Q1_1 to Q1_4), the 2_1st to 2_4th queues (Q2_1 to Q2_4), and the 1st to 4th check-in buffers (Cib_1 to Cib_4) each contain several types of task descriptors. You can save it. The 1_1st to 1_4th queues (Q1_1 to Q1_4), the 2_1st to 2_4th queues (Q2_1 to Q2_4), and the 1st to 4th check-in buffers (Cib_1 to Cib_4) may store different types of task descriptors, and may store the same task descriptors. You can also store the type's task descriptor.

Figure 38 is a block diagram to explain in detail the structure of a neural processing device according to some embodiments of the present invention.

Referring to FIG. 38, the neural core 101, unlike the neural core 100, may have a CGRA structure. The neural core 101 may include an instruction memory 111_1, a CGRA L0 memory 111_2, a PE array 111_3, and a Load/Store Unit (LSU) 111_4.

The instruction memory 111_1 can receive and store instructions. The instruction memory 111_1 may sequentially store instructions internally and provide the stored instructions to the PE array 111_3. At this time, the instruction may direct the operation of the processing element 111_3a included in each PE array 111_3.

The CGRA L0 memory 111_2 is a memory located inside the neural core 101, and allows the neural core 101 to receive all input data required for work from the outside and temporarily store them. Additionally, the CGRA L0 memory 111_2 can temporarily store output data calculated by the neural core 101 in order to transmit it to the outside. The CGRA L0 memory 111_2 may serve as a cache memory for the neural core 101.

The CGRA L0 memory 111_2 can transmit and receive data with the PE array 111_3. The CGRA L0 memory 111_2 may be a memory corresponding to L0 (level 0), which is lower than L1. At this time, the L0 memory may be a private memory of the neural core 101 that is not shared. The CGRA L0 memory (111_2) can transmit data such as activation or weight and programs to the PE array (111_3).

The PE array 111_3 may be a module that performs calculations. The PE array 111_3 can perform not only one-dimensional operations but also two-dimensional or more matrix/tensor operations. The PE array 111_3 may include a plurality of processing elements 111_3a and a specific processing element 111_3b therein.

The processing element 111_3a and the specific processing element 111_3b may be arranged in rows and columns. The processing element 111_3a and the specific processing element 111_3b may be arranged in m columns. Additionally, the processing element 111_3a may be arranged in n rows, and the specific processing element 111_3b may be arranged in l rows. Accordingly, the processing element 111_3a and the specific processing element 111_3b may be arranged in (n+l) rows and m columns.

The LSU 111_4 may receive at least one of data, control signals, and synchronization signals from the outside through the local interconnection 200. The LSU 111_4 may transmit at least one of the received data, control signal, and synchronization signal to the CGRA L0 memory 111_2. Similarly, the LSU 111_4 may transmit at least one of data, a control signal, and a synchronization signal to the outside through the local interconnection 200.

The neural core 101 may have a Coarse Grained Reconfigurable Architecture (CGRA) structure. Accordingly, the neural core 101 configures each processing element 111_3a and the specific processing element 111_3b of the PE array 111_3 at least one of the CGRA L0 memory 111_2, the instruction memory 111_1, and the LSU 111_4. It can be connected to one. That is, the processing element 111_3a and the specific processing element 111_3b do not have to be connected to all of the CGRA L0 memory 111_2, the instruction memory 111_1, and the LSU 111_4, but may be connected to some of them.

Additionally, the processing element 111_3a and the specific processing element 111_3b may be different types of processing elements. Accordingly, among the CGRA L0 memory 111_2, the instruction memory 111_1, and the LSU 111_4, the element connected to the processing element 111_3a and the element connected to the specific processing element 111_3b may be different from each other.

The neural core 101 of the present invention with a CGRA structure is capable of high-level parallel operations and direct data exchange between the processing element 111_3a and the specific processing element 111_3b, so power consumption can be low. Additionally, optimization according to various computational tasks may be possible by including two or more types of processing elements 111_3a.

For example, if the processing element 111_3a is a processing element that performs a two-dimensional operation, the specific processing element 111_3b may be a processing element that performs a one-dimensional operation. However, this embodiment is not limited to this.

FIG. 39 is a diagram illustrating the hierarchical structure of a command processor and a task manager of a neural processing device according to some embodiments of the present invention, and FIG. 33 is a diagram illustrating a command processor and a task manager of a neural processing device according to some embodiments of the present invention. This is a diagram to explain the hierarchy of managers.

Referring to FIG. 39, if the number of task managers 600 increases, it may be difficult for the command processor 7000 to manage all task managers 600. Accordingly, in the neural processing device 1 according to some embodiments of the present invention, the master task manager 600M manages a plurality of task managers 600, and the command processor 7000 manages the master task manager 600M. You can have a hi-key structure to manage.

Additionally, referring to FIG. 40, levels below the master task manager 600M may also be subdivided into various ways. For example, the first sub-task manager 600s1 and the second sub-task manager 600s2 may form separate layers. That is, one first sub-task manager 600s1 manages at least one second sub-task manager 600s2, and one master task manager 600M manages at least one first sub-task manager 600s1. can do. Additionally, multiple layers may be added below the second sub-task manager 600s2.

That is, although three levels of the task manager 600, the master task manager 600M, and the command processor 7000 are shown in FIGS. 39 and 40, the number of levels may be four or more. In other words, the depth of the hi-key structure may vary depending on the number of task managers 600.

Figure 41 is a block diagram for explaining memory reorganization of a neural processing system according to some embodiments of the present invention.

Referring to FIG. 41, the neural core SoC 10 may include first to eighth processing units 160a to 160h and an on-chip memory (OCM). 41 shows eight processing units as an example, but this is only an example and the number of processing units may vary.

The on-chip memory (OCM) may include first to eighth L0 memories 120a to 120h and a shared memory 2000.

The first to eighth L0 memories 120a to 120h may be used as dedicated memories for the first to eighth processing units 160a to 160h, respectively. That is, the first to eighth processing units 160a to 160h and the first to eighth L0 memories 120a to 120h may correspond to each other 1:1.

The shared memory 2000 may include first to eighth memory units 2100a to 2100h. The first to eighth memory units 2100a to 2100h may correspond to the first to eighth processing units 160a to 160h and the first to eighth L0 memories 120a to 120h, respectively. That is, the number of memory units may be 8, which is the same as the number of processing units and L0 memories.

Shared memory 2000 may operate in either of two types of on-chip memory formats. That is, the shared memory 2000 may operate in either an L0 memory format or a global memory format. In other words, the shared memory 2000 can implement two logical memories with one hardware.

When the shared memory 2000 is implemented in an L0 memory format, the shared memory 2000 is a dedicated memory for each of the first to eighth processing units 160a to 160h, such as the first to eighth L0 memories 120a to 120h. It can operate with (private memory). The L0 memory can operate at a relatively high clock speed compared to the global memory, and the shared memory 2000 can also use a relatively faster clock when operating in the L0 memory format.

When the shared memory 2000 is implemented in a global memory format, the shared memory 2000 can operate as a common memory that the first processing unit 100a and the second processing unit 100b use together. there is. At this time, the shared memory 2000 may be shared not only by the first to eighth processing units 160a to 160h but also to the first to eighth L0 memories 120a to 120h.

Global memory can generally use a lower clock than L0 memory, but is not limited to this. When the shared memory 2000 operates in a global memory format, the first to eighth processing units 160a to 160h may share the shared memory 2000. At this time, the shared memory 2000 is connected to the volatile memory 32 of FIG. 2 through the global interconnection 6000, and may operate as a buffer of the volatile memory 32.

At least part of the shared memory 2000 may operate in an L0 memory format, and the remainder may operate in a global memory format. That is, the entire shared memory 2000 may operate in an L0 memory format, or the entire shared memory 2000 may operate in a global memory format. Alternatively, part of the shared memory 2000 may operate in the L0 memory format, and the remaining part may operate in the global memory format.

Figure 42 is a block diagram showing an example of memory reorganization of a neural processing system according to some embodiments of the present invention.

41 and 42, the first, third, fifth, and seventh processing units (100a, 100c, 100e, and 100g) each have first, third, fifth, and seventh dedicated areas (AE1, AE3). , AE5, and AE7) may include only the first, third, fifth, and seventh L0 memories 120a, 120c, 120e, and 120g, respectively. In addition, the second, fourth, sixth, and eighth dedicated areas (AE2, AE4, AE6, and AE8) of the second, fourth, sixth, and eighth processing units (100b, 100d, 100f, and 100h) are respectively It may include second, fourth, sixth, and eighth L0 memories (120b, 120d, 120f, 120h). Additionally, the second, fourth, sixth, and eighth dedicated areas (AE2, AE4, AE6, and AE8) will include the second, fourth, sixth, and eighth memory units (2100b, 2100d, 2100f, 2100h). You can. The first, third, fifth, and seventh memory units 2100a, 2100c, 2100e, and 2100g of the shared memory 2000 may be used as a common area (AC).

The common area AC may be a memory shared by the first to eighth processing units 160a to 160h. The second dedicated area AE2 may include a second L0 memory 120b and a second memory unit 2100b. The second dedicated area AE2 may be an area in which the hardware-separated second L0 memory 120b and the second memory unit 210b operate in the same manner and logically operate as one L0 memory. The fourth, sixth, and eighth dedicated areas (AE4, AE6, and AE8) may also operate in the same manner as the second dedicated area (AE2).

The shared memory 2000 according to this embodiment can be used by converting the area corresponding to each neural core into logical L0 memory and logical global memory at an optimized ratio. The shared memory 2000 can adjust this ratio at run time.

In other words, each neural core may perform the same tasks, but may also perform different tasks. In this case, the capacity of L0 memory and the capacity of global memory required for the work performed by each neural core are bound to be different each time. Accordingly, if the composition ratio of L0 memory and shared memory is set fixedly, as in the existing on-chip memory, inefficiencies may occur due to the computational tasks assigned to each neural core.

Accordingly, the shared memory 2000 of the neural processing device according to this embodiment can set the optimal ratio of L0 memory and global memory according to the computational task during run time and improve computational efficiency and speed.

Figure 43 is an enlarged block diagram of part A of Figure 41.

41 and 43, the shared memory 2000 includes a first L0 memory controller 122_1a, a second L0 memory controller 122_1b, a fifth L0 memory controller 122_1e, and a sixth L0 memory controller 122_1f. , may include first to eighth memory units 2100a to 2100h and a global controller 2200. Other L0 memory controllers not shown may also be included in this embodiment, but descriptions are omitted for convenience.

The first L0 memory controller 122_1a can control the first L0 memory 120a. Additionally, the first L0 memory controller 122_1a may control the first memory unit 2100a. Specifically, when the first memory unit 2100a is implemented in a logical L0 memory format, control by the first L0 memory controller 122_1a may be performed on the first memory unit 2100a.

The second L0 memory controller 122_1b can control the second L0 memory 120b. Additionally, the second L0 memory controller 122_1b may control the second memory unit 2100b. That is, when the second memory unit 2100b is implemented in a logical L0 memory format, control by the first L0 memory controller 122_1a can be performed on the second memory unit 2100b.

The fifth L0 memory controller 122_1e can control the fifth L0 memory 120e. Additionally, the fifth L0 memory controller 122_1e can control the fifth memory unit 2100e. That is, when the fifth memory unit 2100e is implemented in a logical L0 memory format, control by the fifth L0 memory controller 122_1e can be performed on the fifth memory unit 2100e.

The sixth L0 memory controller 122_1f can control the sixth L0 memory 120f. Additionally, the sixth L0 memory controller 122_1f can control the sixth memory unit 2100f. That is, when the sixth memory unit 2100f is implemented in a logical L0 memory format, control by the sixth L0 memory controller 122_1f can be performed on the sixth memory unit 2100f.

The global controller 2200 can control all of the first to eighth memory units 2100a to 2100h. Specifically, when the first to eighth memory units 2100a to 2100h each logically operate in the global memory format (i.e., do not logically operate in the L0 memory format), the global controller 2200 operates the first memory The units 2100a to 8th memory units 2100h can be controlled.

That is, the first to eighth memory units 2100a to 2100h are controlled by the first to eighth L0 memory controllers 122_1a to 122_1h, respectively, or by the global controller 2200, depending on what type of memory they are logically implemented as. It can be controlled by

When L0 memory controllers including the first, second, fifth, and sixth L0 memory controllers 122_1a, 122_1b, 122_1e, and 122_1f control the first to eighth memory units 2100a to 2100h, respectively, the first Since the first to eighth L0 memory controllers 122_1a to 141h control the first to eighth memory units 2100a to 2100h in the same manner as the first to eighth L0 memories 120a to 120h, the first to eighth processing units It can be controlled with a dedicated memory of (160a~160h). Accordingly, the first to eighth memory units 2100a to 2100h may operate at a clock frequency corresponding to the clock frequency of the first to eighth processing units 160a to 160h.

The L0 memory controllers including the first L0 memory controller 122_1a, the second L0 memory controller 122_1b, the fifth L0 memory controller 122_1e, and the sixth L0 memory controller 122_1f are respectively the LSU 110 of FIG. 7. may include.

When the global controller 2200 controls at least one of the first to eighth memory units 2100a to 2100h, the global controller 2200 controls the first to eighth memory units 2100a to 2100h, respectively. It can be controlled by the global memory of the 8th processing unit (160a ~ 160h). Accordingly, at least one of the first to eighth memory units 2100a to 2100h may operate at a clock frequency that is unrelated to the clock frequency of the first to eighth processing units 160a to 160h. However, this embodiment is not limited to this.

The global controller 2200 may connect the first to eighth memory units 2100a to 2100h with the global interconnection 6000 of FIG. 3. The first to eighth memory units 2100a to 2100h exchange data with the off-chip memory 30 of FIG. 1 by the global controller 2200, or exchange data with the first to eighth L0 memories 120a to 120h, respectively. can be exchanged.

The first to eighth memory units 2100a to 2100h may each include at least one memory bank. The first memory unit 2100a may include at least one first memory bank 2110a. The first memory bank 2110a may be an area divided by dividing the first memory unit 2100a into a specific size. Each first memory bank 2110a may be a memory element of the same size. However, this embodiment is not limited to this. In Figure 43, four memory banks are shown as being included in one memory unit.

Similarly, the second, fifth, and sixth memory units 2100b, 2100e, and 2100f may each include at least one second, fifth, and sixth memory bank 2110b, 2110e, and 2110f.

Hereinafter, the description will be made based on the first memory bank 2110a and the fifth memory bank 2110e, which may be the same as other memory banks including the second and sixth memory banks 2110b and 2110f.

The first memory bank 2110a may logically operate in an L0 memory format or logically in a global memory format. At this time, the first memory bank 2110a may operate independently from other memory banks in the first memory unit 2100a. However, this embodiment is not limited to this.

When operating independently for each memory bank, the first memory unit 2100a has a first area that operates in the same manner as the first L0 memory 120a and a second area that operates in a different manner from the first L0 memory 120a. Can include 2 areas. At this time, the first area and the second area do not necessarily coexist, and one area may occupy the entire first memory unit 2100a.

Likewise, the second memory unit 2100b may include a third area that operates in the same manner as the second L0 memory 120b and a fourth area that operates in a different manner from the second L0 memory 120b. At this time, the third area and the fourth area do not necessarily coexist, and one area may occupy the entire first memory unit 2100a.

At this time, the ratio of the first area to the second area may be different from the ratio of the third area to the fourth area. However, this embodiment is not limited to this. Accordingly, the ratio of the first area to the second area may be the same as the ratio of the third area to the fourth area. That is, the memory configuration ratio in each memory unit can vary as much as desired.

In general, in the case of existing system-on-chip, the on-chip memory, excluding high-speed L0 memory, is often composed of high-density, low-power SRAM. This is because SRAM has high efficiency in terms of chip area and power usage compared to the required capacity. However, the processing speed of the existing on-chip memory inevitably slowed down significantly in the case of tasks that required more data quickly than the predetermined capacity of the L0 memory, and even when the need for global memory was not large, there was no way to utilize the remaining global memory. There was no use at all, resulting in inefficiency.

In contrast, the shared memory 2000 according to some embodiments of the present invention may be selectively controlled by one of two controllers, depending on the case. At this time, the shared memory 2000 is not controlled as a whole by only one of the two controllers, but can be independently controlled on a memory unit basis or a memory bank basis.

Through this, the shared memory 2000 according to this embodiment can obtain the optimal memory configuration ratio according to the computational task during run time and perform faster and more efficient computational tasks. In the case of processing units specialized in artificial intelligence, the required sizes of L0 memory and global memory may vary for each specific application. Furthermore, even for the same application, when using a deep learning network, the required sizes of L0 memory and global memory for each layer may vary. The shared memory 2000 according to this embodiment can enable fast and efficient deep learning work because the memory composition ratio can be changed during run time despite changes in the calculation steps for each layer.

FIG. 44 is a diagram for explaining the first memory bank of FIG. 43 in detail.

Although FIG. 44 illustrates the first memory bank 2110a, other memory banks may also have the same structure as the first memory bank 2110a.

Referring to FIG. 44, the first memory bank 2110a may include a cell array (Ca), a bank controller (Bc), a first path unit (P1), and a second path unit (P2).

The cell array (Ca) may include a plurality of memory elements (Cells) therein. The cell array Ca may have a plurality of memory elements arranged in a lattice structure. The cell array (Ca) may be, for example, a Static Random Access Memory (SRAM) cell array.

The bank controller (Bc) can control the cell array (Ca). The bank controller Bc may determine whether the cell array Ca will operate in an L0 memory format or a global memory format and control the cell array Ca accordingly.

Specifically, the bank controller (Bc) can determine whether to transmit and receive data in the first path unit (P1) direction or the second path unit (P2) direction during run time. The bank controller (Bc) can determine the direction of data transmission and reception according to the path control signal (Spc).

The path control signal (Spc) can be generated by a pre-designed device driver or compiler. A path control signal (Spc) can be generated according to the characteristics of the computational task. Alternatively, the path control signal (Spc) may be generated by input received from the user. In other words, the user can directly apply the input to the path control signal (Spc) in order to select the most optimal memory configuration ratio.

The bank controller (Bc) can determine a path for transmitting and receiving data stored in the cell array (Ca) through the path control signal (Spc). The data exchange interface may vary depending on how the bank controller (Bc) determines the path through which data is transmitted and received. That is, the bank controller Bc can use the first interface when exchanging data with the first path unit P1, and can use the second interface when exchanging data with the second path unit P2. At this time, the first interface and the second interface may be different from each other.

Additionally, the address system in which data is stored may also vary. That is, when a specific interface is selected, read and write operations can be performed using the corresponding address system.

The bank controller (Bc) can operate at a specific clock frequency. For example, when the cell array Ca is an SRAM cell array, the bank controller Bc can operate at a typical SRAM operating clock frequency.

The first path unit (P1) may be connected to the bank controller (Bc). The first path unit P1 may directly exchange data of the cell array Ca with the first processing unit 100a. At this time, “directly” may mean that they are exchanged without going through the global interconnection (6000). That is, the first processing unit 100a can directly exchange data with the first L0 memory 120a, and the first processing unit 100a can first process the first processing unit 100a when the shared memory 2000 is logically implemented in the L0 memory format. 1 Data can be exchanged through the path unit (P1). The first path unit P1 may include an L0 memory controller including the first L0 memory controller 122_1a and the second L0 memory controller 122_1b of FIG. 43 .

The first path unit (P1) may configure a multi-cycle sync path. That is, the operating clock frequency of the first path unit P1 may be the same as the operating clock frequency of the first processing unit 100a. The first L0 memory 120a can quickly exchange data at the same clock frequency as the operating clock frequency of the first processing unit 100a in order to quickly exchange data at the same speed as the operation of the first processing unit 100a. . The first path unit P1 may also operate at the same clock frequency as the operating clock frequency of the first processing unit 100a.

At this time, the operating clock frequency of the first path unit (P1) may be a multiple of the operating clock frequency of the bank controller (Bc). In this case, a separate CDC (Clock Domain Crossing) operation for clock synchronization between the bank controller (Bc) and the first path unit (P1) is not required, and accordingly, a delay in data transmission may not occur. there is. Accordingly, faster and more efficient data exchange may be possible.

In FIG. 44 , as an example, the operating clock frequency of the first path unit P1 may be 1.5 GHz. This may be twice the frequency of 750 MHz of the bank controller (Bc). However, this embodiment is not limited to this, and it may be possible as long as the first path unit (P1) operates at an integer multiple of the clock frequency of the bank controller (Bc).

The second path unit (P2) may be connected to the bank controller (Bc). The second path unit P2 may exchange data of the cell array Ca through the global interconnection 6000 instead of directly exchanging it with the first processing unit 100a. That is, the first processing unit 100a can exchange data with the cell array Ca through the global interconnection 6000 and the second path unit P2. At this time, the cell array Ca can exchange data not only with the first processing unit 100a but also with other neural cores.

That is, the second path unit P2 may be a data exchange path between the cell array Ca and all neural cores when the first memory bank 2110a is logically implemented in a global memory format. The second path unit P2 may include the global controller 2200 of FIG. 43.

The second path unit (P2) may configure an Async-Path. The operating clock frequency of the second path unit P2 may be the same as the operating clock frequency of the global interconnection 6000. The second path unit P2 may also operate at the same clock frequency as the operating clock frequency of the global interconnection 6000.

At this time, the operating clock frequency of the second path unit (P2) may not be synchronized with the operating clock frequency of the bank controller (Bc). In this case, a Clock Domain Crossing (CDC) operation may be required to synchronize clocks between the bank controller (Bc) and the second path unit (P2). If the operating clock frequency of the bank controller (Bc) and the operating clock frequency of the second path unit (P2) are not synchronized with each other, the degree of freedom in designing the clock domain may increase. Accordingly, the difficulty of hardware design is lowered and hardware operations can be derived more easily.

The bank controller Bc may use different address systems when exchanging data through the first path unit P1 and when exchanging data through the second path unit P2. That is, the bank controller Bc can use the first address system through the first path unit P1 and the second address system through the second path unit P2. At this time, the first address system and the second address system may be different from each other.

The bank controller (Bc) does not necessarily need to exist for each memory bank. In other words, the bank controller (Bc) is not a part for scheduling but serves to transmit signals, so it is not an essential part for each memory bank with two ports. Therefore, one bank controller (Bc) can control multiple memory banks. Multiple memory banks can operate independently although controlled by the bank controller (Bc). However, this embodiment is not limited to this.

Of course, a bank controller (Bc) may exist for each memory bank. In this case, the bank controller Bc can individually control each memory bank.

Referring to FIGS. 43 and 44, when the first memory unit 210a exchanges data through the first path unit (P1), the first address system is used and data is exchanged through the second path unit (P2). In case of exchange, a second address system can be used. Similarly, when the second memory unit 210b exchanges data through the first path unit P1, a third address system is used, and when data is exchanged through the second path unit P2, the second address system is used. You can use the system. At this time, the first address system and the third address system may be the same. However, this embodiment is not limited to this.

The first address system and the third address system may be used exclusively for the first processing unit 100a and the second processing unit 100b, respectively. The second address system may be commonly applied to the first processing unit 100a and the second processing unit 100b.

In Figure 44, as an example, the operating clock frequency of the second path unit (P2) may operate at 1 GHz. This may be a frequency that is not synchronized with the 750 MHz operating clock frequency of the bank controller (Bc). That is, the operating clock frequency of the second path unit (P2) can be freely set without being at all dependent on the operating clock frequency of the bank controller (Bc).

Typical global memory uses slow SRAM (e.g., 750 MHz) and faster global interconnection (e.g., 1 GHz), which inevitably causes delays due to CDC operations. In contrast, the shared memory 2000 according to some embodiments of the present invention has room to use the first path unit (P1) in addition to the second path unit (P2), thereby avoiding delays due to the CDC task.

In addition, since a typical global memory uses a single global interconnection 6000 for multiple neural cores, a decrease in overall processing speed can easily occur when data transmission occurs simultaneously. On the other hand, the shared memory 2000 according to some embodiments of the present invention has room to use the first path unit (P1) in addition to the second path unit (P2), thereby appropriately reducing the data processing load on the global controller 2200. A dispersing effect can also be achieved.

Figure 45 is a block diagram illustrating the software hierarchy of a neural processing device according to some embodiments of the present invention.

Referring to FIG. 45, the software layer structure of a neural processing device according to some embodiments of the present invention may include a DL framework 10000, a compiler stack 20000, and a backend module 30000.

The DL framework (10000) may refer to a framework for a deep learning model network used by users. For example, a fully trained neural network can be created using programs such as TensorFlow or PyTorch.

The compiler stack 20000 may include an adaptation layer 21000, a compute library 22000, a front-end compiler 23000, a back-end compiler 24000, and a runtime driver 25000.

The adaptation layer (21000) may be a layer in contact with the DL framework (10000). The adaptation layer (21000) can quantize the user's neural network model created in the DL framework (10000) and modify the graph. Additionally, the adaptation layer 21000 can convert the model type into a required type.

The front-end compiler (23000) can convert various neural network models and graphs received from the adaptation layer (21000) into a certain intermediate representation (IR). The converted IR may be a preset expression that is easy to handle later in the backend compiler 24000.

The IR of the front-end compiler 23000 can be optimized in advance at the graph level. Additionally, the front-end compiler 23000 can ultimately generate the IR by converting it into a hardware-optimized layout.

The backend compiler (24000) optimizes the IR converted from the frontend compiler (23000) and converts it into a binary file so that the runtime driver can use it. The back-end compiler (24000) can generate optimized code by dividing the job at a scale that matches the details of the hardware.

The compute library 22000 can store template operations designed in a form suitable for hardware among various operations. The compute library 22000 provides several template operations that require hardware to the backend compiler 24000, allowing optimized code to be generated.

The runtime driver 25000 may continuously perform monitoring while driving to drive the neural network device according to some embodiments of the present invention. Specifically, it may be responsible for executing the interface of the neural network device.

The backend module 30000 may include an application specific integrated circuit (ASIC) 31000, a field programmable gate array (FPGA) 32000, and a C-model (33000). ASIC (31000) may refer to a hardware chip determined according to a predetermined design method. FPGA 32000 may be a programmable hardware chip. C-model (33000) may refer to a model implemented by simulating hardware in software.

The backend module 30000 can perform various tasks and derive results using binary code generated through the compiler stack 20000.

Figure 46 is a conceptual diagram to explain a deep learning operation performed by a neural processing device according to some embodiments of the present invention.

Referring to FIG. 46, the artificial neural network model 40000 is an example of a machine learning model, and in machine learning technology and cognitive science, a statistical learning algorithm implemented based on the structure of a biological neural network or an algorithm thereof. It is a structure that executes .

In the artificial neural network model (40000), as in a biological neural network, nodes, which are artificial neurons that form a network through the combination of synapses, repeatedly adjust the weight of the synapse, creating a gap between the correct output corresponding to a specific input and the inferred output. By learning to reduce the error of , a machine learning model with problem-solving capabilities can be expressed. For example, the artificial neural network model 40000 may include random probability models, neural network models, etc. used in artificial intelligence learning methods such as machine learning and deep learning.

A neural processing device according to some embodiments of the present invention may perform calculations by implementing this type of artificial neural network model (40000). For example, the artificial neural network model 40000 may receive an input image and output information about at least a portion of the object included in the input image.

The artificial neural network model (40000) is implemented as a multilayer perceptron (MLP) consisting of multiple layers of nodes and connections between them. The artificial neural network model 40000 according to this embodiment can be implemented using one of various artificial neural network model structures including MLP. As shown in FIG. 46, the artificial neural network model 40000 includes an input layer 41000 that receives an input signal or data 40100 from the outside, and an output layer that outputs an output signal or data 40200 corresponding to the input data. (44000), located between the input layer 41000 and the output layer 44000, receives n signals from the input layer 41000, extracts the characteristics, and transmits them to the output layer 44000 (where n is a positive integer). It consists of a hidden layer (42000 to 43000). Here, the output layer 44000 receives signals from the hidden layers 42000 to 43000 and outputs them to the outside.

The learning methods of the artificial neural network model (40000) include supervised learning, which learns to optimize problem solving by inputting teacher signals (correct answers), and unsupervised learning, which does not require teacher signals. ) There is a way.

The neural processing device can directly generate learning data for training the artificial neural network model (40000) through simulation. In this way, a plurality of input variables and a plurality of output variables corresponding to the input layer 41000 and the output layer 44000 of the artificial neural network model 40000 are matched, respectively, and the input layer 41000, hidden layers 42000 to 43000, and By adjusting the synapse values between nodes included in the output layer 44000, learning can be done so that the correct output corresponding to a specific input can be extracted. Through this learning process, the characteristics hidden in the input variables of the artificial neural network model (40000) can be identified, and the nodes of the artificial neural network model (40000) can be used to reduce the error between the output variables calculated based on the input variables and the target output. You can adjust the synapse value (or weight) between them.

Figure 47 is a conceptual diagram for explaining learning and inference operations of a neural network of a neural processing device according to some embodiments of the present invention.

Referring to FIG. 47, in the training phase, a number of learning data (TD) may be forwarded to the artificial neural network model (NN) and then forwarded back again. Through this, the weights and biases of each node of the artificial neural network model (NN) are adjusted, and through this, learning can be performed to produce increasingly accurate results. Through this learning process (Training Phase), the artificial neural network model (NN) can be converted into a learned neural network model (NN_T).

In the inference phase, new data (ND) can be input back into the learned neural network model (NN_T). The learned neural network model (NN_T) can take new data (ND) as input and derive result data (RD) through already learned weights and biases. This result data (RD) may be important in terms of which learning materials (TD) were used in the training process (Training Phase) and how much learning materials (TD) were used.

Hereinafter, with reference to FIGS. 48 to 50, a method of setting task dependency of a neural processing device according to some embodiments of the present invention will be described. Parts that overlap with the above-described embodiments are omitted or simplified. For description of the task dependency setting method according to this embodiment, FIGS. 1 to 47 and the above-described related description may be referred to.

FIG. 48 is a flowchart for explaining a method of setting task dependencies of a neural processing device according to some embodiments of the present invention, and FIG. 49 is a flowchart for explaining in detail the step of receiving the dependency set code of FIG. 48. This is a flowchart. FIG. 50 is a flowchart illustrating in detail the step of transmitting the task descriptor of FIG. 48.

Referring to FIG. 48, a dependency set code is received (S100).

In detail, referring to FIG. 49, the task descriptor is checked out (S110). next,

Specifically, referring to FIG. 19, the dependency setters (DPs) may also check out each of the first to nth task descriptors (Tsk_d1 to Tsk_dn) according to the completion signal. Accordingly, dependency setters (DPs) can generate a checkout report (COrp) about which tasks have been completed and checked out.

Again, referring to FIG. 49, a dependency set code is generated (S120).

Specifically, referring to FIGS. 17 and 19, dependency setters (DPs) may receive a completion signal from the check-in buffer (Cib) and generate a dependency update request (DFURQ). That is, dependency setters (DPs) can generate a dependency update request (DFURQ) depending on which task corresponding to the task descriptor has been completed.

At this time, the dependency update request (DFURQ) may include content about the above-described dependency set code. That is, the dependency update request (DFURQ) may include dependency information of the task descriptor that is currently checked out. That is, the dependency update request (DFURQ) may include information about which task descriptor can be executed as the dependency limit is removed by the task descriptor currently checked out.

The dependency checker (DPc) can generate dependency set codes (DFURQ1 to DFURQ4) through a dependency update request (DFURQ).

Again, referring to FIG. 49, the dependency set code is transmitted (S130).

Specifically, referring to FIG. 19 , dependency setters (DPs) may transmit a dependency update request (DFURQ) to the task passage 620.

Again, referring to FIG. 49, the dependency set code is received (S140).

Specifically, referring to FIG. 17, the dependency checker (DPc) may receive a dependency update request (DFURQ). A dependency update request (DFURQ) can notify changes in dependencies as completed tasks occur according to defined dependencies between specific tasks.

Again, referring to FIG. 48, the dependency matrix is stored according to the dependency set code (S200).

Specifically, referring to FIG. 18, each column of the first dependency matrix (DPmx1) contains the first to fourth dependency set codes (DFURQ1 to DFURQ4) generated by the dependency update request (DFURQ). It can be updated through

Again, referring to FIG. 48, it is determined whether the dependency check code and the corresponding row of the dependency matrix are the same (S300). If the same, the corresponding row of the dependency matrix is cleared (S400) and the task descriptor is transmitted (S500). If not, wait for steps S100 and S200 again.

In detail, referring to FIG. 50, a dummy check descriptor and a task descriptor are identified (S510). Next, the dummy check descriptor is burned (S520)

Specifically, referring to FIG. 32, the dependency checker (DPc) can identify a task descriptor and a dummy check descriptor among received descriptors. Additionally, dummy set descriptors and dummy check descriptors can also be identified. At this time, the task descriptor and the dummy set descriptor must be checked out by passing the dependency checker (DPc), but the dummy check descriptor does not need to be checked out and can be identified.

The second dummy check descriptor (DC_Tsk_d2) may be burned out within the dependency checker (DPc) when the dependency check is completed. That is, since the second dummy check descriptor (DC_Tsk_d2) does not need to be transmitted to the core global 500 or checked out, it can be burned by the dependency checker (DPc).

The above description is merely an illustrative explanation of the technical idea of the present embodiment, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these examples. The scope of protection of this embodiment should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this embodiment.

Claims (20)

a task buffer receiving a first task of a different first type and a second task of a second type;
A 1_1 queue that receives a first task descriptor for the first task from the task buffer;
A 1_2 queue that receives a second task descriptor for the second task from the task buffer;
a dependency checker that checks dependencies of the first and second task descriptors;
A 2_1 queue that receives the first task descriptor from the dependency checker; and
It includes a 2_2 queue that receives the second task descriptor from the dependency checker,
The dependency checker includes a plurality of dependency matrices,
Each of the plurality of dependency matrices is:
First and second rows corresponding to the first and second types, respectively,
Comprising first and second rows corresponding to the first and second types, respectively,
Task manager.
According to claim 1,
The plurality of dependency matrices are:
With the first group,
comprising a second group,
The first group processes the first task descriptor,
The second group processes a third task descriptor, and the third task descriptor is a descriptor for a third task of the first type,
Task manager.
According to claim 1,
The first and second tasks are each dependent on the third task,
The dependency checker receives a third task descriptor and a third dummy set descriptor for the third task,
The third task descriptor and the third dummy set descriptor are allocated to the dependency matrices of different groups,
Task manager.
According to clause 3,
A task passage including the task buffer, the 1_1 queue, the 1_2 queue, the dependency checker, the 2_1 queue, and the 2_2 queue,
Containing a dun passage that checks out the first to third task descriptors and the third dummy set descriptor,
Task manager.
According to clause 4,
The task passage transmits the first to third task descriptors to the core global,
Task manager.
According to clause 4,
Further comprising a void core that receives the third dummy set descriptor from the task passage and transmits the dummy set descriptor to the dune passage,
Task manager.
According to clause 6,
The void core transmits the dummy set descriptor to the dune passage,
Task manager.
According to clause 4,
The fourth task is dependent on the fifth and sixth tasks, respectively,
The dependency checker receives a fourth task descriptor and a fourth dummy check descriptor for the fourth task,
The fourth task descriptor and the fourth dummy check descriptor are allocated to dependency matrices of different groups,
Task manager.
According to clause 8,
The fourth dummy check descriptor is incinerated within the dependency checker,
Task manager.
at least one neural processor;
a shared memory shared by the at least one neural processor;
a global interconnection through which the at least one neural processor and the shared memory exchange data;
The at least one neural processor each includes a task manager,
The task manager receives a plurality of descriptors, checks dependencies between the plurality of descriptors, and transmits them to the neural core,
The plurality of descriptors include a task descriptor, a dummy check descriptor, and a dummy set descriptor,
The dummy check descriptor is burned inside the task manager,
Neural processing device.
According to claim 10,
When there are two or more subsequent task descriptors dependent on the task descriptor, the dummy set descriptor is received one less than the number of subsequent task descriptors.
Neural processing device.
According to claim 10,
When two or more of the task descriptors depend on one subsequent task descriptor, one less dummy check descriptor is received than the number of task descriptors.
Neural processing device.
According to claim 10,
The task manager is,
A task passage that receives the plurality of descriptors and transmits them to the neural core according to dependency,
Comprising a Dawn passage that receives the task descriptor and the dummy set descriptor from the neural core and reports completion to the command processor,
Neural processing device.
According to claim 13,
The completion status includes whether the task descriptor is completed and does not include whether the dummy set descriptor is completed.
Neural processing device.
According to claim 14,
The Dawn passage transmits a dependency field update request to the task passage when the task descriptor and the dummy set descriptor are checked out.
Neural processing device.
According to claim 15,
The task passage includes a dependency checker that receives the dependency field update request and checks dependencies between the plurality of descriptors,
Neural processing device.
According to claim 16,
The dependency checker includes a plurality of groups of dependency matrices,
Neural processing device.
Receive a dependency set code,
Store the dependency matrix according to the dependency set code,
Compare the dependency check code of the task descriptor with the corresponding row of the dependency matrix,
If the dependency check code and the corresponding row are the same, the corresponding row is cleared,
Including transmitting the task descriptor,
How to handle task dependency in a neural processing device.
According to clause 18,
Receiving the dependency set code is,
Check out the task descriptor and dummy set descriptor,
Including generating the dependency set code for the task descriptor,
How to handle task dependency in a neural processing device.
According to clause 18,
Transmitting the task descriptor involves:
Identify the dummy check descriptor and task descriptor,
Including burning the dummy check descriptor,
How to handle task dependency in a neural processing device.

Images