KR102509472B1 - Neural processing device and Method for using shared page table thereof - Google Patents

Abstract

Disclosed are a neural processing device and a method for using a shared page table thereof. The neural processing device comprises: one or more neural processors; a shared memory shared by the one or more neural processors; and a global interconnection exchanging data between the one or more neural processors and the shared memory. Also included are: a processing unit included in the neural processors, and providing a logical address; a memory management unit receiving the logical address and converting the logical address into a physical address; and a physical memory which can be accessed through the physical address. The memory management unit includes a shared page table having the conversion information between the logical address and the physical address, and shared by one or more processes. Therefore, storage efficiency of a physical memory can be increased.

Description

Neural processing device and method for using shared page table thereof

The present invention relates to a neural processing apparatus and a method of using the shared page table. Specifically, the present invention relates to a neural processing apparatus that increases a hit rate of a conversion lookaside buffer by sharing a page table in a multi-process environment and a method of using the same page table.

In the past few years, artificial intelligence (AI) technology has been discussed as the most promising technology worldwide as a core technology of the 4th industrial revolution. The biggest problem with these artificial intelligence technologies is computing power. The most important thing in artificial intelligence technology that realizes human learning, reasoning, perception, and natural language implementation is to process a lot of data quickly.

Although the central processing unit (CPU) or graphics processing unit (GPU) of an off-the-shelf computer was used for deep learning learning and reasoning of early artificial intelligence, deep learning learning and reasoning tasks with high workloads were used. has limitations, so a neural processing unit (NPU) structurally specialized for deep learning tasks is in the limelight.

Since a general multi-process environment accesses separate memory areas, logical address and physical address conversion can be performed through different page tables. However, if page tables are shared by multiple processes, efficiency can be increased by continuing to use the same page tables even when processes change.

Registered Patent Publication No. 10-2258566

An object of the present invention is to provide a neural processing apparatus with improved efficiency through a shared page table.

Another object of the present invention is to provide a method of using a shared page table in a neural processing apparatus with improved efficiency through the shared page table.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention not mentioned above can be understood by the following description and will be more clearly understood by the examples of the present invention. It will also be readily apparent that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations indicated in the claims.

In order to solve the above problem, a neural processing apparatus according to some embodiments of the present invention provides at least one neural processor, a shared memory shared by the at least one neural processor, and between the at least one neural processor and the shared memory. A neural processing apparatus including a global interconnection for exchanging data, comprising: a processing unit included in the neural processor to provide a logical address; a memory management unit to receive and convert the logical address into a physical address; and the physical address and a physical memory accessible to , and the memory management unit includes a shared page table that has the logical address and conversion information of the physical address and is shared by at least one process.

Also, the shared page table may include a page table entry (PTE) recording a start position of the physical memory corresponding to the logical address and mapping type data recording a mapping type of the logical address.

Also, the mapping type may include any one of a shared type shared by the at least one process and a private type not shared by the at least one process.

Also, in the shared logical address, all of the at least one process may access the same shared physical page in the physical memory.

Also, context data of a context to which the at least one process belongs may be stored in the shared physical page.

Also, in the private type logical address, each of the at least one process may access a different private physical page within the physical memory.

Also, the private physical pages may be sequentially arranged.

In addition, the private physical page may store input/output data of each of the at least one process.

In addition, the at least one process includes first to fourth processes, the first and second processes operate concurrently with each other, the third and fourth processes operate concurrently with each other, and the first and second processes operate concurrently with each other. process, and the third and fourth processes may not operate concurrently with each other.

Also, the mapping type may further include an intermediate type shared by some of the at least one process.

In addition, the intermediate type logical address is such that the first and third processes access the same first intermediate physical page in the physical memory, and the second and fourth processes access the same intermediate physical page in the physical memory. 2 Intermediate physical pages can be accessed.

Also, the first and second intermediate physical pages may be continuously arranged.

In addition, the memory management unit may further include a conversion lookaside buffer for storing a physical address for a recently used logical address.

In addition, the translation lookaside buffer converts the stored logical address into the physical address when a logical address identical to the stored logical address is received, and when a logical address not identical to the stored logical address is received, the shared page table is converted. Referring to the received logical address, it can be converted into a physical address.

The neural processor may include at least one neural core, and the neural core may transmit and receive data to and from the processing unit, an L0 memory, and load data into the L0 memory, and load data into the L0 memory. may include a load store unit that transmits data from

Also, the load store unit may include the conversion lookaside buffer.

In order to solve the another problem, a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention may access a physical memory by referring to a shared page table in a first process, and access the first process in the first process. If there is process switching to two processes, the second process accesses the physical memory by referring to the shared page table.

Also, accessing the physical memory may include checking a PTE and a mapping type from the shared page table, and accessing a physical page according to the PTE when the mapping type is a shared type.

Further, accessing the physical memory may include checking a PTE and a mapping type from the shared page table and, when the mapping type is a private type, correcting a process ID in the PTE with an offset to access the physical page. can

In addition, accessing the physical memory checks the PTE and mapping type from the shared page table, and when the mapping type is an intermediate type, calculates a final offset according to a process ID, and assigns the final offset to the PTE. It may include accessing the physical page by correcting with .

The neural processing apparatus and the method of using the shared page table according to the present invention can increase the storage efficiency of the physical memory by not individually storing data commonly used in each process.

Also, when process switching occurs, there is no need to replace the page table, so that the hit rate of the translation lookaside buffer can be kept high without being reduced.

In addition to the above description, specific effects of the present invention will be described together while explaining specific details for carrying out the present invention.

1 is a block diagram illustrating a neural processing system according to some embodiments of the present invention.
FIG. 2 is a block diagram for explaining the neural processing apparatus of FIG. 1 in detail.
FIG. 3 is a block diagram illustrating the neural core SoC of FIG. 2 in detail.
FIG. 4 is a structural diagram for explaining the global interconnection of FIG. 3 in detail.
FIG. 5 is a block diagram for explaining the neural processor of FIG. 3 in detail.
6 is a diagram for explaining a hierarchical structure of a neural processing apparatus according to some embodiments of the present invention.
FIG. 7 is a block diagram for explaining the neural core of FIG. 5 in detail.
8 is a block diagram for explaining the LSU of FIG. 7 in detail.
FIG. 9 is a block diagram for explaining the processing unit of FIG. 7 in detail.
FIG. 10 is a block diagram for explaining the L0 memory of FIG. 7 in detail.
FIG. 11 is a block diagram for explaining the local memory bank of FIG. 10 in detail.
FIG. 12 is a block diagram for explaining the structure of the neural processing apparatus of FIG. 1 in detail.
FIG. 13 is a block diagram illustrating memory reconfiguration of the neural processing system of FIG. 1 .
14 is a block diagram illustrating an example of memory reorganization of the neural processing system of FIG. 1;
FIG. 15 is an enlarged block diagram of part A of FIG. 13 .
FIG. 16 is a diagram for explaining the first memory bank of FIG. 15 in detail.
FIG. 17 is a block diagram for explaining a software layer structure of the neural processing apparatus of FIG. 1 .
18 is a block diagram for explaining multi-processes of a neural processing system according to some embodiments of the present invention.
19 is a block diagram illustrating address conversion of a neural processing system according to some embodiments of the present invention.
FIG. 20 is a block diagram illustrating allocation of a physical page of the physical memory of FIG. 19 .
21 is a block diagram illustrating allocation of a physical page of a physical memory of a neural processing system according to some embodiments of the present invention.
22 is a block diagram illustrating allocation of a physical page of a physical memory of a neural processing system according to some embodiments of the present invention.
FIG. 23 is a conceptual diagram for explaining mapping type data of FIG. 19 .
FIG. 24 is a conceptual diagram for explaining a deep learning operation performed by the neural processing apparatus of FIG. 1 .
FIG. 25 is a conceptual diagram for explaining learning and reasoning operations of a neural network of the neural processing apparatus of FIG. 1 .
26 is a flowchart illustrating a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention.
FIG. 27 is a flowchart for explaining the step of accessing the physical memory of FIG. 26 .
28 is a flowchart illustrating a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention.
29 is a flowchart illustrating a method of allocating a physical page in a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention.
30 is a flowchart illustrating a method of allocating a physical page in a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention.

Terms or words used in this specification and claims should not be construed as being limited to a general or dictionary meaning. According to the principle that an inventor may define a term or a concept of a word in order to best describe his/her invention, it should be interpreted as meaning and concept consistent with the technical spirit of the present invention. In addition, the embodiments described in this specification and the configurations shown in the drawings are only one embodiment in which the present invention is realized, and do not represent all of the technical spirit of the present invention, so they can be replaced at the time of the present application. It should be understood that there may be many equivalents and variations and applicable examples.

Terms such as first, second, A, and B used in this specification and claims may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The term 'and/or' includes a combination of a plurality of related recited items or any one of a plurality of related recited items.

Terms used in this specification and claims are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. It should be understood that terms such as "include" or "having" in this application do not exclude in advance the possibility of existence or addition of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. .

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this application, it should not be interpreted in an ideal or excessively formal meaning. don't

In addition, each configuration, process, process or method included in each embodiment of the present invention may be shared within a range that does not contradict each other technically.

Hereinafter, a neural processing apparatus according to some embodiments of the present invention will be described with reference to FIGS. 1 to 25 .

1 is a block diagram illustrating a neural processing system according to some embodiments of the present invention.

Referring to FIG. 1 , a neural processing system (NPS) according to some embodiments of the present invention may include a first neural processing device 1, a second neural processing device 2, and an external interface 3. .

The first neural processing device 1 may be a device that performs calculations using an artificial neural network. The first neural processing device 1 may be, for example, a device specialized for performing a deep learning calculation task. However, this embodiment is not limited thereto.

The second neural processing device 2 may have a configuration identical to or similar to that of the first neural processing device 1 . The first neural processing device 1 and the second neural processing device 2 may be connected to each other through an external interface 3 to share data and control signals.

Although FIG. 1 shows two neural processing devices, a neural processing system (NPS) according to some embodiments of the present invention is not limited thereto. That is, in the neural processing system (NPS) according to some embodiments of the present invention, three or more neural processing devices may be connected to each other through the external interface 3. Also, conversely, a neural processing system (NPS) according to some embodiments of the present invention may include only one neural processing device.

In this case, each of the first neural processing device 1 and the second neural processing device 2 may be a processing device other than the neural processing device. That is, the first neural processing unit 1 and the second neural processing unit 2 are each a graphics processing unit (GPU), a central processing unit (CPU), and other types of processing units. It may be. Hereinafter, for convenience, the first neural processing device 1 and the second neural processing device 2 will be described as neural processing devices.

FIG. 2 is a block diagram for explaining the neural processing apparatus of FIG. 1 in detail.

Referring to FIG. 2 , the first neural processing device 1 includes a neural core SoC 10, a CPU 20, an off-chip memory 30, a first non-volatile memory interface 40, and a first volatile memory interface ( 50), a second non-volatile memory interface 60 and a second volatile memory interface 70.

The neural core SoC 10 may be a system on chip device. The neural core SoC 10 is an artificial intelligence computing unit and may be an accelerator. The neural core SoC 10 may be, for example, any one of a graphics processing unit (GPU), a field programmable gate array (FPGA), and an application-specific integrated circuit (ASIC). However, this embodiment is not limited thereto.

The neural core SoC 10 may exchange data with other external computing units through the external interface 3 . In addition, the neural core SoC 10 may be connected to the non-volatile memory 31 and the volatile memory 32 through the first non-volatile memory interface 40 and the first volatile memory interface 50 , respectively.

The CPU 20 may be a controller that controls the system of the first neural processing device 1 and executes program operations. The CPU 20 is a general-purpose computational unit and may have low efficiency to perform parallel simple computations widely used in deep learning. Accordingly, the neural core SoC 10 may perform calculations for deep learning reasoning and learning tasks, thereby achieving high efficiency.

The CPU 20 may exchange data with other external computing units through the external interface 3 . In addition, the CPU 20 may be connected to the non-volatile memory 31 and the volatile memory 32 through the second non-volatile memory interface 60 and the second volatile memory interface 70, respectively.

The off-chip memory 30 may be a memory disposed outside a chip of the neural core SoC 10 . The off-chip memory 30 may include a non-volatile memory 31 and a volatile memory 32 .

The non-volatile memory 31 may be a memory that continuously retains stored information even when power is not supplied. The non-volatile memory 31 includes, for example, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable alterable ROM (EAROM), an erasable programmable read-only memory (EPROM), and an electrically erasable programmable memory (EEPROM). Read-Only Memory) (e.g., NAND Flash memory, NOR Flash memory), Ultra-Violet Erasable Programmable Read-Only Memory (UVEPROM), Ferroelectric Random Access Memory (FeRAM), MRAM (Magnetoresistive Random Access Memory), PRAM (Phase-change Random Access Memory), SONOS (silicon-oxide-nitride-oxide-silicon), RRAM (Resistive Random Access Memory), NRAM (Nanotube Random Access Memory), magnetic computer memory It may include at least one of a device (eg, hard disk, diskette drive, magnetic tape), an optical disk drive, and a 3D XPoint memory. However, this embodiment is not limited thereto.

Unlike the non-volatile memory 31, the volatile memory 32 may be a memory that continuously requires power to maintain stored information. The volatile memory 32 may include, for example, at least one of Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Synchronous Dynamic Random Access Memory (SDRAM), and Double Data Rate SDRAM (DDR SDRAM). there is. However, this embodiment is not limited thereto.

The first non-volatile memory interface 40 and the second non-volatile memory interface 60 may be, for example, PATA (Parallel Advanced Technology Attachment), SCSI (Small Computer System Interface), SAS (Serial Attached SCSI), SATA ( Serial Advanced Technology Attachment) and PCI Express (PCIe). However, this embodiment is not limited thereto.

The first volatile memory interface 50 and the second volatile memory interface 70 are respectively, for example, SDR (Single Data Rate), DDR (Double Data Rate), QDR (Quad Data Rate), and XDR (eXtreme Data Rate) , Octal Data Rate). However, this embodiment is not limited thereto.

FIG. 3 is a block diagram illustrating the neural core SoC of FIG. 2 in detail.

2 and 3, the neural core SoC 10 includes at least one neural processor 1000, a shared memory 2000, a direct memory access (DMA) 3000, a non-volatile memory controller 4000, a volatile A memory controller 5000 and a global interconnection 5000 may be included.

The neural processor 1000 may be an arithmetic unit that directly performs a arithmetic task. When there are a plurality of neural processors 1000 , calculation tasks may be allocated to each of the neural processors 1000 . Each of the neural processors 1000 may be connected to each other through the global interconnection 5000 .

The shared memory 2000 may be a memory shared by several neural processors 1000 . The shared memory 2000 may store data of each neural processor 1000 . In addition, the shared memory 2000 may receive data from the off-chip memory 30, temporarily store the data, and transfer the data to each neural processor 1000. Conversely, the shared memory 2000 may receive data from the neural processor 1000, temporarily store the data, and transfer the data to the off-chip memory 30 of FIG. 2 .

The shared memory 2000 may require a relatively fast memory. Accordingly, the shared memory 2000 may include SRAM, for example. However, this embodiment is not limited thereto. That is, the shared memory 2000 may include DRAM.

The shared memory 2000 may be a memory corresponding to an SoC level, that is, level 3 (L3). Accordingly, the shared memory 2000 may be defined as an L3 shared memory.

The DMA 3000 can directly control the movement of data without the need for the neural processor 1000 to control input/output of data. Accordingly, the number of interrupts of the neural processor 1000 can be minimized by the DMA 3000 controlling data movement between memories.

The DMA 3000 may control movement of data between the shared memory 2000 and the off-chip memory 30 . The non-volatile memory controller 4000 and the volatile memory controller 5000 may transfer data through the authority of the DMA 3000 .

The nonvolatile memory controller 4000 may control a read or write operation of the nonvolatile memory 31 . The nonvolatile memory controller 4000 may control the nonvolatile memory 31 through the first nonvolatile memory interface 40 .

The volatile memory controller 5000 may control a read or write operation of the volatile memory 32 . Also, the volatile memory controller 5000 may perform a refresh operation of the volatile memory 32 . The volatile memory controller 5000 may control the non-volatile memory 31 through the first volatile memory interface 50 .

The global interconnection 5000 may connect at least one of the neural processor 1000 , the shared memory 2000 , the DMA 3000 , the non-volatile memory controller 4000 and the volatile memory controller 5000 to each other. In addition, the external interface 3 may also be connected to the global interconnection 5000 . The global interconnection 5000 includes at least one neural processor 1000, a shared memory 2000, a DMA 3000, a nonvolatile memory controller 4000, a volatile memory controller 5000, and data between the external interface 3. may be a moving path.

The global interconnection 5000 may transmit signals for transmission and synchronization of control signals as well as data. That is, in the neural processing apparatus according to some embodiments of the present invention, each neural processor 1000 may directly transmit and receive a synchronization signal, rather than a separate control processor managing synchronization signals. Accordingly, the latency of the synchronization signal generated by the control processor can be blocked.

That is, when there are a plurality of neural processors 1000, there may be dependencies of individual tasks in which the task of one neural processor 1000 must be completed before the next neural processor 1000 can start a new task. The end and start of these individual tasks can be confirmed through a synchronization signal, and in the conventional technology, the control processor performs the reception of the synchronization signal and the instruction to start a new task.

However, as the number of neural processors 1000 increases and the dependencies of tasks are designed more complexly, the number of requests and instructions for synchronization tasks increases exponentially. Therefore, the latency according to each request and instruction can greatly reduce work efficiency.

Accordingly, in the neural processing apparatus according to some embodiments of the present invention, each neural processor 1000 may directly transmit a synchronization signal to other neural processors 1000 according to task dependencies instead of a control processor. In this case, compared to the method managed by the control processor, several neural processors 1000 can perform synchronization tasks in parallel, so that latency due to synchronization can be minimized.

In addition, the control processor needs to perform task scheduling of the neural processors 1000 according to task dependencies, and the overhead of such scheduling can greatly increase as the number of the neural processors 1000 increases. Therefore, in the neural processing device according to some embodiments of the present invention, the scheduling task is also performed by the individual neural processor 1000, and thus the performance of the device can be improved without the scheduling burden.

FIG. 4 is a structural diagram for explaining the global interconnection of FIG. 3 in detail.

Referring to FIG. 4 , a global interconnection 5000 may include a data channel 5100, a control channel 5200, and an L3 sync channel 5300.

The data channel 5100 may be a dedicated channel for transmitting data. At least one of the neural processor 1000, the shared memory 2000, the DMA 3000, the non-volatile memory controller 4000, the volatile memory controller 5000, and the external interface 3 transmit data to each other through the data channel 5100. can be exchanged.

The control channel 5200 may be a dedicated channel for transmitting a control signal. At least one of the neural processor 1000, the shared memory 2000, the DMA 3000, the non-volatile memory controller 4000, the volatile memory controller 5000, and the external interface 3 control each other through the control channel 5200 signals can be exchanged.

The L3 sync channel 5300 may be a dedicated channel for transmitting a synchronization signal. At least one neural processor 1000, shared memory 2000, DMA 3000, non-volatile memory controller 4000, volatile memory controller 5000, and the external interface 3 communicate with each other through the L3 sync channel 5300. Synchronization signals can be exchanged.

The L3 sync channel 5300 is set as a dedicated channel within the global interconnection 5000 and can quickly transmit a synchronization signal without overlapping with other channels. Accordingly, the neural processing apparatus according to some embodiments of the present invention does not require a new wiring work and can smoothly perform a synchronization task using the existing global interconnection 5000 .

FIG. 5 is a block diagram for explaining the neural processor of FIG. 3 in detail.

3 to 5 , the neural processor 1000 may include at least one neural core 100, an L2 shared memory 400, a local interconnection 200, and an L2 sync path 300.

At least one neural core 100 may divide and perform tasks of the neural processor 1000 . The number of neural cores 100 may be, for example, 8. However, this embodiment is not limited thereto. 3 and 5 show that several neural cores 100 are included in the neural processor 1000, but the present embodiment is not limited thereto. That is, the neural processor 1000 may be configured with only one neural core 100 .

The L2 shared memory 400 may be a memory shared by each of the neural cores 100 in the neural processor 1000 . The L2 shared memory 400 may store data of each neural core 100 . In addition, the L2 shared memory 400 may receive data from the shared memory 2000 of FIG. 4 , temporarily store the data, and transfer the data to each neural core 100 . Conversely, the L2 shared memory 400 may receive data from the neural core 100, temporarily store the data, and transfer the data to the shared memory 2000 of FIG. 3 .

The L2 shared memory 400 may be memory corresponding to a neural processor level, that is, level 2 (L2). The L3 shared memory, that is, the shared memory 2000 may be shared by the neural processor 1000 and the L2 shared memory 400 may be shared by the neural core 100 .

The local interconnection 200 may connect at least one neural core 100 and the L2 shared memory 400 to each other. The local interconnection 200 may be a path through which data moves between at least one neural core 100 and the L2 shared memory 400 . The local interconnection 200 may be connected to the global interconnection 5000 of FIG. 3 to transmit data.

The L2 sync pass 300 may connect at least one neural core 100 and the L2 shared memory 400 to each other. The L2 sync path 300 may be a path along which synchronization signals of at least one neural core 100 and the L2 shared memory 400 move.

The L2 sync path 300 may be formed physically separately from the local interconnection 200 . In the case of the local interconnection 200, unlike the global interconnection 5000, sufficient internal channels may not be formed. In this case, the L2 sync path 300 is formed separately so that synchronization signal transmission can be performed quickly and without delay. The L2 sync pass 300 may be used for synchronization performed at a level lower than that of the L3 sync channel 5300 of the global interconnection 5000.

6 is a diagram for explaining a hierarchical structure of a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 6 , the neural core SoC 10 may include at least one neural processor 1000 . Each of the neural processors 1000 may transmit data to each other through the global interconnection 5000 .

Each of the neural processors 1000 may include at least one neural core 100 . The neural core 100 may be a processing unit optimized for deep learning calculation tasks. The neural core 100 may be a processing unit corresponding to one operation of a deep learning operation task. That is, a deep learning computation task can be expressed as a sequential or parallel combination of several operations. The neural core 100 is a processing unit capable of processing one operation, and may be a minimum operation unit that can be considered for scheduling from the point of view of a compiler.

In the neural processing apparatus according to the present embodiment, the scale of the minimum operation unit considered in terms of compiler scheduling and the hardware processing unit unit are configured to be the same, so that fast and efficient scheduling and operation execution can be promoted.

That is, if the divisible processing unit of the hardware is too large compared to the calculation task, inefficiency of the calculation task may occur in driving the processing unit. Conversely, it is not appropriate to schedule each time a processing unit smaller than the operation, which is the minimum scheduling unit of the compiler, because scheduling inefficiency may occur and hardware design cost may increase.

Therefore, according to the present embodiment, scheduling of a fast operation task and efficient execution of a calculation task without wasting hardware resources can be simultaneously satisfied by similarly adjusting the scale of the scheduling unit of the compiler and the hardware processing unit.

FIG. 7 is a block diagram for explaining the neural core of FIG. 5 in detail.

Referring to FIG. 7 , the neural core 100 includes a load/store unit (LSU) 110, an L0 memory 120, a weight buffer 130, an activation LSU 140, an activation buffer 150, and a processing unit ( 160) may be included.

The LSU 110 may receive at least one of data, a control signal, and a synchronization signal from the outside through the local interconnection 200 and the L2 sync path 300 . The LSU 110 may transmit at least one of received data, a control signal, and a synchronization signal to the L0 memory 120 . Similarly, the LSU 110 may transfer at least one of data, control signals, and synchronization signals to the outside through the local interconnection 200 and the L2 sync path 300 .

8 is a block diagram for explaining the LSU of FIG. 7 in detail.

Referring to FIG. 8 , the LSU 110 includes a local memory load unit 111a, a local memory store unit 111b, a neural core load unit 112a, a neural core store unit 112b, a load buffer LB, and a store. It may include a buffer SB, a load engine 113a, a store engine 113b and a conversion lookaside buffer 114.

The local memory load unit 111a may fetch a load instruction for the L0 memory 120 and issue the load instruction. When the local memory load unit 111a provides an issued load instruction to the load buffer LB, the load buffer LB may sequentially transmit memory access requests to the load engine 113a according to the input order.

Also, the local memory store unit 111b may fetch a store instruction for the L0 memory 120 and issue the store instruction. When the local memory store unit 111b provides the store instruction at issue to the store buffer SB, the store buffer SB may sequentially transmit memory access requests to the store engine 113b according to the input order.

The neural core load unit 112a may fetch load instructions for the neural core 100 and issue load instructions. When the neural core load unit 112a provides the issued load instruction to the load buffer LB, the load buffer LB may sequentially transmit memory access requests to the load engine 113a according to the input order.

Also, the neural core store unit 112b may fetch a store instruction for the neural core 100 and issue the store instruction. When the neural core store unit 112b provides the stored instruction to the store buffer SB, the store buffer SB may sequentially transmit memory access requests to the store engine 113b according to the input order.

The load engine 113a may receive a memory access request and load data through the local interconnection 200 . At this time, the load engine 113a can quickly find data by using a conversion table of recently used logical addresses and physical addresses in the conversion lookaside buffer 114 . If the logical address of the load engine 113a does not exist in the translation lookaside buffer 114, address translation information may be found in another memory.

The store engine 113b may receive a memory access request and load data through the local interconnection 200 . At this time, the store engine 113b can quickly find data by using a conversion table of recently used logical addresses and physical addresses in the conversion lookaside buffer 114 . When the logical address of the store engine 113b does not exist in the translation lookaside buffer 114, address translation information may be found in another memory.

The load engine 113a and the store engine 113b may send synchronization signals to the L2 sync path 300 . At this time, the synchronization signal may have a meaning that the work is finished.

Referring again to FIG. 7 , the L0 memory 120 is a memory located inside the neural core 100, and may receive all input data necessary for the neural core 100 to work from the outside and temporarily store them. In addition, the L0 memory 120 may temporarily store output data calculated by the neural core 100 to be transmitted to the outside. The L0 memory 120 may serve as a cache memory of the neural core 100 .

The L0 memory 120 may transmit the input activation (Act_In) to the activation buffer 150 and receive the output activation (Act_Out) by the activation LSU 140 . The L0 memory 120 may directly transmit/receive data with the processing unit 160 in addition to the activation LSU 140 . That is, the L0 memory 120 may exchange data with each of the PE array 163 and the vector unit 164 . The LO memory 120 may be a memory corresponding to a neural core level. In this case, the L0 memory 120 may be a private memory of the neural core.

The L0 memory 120 may transmit data such as activation or weight through a data path. The L0 memory 120 may transmit and receive synchronization signals through an L1 sync path, which is a separate dedicated path. The L0 memory 120 may exchange synchronization signals with, for example, the LSU 110, the weight buffer 130, the activation LSU 140, and the processing unit 160 through an L1 sync path. .

The weight buffer 130 may receive a weight from the L0 memory 120 . The weight buffer 130 may transfer the weight to the processing unit 160 . The weight buffer 130 may temporarily store weights before transferring them.

The input activation (Act_In) and the output activation (Act_Out) may refer to an input value and an output value of a layer of a neural network. In this case, when the neural network has a plurality of layers, the output activation value of the previous layer becomes the input value of the next layer, so the output activation (Act_Out) of the previous layer may be used as the input activation (Act_In) of the next layer.

The weight may mean a parameter that is multiplied with the input activation (Act_In) input in each layer. The weight is adjusted and determined in the deep learning step, and may be used to derive the output activation (Act_Out) through a fixed value in the inference step.

The activation LSU 140 may transfer input activation (Act_In) from the L0 memory 120 to the activation buffer 150 and transfer output activation (Act_Out) from the activation buffer 150 to the on-chip buffer. That is, the activation LSU 140 may perform both a load operation and a store operation of activation.

The activation buffer 150 may provide an input activation (Act_In) to the processing unit 160 and receive an output activation (Act_Out) from the processing unit 160. The activation buffer 150 may temporarily store input activation (Act_In) and output activation (Act_Out).

The activation buffer 150 can quickly provide activation to the processing unit 160 , particularly the PE array 163 , which requires a large amount of computation, and quickly receive the activation, thereby increasing the computational speed of the neural core 100 .

The processing unit 160 may be a module that performs calculations. The processing unit 160 may perform not only a 1-dimensional operation but also a 2-dimensional matrix operation, that is, a convolution operation. The processing unit 160 may generate an output activation (Act_Out) by receiving the input activation (Act_In), multiplying the received input activation (Act_In), and then adding the result.

FIG. 9 is a block diagram for explaining the processing unit of FIG. 7 in detail.

Referring to FIGS. 7 and 9 , the processing unit 160 may include a PE array 163 , a vector unit 164 , a column register 161 and a row register 162 .

The PE array 163 may perform multiplication by receiving the input activation (Act_In) and the weight (Weight). In this case, the input activation (Act_In) and the weight (Weight) may be calculated through convolution in the form of a matrix. Through this, the PE array 163 may generate an output activation (Act_Out). However, this embodiment is not limited thereto. The PE array 163 can also generate other types of outputs other than the output activation (Act_Out).

The PE array 163 may include at least one processing element 163_1. The processing elements 163_1 may be aligned with each other to perform multiplication of one input activation (Act_In) and one weight (Weight).

The PE array 163 may produce a subtotal sum of values for each multiplication. This subtotal can be utilized as an output activation (Act_Out). Since the PE array 163 performs 2D matrix multiplication, it may also be referred to as a 2D matrix compute unit.

The vector unit 164 may perform one-dimensional operations. The vector unit 164 may perform deep learning operations together with the PE array 163 . Through this, the processing unit 160 may be specialized for necessary operations. That is, the neural core 100 can efficiently perform deep learning tasks because each of the calculation modules that performs a large amount of 2D matrix multiplication and 1D calculation is performed.

The column register 161 may receive the first input I1. The column register 161 may receive the first input I1, divide it, and provide it to each column of the processing element 163_1.

The low register 162 may receive the second input I2. The row register 162 may receive the second input I2, divide it, and provide it to each row of the processing element 163_1.

The first input (I1) may be an input activation (Act_In) or a weight (Weight). The second input I2 may be a value other than the first input I1 among input activation Act_In or weight. Alternatively, the first input I1 and the second input I2 may be values other than the input activation Act_In and the weight.

FIG. 10 is a block diagram for explaining the L0 memory of FIG. 7 in detail.

Referring to FIG. 10 , the L0 memory 120 may include an arbiter 121 and at least one local memory bank 122 .

When data is stored in the LO memory 120, the arbiter 121 may receive the data from the load engine 113a. At this time, data may be allocated to the local memory bank 122 in a round robin manner. Accordingly, data may be stored in any one of the at least one local memory bank 122 .

Conversely, when data is loaded from the LO memory 120, the arbiter 121 may receive data from the local memory bank 122 and transfer the data to the store engine 113b. The store engine 113b may store data externally through the local interconnection 200 .

FIG. 11 is a block diagram for explaining the local memory bank of FIG. 10 in detail.

Referring to FIG. 11 , the local memory bank 122 may include a local memory bank controller 122_1 and a local memory bank cell array 122_2.

The local memory bank controller 122_1 may manage read and write operations through addresses of data stored in the local memory bank 122 . That is, the local memory bank controller 122_1 may manage the input/output of data as a whole.

The local memory bank cell array 122_2 may have a structure in which cells directly storing data are aligned in rows and columns. The local memory bank cell array 122_2 may be controlled by the local memory bank controller 122_1.

12 is a block diagram for explaining in detail the structure of a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 12 , the neural core 101 may have a CGRA structure unlike the neural core 100 . The neural core 101 may include an instruction memory 111_1, a CGRA LO memory 111_2, a PE array 111_3, and a load/store unit (LSU) 111_4.

The instruction memory 111_1 may receive and store instructions. The instruction memory 111_1 may sequentially store instructions therein and provide the stored instructions to the PE array 111_3. In this case, the instruction may indicate an operation of the processing element 111_3a included in each PE array 111_3.

The CGRA LO memory 111_2 is a memory located inside the neural core 101, and may receive all input data necessary for the neural core 101 to work from the outside and temporarily store them. In addition, the CGRA LO memory 111_2 may temporarily store output data calculated by the neural core 101 to be transmitted to the outside. The CGRA LO memory 111_2 may serve as a cache memory of the neural core 101 .

The CGRA LO memory 111_2 may transmit and receive data to and from the PE array 111_3. The CGRA L0 memory 111_2 may be a memory corresponding to L0 (level 0) lower than L1. In this case, the L0 memory may be a private memory of the neural core 101 that is not shared. The CGRA L0 memory 111_2 may transmit data such as activation or weight and programs to the PE array 111_3.

The PE array 111_3 may be a module that performs calculations. The PE array 111_3 may perform not only 1-dimensional operations but also 2-dimensional or higher matrix/tensor operations. The PE array 111_3 may include a plurality of processing elements 111_3a and specific processing elements 111_3b therein.

Processing elements 111_3a and specific processing elements 111_3b may be arranged in rows and columns. Processing elements 111_3a and specific processing elements 111_3b may be arranged in m columns. Also, processing elements 111_3a may be arranged in n rows, and specific processing elements 111_3b may be arranged in l rows. Accordingly, the processing element 111_3a and the specific processing element 111_3b may be arranged in (n+l) rows and m columns.

The LSU 111_4 may receive at least one of data, a control signal, and a synchronization signal from the outside through the local interconnection 200 . The LSU 111_4 may transmit at least one of received data, a control signal, and a synchronization signal to the CGRA LO memory 111_2. Similarly, the LSU 111_4 may transfer at least one of data, control signals, and synchronization signals to the outside through the local interconnection 200 .

The neural core 101 may have a Coarse Grained Reconfigurable Architecture (CGRA) structure. Accordingly, in the neural core 101, each processing element 111_3a and a specific processing element 111_3b of the PE array 111_3 are configured to include at least one of the CGRA LO memory 111_2, the instruction memory 111_1, and the LSU 111_4. can be connected to one. That is, the processing element 111_3a and the specific processing element 111_3b do not have to be connected to all of the CGRA L0 memory 111_2, the instruction memory 111_1, and the LSU 111_4, but may be connected to some of them.

Also, the processing element 111_3a and the specific processing element 111_3b may be different types of processing elements. Accordingly, among the CGRA LO memory 111_2, the instruction memory 111_1, and the LSU 111_4, an element connected to the processing element 111_3a and an element connected to the specific processing element 111_3b may be different from each other.

The neural core 101 of the present invention having a CGRA structure can perform a high level of parallel operation and directly exchange data between the processing element 111_3a and a specific processing element 111_3b, so power consumption can be low. In addition, optimization according to various calculation tasks may be possible by including two or more types of processing elements 111_3a.

For example, when the processing element 111_3a is a processing element that performs a two-dimensional operation, a specific processing element 111_3b may be a processing element that performs a one-dimensional operation. However, this embodiment is not limited thereto.

13 is a block diagram illustrating memory reconfiguration of a neural processing system according to some embodiments of the present invention.

Referring to FIG. 13 , the neural core SoC 10 may include first to eighth processing units 160a to 160h and an on-chip memory (OCM). 24 shows 8 processing units as an example, but this is only an example and the number of processing units may vary.

The on-chip memory OCM may include first to eighth L0 memories 120a to 120h and a shared memory 2000 .

The first to eighth L0 memories 120a to 120h may be used as dedicated memories of the first to eighth processing units 160a to 160h, respectively. That is, the first to eighth processing units 160a to 160h and the first to eighth L0 memories 120a to 120h may correspond 1:1 to each other.

The shared memory 2000 may include first to eighth memory units 2100a to 2100h. The first to eighth memory units 2100a to 2100h may respectively correspond to the first to eighth processing units 160a to 160h and the first to eighth L0 memories 120a to 120h. That is, the number of memory units may be eight, the same as the number of processing units and L0 memories.

The shared memory 2000 may operate in one of two types of on-chip memory formats. That is, the shared memory 2000 can operate in either a L0 memory format or a global memory format. That is, the shared memory 2000 can implement two types of logical memories with one hardware.

When the shared memory 2000 is implemented in an L0 memory format, the shared memory 2000 is a dedicated memory for each of the first to eighth processing units 160a to 160h, such as the first to eighth L0 memories 120a to 120h. (private memory) can operate. The L0 memory can operate at a relatively high-speed clock compared to the global memory, and when the shared memory 2000 operates in the L0 memory format, it can use a relatively faster clock.

When the shared memory 2000 is implemented in a global memory format, the shared memory 2000 may operate as a common memory used by the first processing unit 100a and the second processing unit 100b. there is. In this case, the shared memory 2000 may be shared by the first to eighth L0 memories 120a to 120h as well as the first to eighth processing units 160a to 160h.

The global memory may generally use a lower clock than the L0 memory, but is not limited thereto. When the shared memory 2000 operates in a global memory format, the first to eighth processing units 160a to 160h may share the shared memory 2000 . At this time, the shared memory 2000 is connected to the volatile memory 32 of FIG. 2 through the global interconnection 5000 and may operate as a buffer of the volatile memory 32 .

At least a part of the shared memory 2000 may operate in an L0 memory format and the rest may operate in a global memory format. That is, the entire shared memory 2000 may operate in the L0 memory format or the entire shared memory 2000 may operate in the global memory format. Alternatively, a part of the shared memory 2000 may operate in the L0 memory format and the remaining part may operate in the global memory format.

14 is a block diagram illustrating an example of memory reorganization of a neural processing system in accordance with some embodiments of the present invention.

Referring to FIGS. 13 and 14 , the first, third, fifth, and seventh exclusive areas AE1 and AE3 of the first, third, fifth, and seventh processing units 100a, 100c, 100e, and 100g, respectively. , AE5, and AE7 may include only the first, third, fifth, and seventh L0 memories 120a, 120c, 120e, and 120g, respectively. In addition, the second, fourth, sixth, and eighth dedicated regions AE2, AE4, AE6, and AE8 of each of the second, fourth, sixth, and eighth processing units 100b, 100d, 100f, and 100h respectively Second, fourth, sixth, and eighth L0 memories 120b, 120d, 120f, and 120h may be included. Also, the second, fourth, sixth, and eighth dedicated areas AE2 , AE4 , AE6 , and AE8 may include second, fourth, sixth, and eighth memory units 2100b, 2100d, 2100f, and 2100h. can The first, third, fifth, and seventh memory units 2100a, 2100c, 2100e, and 2100g of the shared memory 2000 may be used as a common area AC.

The common area AC may be a memory shared by the first to eighth processing units 160a to 160h. The second dedicated area AE2 may include a second LO memory 120b and a second memory unit 2100b. The second dedicated area AE2 may be an area in which the hardware-separated second L0 memory 120b and the second memory unit 210b operate in the same way and logically operate as one L0 memory. The fourth, sixth, and eighth dedicated areas AE4, AE6, and AE8 may also operate in the same manner as the second dedicated area AE2.

The shared memory 2000 according to the present embodiment can be used by converting an area corresponding to each neural core into a logical L0 memory and a logical global memory at an optimized ratio. The shared memory 2000 can adjust this ratio at run time.

That is, each neural core may perform the same task in some cases, but may also perform different tasks in other cases. In this case, the capacity of the L0 memory and the capacity of the global memory required for the work performed by each neural core are inevitably different each time. Accordingly, in the case where the composition ratio of the L0 memory and the shared memory is set to be fixed, as in the conventional on-chip memory, inefficiency may occur due to calculation tasks allocated to each neural core.

Accordingly, the shared memory 2000 of the neural processing apparatus according to the present embodiment may set an optimal ratio of the L0 memory and the global memory according to the computational task during run time, and improve computation efficiency and speed.

FIG. 15 is an enlarged block diagram of part A of FIG. 13 .

13 and 15, the shared memory 2000 includes a first L0 memory controller 122_1a, a second L0 memory controller 122_1b, a fifth L0 memory controller 122_1e, and a sixth L0 memory controller 122_1f. , first to eighth memory units 2100a to 2100h and a global controller 2200 . Other L0 memory controllers not shown may also be included in the present embodiment, but descriptions thereof are omitted for convenience.

The first L0 memory controller 122_1a may control the first L0 memory 120a. Also, the first L0 memory controller 122_1a may control the first memory unit 2100a. Specifically, when the first memory unit 2100a is implemented in a logical L0 memory format, control by the first L0 memory controller 122_1a may be performed on the first memory unit 2100a.

The second L0 memory controller 122_1b may control the second L0 memory 120b. Also, the second L0 memory controller 122_1b may control the second memory unit 2100b. That is, when the second memory unit 2100b is implemented in a logical L0 memory format, control by the first L0 memory controller 122_1a may be performed on the second memory unit 2100b.

The fifth L0 memory controller 122_1e may control the fifth L0 memory 120e. Also, the fifth L0 memory controller 122_1e may control the fifth memory unit 2100e. That is, when the fifth memory unit 2100e is implemented in a logical L0 memory format, control by the fifth L0 memory controller 122_1e may be performed on the fifth memory unit 2100e.

The sixth L0 memory controller 122_1f may control the sixth L0 memory 120f. Also, the sixth L0 memory controller 122_1f may control the sixth memory unit 2100f. That is, when the sixth memory unit 2100f is implemented in a logical L0 memory format, control by the sixth L0 memory controller 122_1f may be performed on the sixth memory unit 2100f.

The global controller 2200 may control all of the first to eighth memory units 2100a to 2100h. Specifically, when the first to eighth memory units 2100a to 2100h logically operate in the global memory format (that is, do not logically operate in the L0 memory format), the global controller 2200 operates the first memory unit 2100a to 2100h. Units 2100a to 8th memory units 2100h may be controlled.

That is, the first to eighth memory units 2100a to 2100h are each controlled by the first to eighth L0 memory controllers 122_1a to 122_1h or controlled by the global controller 2200 depending on what type of memory is logically implemented. can be controlled by

When the L0 memory controller including the first, second, fifth, and sixth L0 memory controllers 122_1a, 122_1b, 122_1e, and 122_1f respectively control the first to eighth memory units 2100a to 2100h, the first Since the eighth L0 memory controllers 122_1a to 141h control the first to eighth memory units 2100a to 2100h in the same way as the first to eighth L0 memories 120a to 120h, the first to eighth processing units (160a ~ 160h) can be controlled by dedicated memory. Accordingly, the first to eighth memory units 2100a to 2100h may operate at a clock frequency corresponding to that of the first to eighth processing units 160a to 160h.

Each of the L0 memory controllers including the first L0 memory controller 122_1a, the second L0 memory controller 122_1b, the fifth L0 memory controller 122_1e, and the sixth L0 memory controller 122_1f is the LSU 110 of FIG. 7 . can include

When the global controller 2200 controls at least one of the first to eighth memory units 2100a to 2100h, the global controller 2200 controls the first to eighth memory units 2100a to 2100h, respectively. It can be controlled by the global memory of the eighth processing units 160a to 160h. Accordingly, at least one of the first to eighth memory units 2100a to 2100h may operate at a clock frequency independent of the clock frequency of the first to eighth processing units 160a to 160h, respectively. However, this embodiment is not limited thereto.

The global controller 2200 may connect the first to eighth memory units 2100a to 2100h with the global interconnection 5000 of FIG. 3 . The first to eighth memory units 2100a to 2100h exchange data with the off-chip memory 30 of FIG. 1 by the global controller 2200 or exchange data with the first to eighth L0 memories 120a to 120h, respectively. can be exchanged.

Each of the first to eighth memory units 2100a to 2100h may include at least one memory bank. The first memory unit 2100a may include at least one first memory bank 2110a. The first memory bank 2110a may be an area obtained by dividing the first memory unit 2100a into a specific size. Each of the first memory banks 2110a may be memory devices having the same size. However, this embodiment is not limited thereto. 15 shows that four memory banks are included in one memory unit.

Similarly, the second, fifth, and sixth memory units 2100b, 2100e, and 2100f may each include at least one second, fifth, and sixth memory bank 2110b, 2110e, and 2110f.

Hereinafter, the first memory bank 2110a and the fifth memory bank 2110e will be described, which may be the same as other memory banks including the second and sixth memory banks 2110b and 2110f.

Each of the first memory banks 2110a may logically operate in an L0 memory format or logically operate in a global memory format. In this case, the first memory bank 2110a may operate independently of other memory banks in the first memory unit 2100a. However, this embodiment is not limited thereto.

When each memory bank operates independently, the first memory unit 2100a includes a first area that operates in the same way as the first L0 memory 120a and a second region that operates in a different way from the first L0 memory 120a. It can contain 2 areas. In this case, the first area and the second area do not necessarily coexist, and either area may occupy the entire first memory unit 2100a.

Similarly, the second memory unit 2100b may include a third area operating in the same way as the second L0 memory 120b and a fourth area operating in a different way from the second L0 memory 120b. In this case, the third area and the fourth area do not necessarily coexist, and either area may occupy the entire first memory unit 2100a.

In this case, the ratio of the first area to the second area may be different from the ratio of the third area to the fourth area. However, this embodiment is not limited thereto. Accordingly, the ratio of the first area to the second area may be the same as the ratio of the third area to the fourth area. That is, the memory configuration ratio in each memory unit can be varied as desired.

In general, in the case of a conventional system-on-chip, on-chip memories other than high-speed L0 memory are often composed of high-density low-power SRAM. This is because SRAM has high efficiency in terms of chip area and power consumption compared to required capacity. However, the existing on-chip memory inevitably slows down the processing speed for tasks that require more data than the predetermined capacity of the L0 memory. There was no inefficiency at all.

In contrast, the shared memory 2000 according to some embodiments of the present invention may be selectively controlled by any one of the two controllers in some cases. In this case, the shared memory 2000 is not entirely controlled by a predetermined one of the two controllers, and may be independently controlled in units of memory units or units of memory banks.

Through this, the shared memory 2000 according to the present embodiment can obtain an optimal memory configuration ratio according to a computational task during run time, and thus perform a faster and more efficient calculation task. In the case of a processing unit specialized in artificial intelligence, the required size of L0 memory and global memory may vary for each specific application unit. Furthermore, when a deep learning network is used even for the same application, required sizes of the L0 memory and the global memory may be different for each layer. In the shared memory 2000 according to the present embodiment, the configuration ratio of the memory can be changed during run time even if the calculation step according to each layer is changed, so that fast and efficient deep learning work can be possible.

FIG. 16 is a diagram for explaining the first memory bank of FIG. 15 in detail. 16 illustrates the first memory bank 2110a, other memory banks may also have the same structure as the first memory bank 2110a.

Referring to FIG. 16 , the first memory bank 2110a may include a cell array Ca, a bank controller Bc, a first path unit P1 and a second path unit P2.

The cell array Ca may include a plurality of memory elements Cell therein. In the cell array Ca, a plurality of memory elements may be arranged in a lattice structure. The cell array Ca may be, for example, a static random access memory (SRAM) cell array.

The bank controller Bc may control the cell array Ca. The bank controller Bc may determine whether the cell array Ca operates in the L0 memory format or the global memory format and controls the cell array Ca accordingly.

Specifically, the bank controller Bc may determine whether to transmit/receive data in the direction of the first path unit P1 or the direction of the second path unit P2 during run time. The bank controller Bc may determine the transmission/reception direction of data according to the path control signal Spc.

The path control signal Spc may be generated by a pre-designed device driver or compiler. The path control signal Spc may be generated according to the characteristics of an arithmetic operation. Alternatively, the path control signal Spc may be generated by an input received from a user. That is, the user may directly apply an input to the path control signal Spc in order to select the most optimal memory configuration ratio.

The bank controller Bc may determine a transmission/reception path of data stored in the cell array Ca through the path control signal Spc. The data exchange interface may vary according to the bank controller Bc determining the transmission/reception path of the data. That is, when the bank controller Bc exchanges data with the first path unit P1, the first interface may be used, and when data is exchanged with the second path unit P2, the second interface may be used. In this case, the first interface and the second interface may be different from each other.

Also, an address system in which data is stored may be different. That is, when a specific interface is selected, read and write operations can be performed with an address system corresponding thereto.

The bank controller Bc may operate at a specific clock frequency. For example, when the cell array Ca is an SRAM cell array, the bank controller Bc may operate at an operating clock frequency of a general SRAM.

The first path unit P1 may be connected to the bank controller Bc. The first path unit P1 may directly exchange data of the cell array Ca with the first processing unit 100a. In this case, “direct” may mean mutual exchange without going through the global interconnection 5000. That is, the first processing unit 100a may directly exchange data with the first L0 memory 120a, and the first processing unit 100a may perform the first processing unit 100a when the shared memory 2000 is logically implemented in the form of an L0 memory. Data can be exchanged through 1 path unit P1. The first path unit P1 may include an L0 memory controller including the first L0 memory controller 122_1a and the second L0 memory controller 122_1b of FIG. 14 .

The first path unit P1 may configure a multi-cycle sync-path. That is, the operating clock frequency of the first path unit P1 may be the same as the operating clock frequency of the first processing unit 100a. The first L0 memory 120a may rapidly exchange data at the same clock frequency as the operating clock frequency of the first processing unit 100a in order to rapidly exchange data at the same speed as the operation of the first processing unit 100a. . The first path unit P1 may also operate at the same clock frequency as the operating clock frequency of the first processing unit 100a.

In this case, the operating clock frequency of the first path unit P1 may be a multiple of the operating clock frequency of the bank controller Bc. In this case, a separate CDC (Clock Domain Crossing) operation for clock synchronization between the bank controller Bc and the first path unit P1 is not required, and thus data transmission delay may not occur. there is. Accordingly, faster and more efficient data exchange may be possible.

16 exemplarily, the operating clock frequency of the first path unit P1 may be 1.5 GHz. This may be twice the frequency of 750 MHz of the bank controller (Bc). However, the present embodiment is not limited thereto, and any number may be possible if the first path unit P1 operates at an integer multiple of the clock frequency of the bank controller Bc.

The second path unit P2 may be connected to the bank controller Bc. The second path unit P2 may exchange data of the cell array Ca through the global interconnection 5000 without directly exchanging the data of the first processing unit 100a. That is, the first processing unit 100a may exchange data with the cell array Ca through the global interconnection 5000 and the second path unit P2. In this case, the cell array Ca may exchange data not only with the first processing unit 100a but also with other neural cores.

That is, the second path unit P2 may be a data exchange path between the cell array Ca and all neural cores when the first memory bank 2110a is logically implemented in a global memory format. The second path unit P2 may include the global controller 2200 of FIG. 22 .

The second path unit P2 may constitute an async-path. The operating clock frequency of the second path unit P2 may be the same as that of the global interconnection 5000 . The second path unit P2 may also operate at the same clock frequency as the operating clock frequency of the global interconnection 5000 .

At this time, the operating clock frequency of the second path unit P2 may not be synchronized with the operating clock frequency of the bank controller Bc. In this case, a clock domain crossing (CDC) operation may be required for clock synchronization between the bank controller Bc and the second path unit P2. When the operating clock frequency of the bank controller Bc and the operating clock frequency of the second path unit P2 are not synchronized with each other, the degree of freedom in clock domain design can be increased. Accordingly, the difficulty of hardware design is lowered, and the hardware operation can be derived more easily.

The bank controller Bc may use different address systems when exchanging data through the first path unit P1 and when exchanging data through the second path unit P2. That is, the bank controller Bc may use the first address system through the first path unit P1 and use the second address system through the second path unit P2. In this case, the first address system and the second address system may be different from each other.

The bank controller Bc does not necessarily exist for each memory bank. That is, since the bank controller (Bc) is not a part for scheduling but serves to transmit signals, it is not an essential part for each memory bank having two ports. Thus, one bank controller Bc can control several memory banks. Several memory banks can operate independently even though they are controlled by the bank controller Bc. However, this embodiment is not limited thereto.

Of course, the bank controller Bc may exist for each memory bank. In this case, the bank controller Bc can individually control each memory bank.

15 and 16, when the first memory unit 210a exchanges data through the first path unit P1, the first address system is used and data is exchanged through the second path unit P2. In case of exchange, a second address scheme may be used. Similarly, when the second memory unit 210b exchanges data through the first path unit P1, the third address system is used, and when data is exchanged through the second path unit P2, the second address system is used. system can be used. In this case, the first address system and the third address system may be identical to each other. However, this embodiment is not limited thereto.

The first address system and the third address system may be used exclusively for the first processing unit 100a and the second processing unit 100b, respectively. The second address scheme may be commonly applied to the first processing unit 100a and the second processing unit 100b.

15 exemplarily, the operating clock frequency of the second path unit P2 may operate at 1 GHz. This may be a frequency that is not synchronized with the 750 MHz operating clock frequency of the bank controller (Bc). That is, the operating clock frequency of the second path unit P2 is not dependent on the operating clock frequency of the bank controller Bc and can be freely set.

A general global memory uses a slow SRAM (eg, 750 MHz) and a faster global interconnection (eg, 1 GHz), so that delays due to CDC work inevitably occur. In contrast, in the shared memory 2000 according to some embodiments of the present invention, there is room to use the first path unit P1 in addition to the second path unit P2, so delay due to CDC work can be avoided.

In addition, since a plurality of neural cores in a general global memory use one global interconnection 5000, the overall processing speed may be easily reduced when the amount of data transfer occurs simultaneously. In contrast, in the shared memory 2000 according to some embodiments of the present invention, there is room to use the first path unit P1 in addition to the second path unit P2, so that the amount of data processed in the global controller 2200 is adequately reduced. A dispersing effect can also be obtained.

17 is a block diagram illustrating a software layer structure of a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 17 , a software layer structure of a neural processing apparatus according to some embodiments of the present invention may include a DL framework 10000, a compiler stack 20000, and a backend module 30000.

The DL framework 10000 may refer to a framework for a deep learning model network used by a user. For example, a trained neural network may be created using a program such as TensorFlow or PyTorch.

The compiler stack 20000 may include an adaptation layer 21000, a compute library 22000, a front-end compiler 23000, a back-end compiler 24000, and a runtime driver 25000.

The adaptation layer 21000 may be a layer in contact with the DL framework 10000. The adaptation layer 21000 may quantize the user's neural network model generated in the DL framework 10000 and perform graph correction. Also, the adaptation layer 21000 may convert a model type into a required type.

The front-end compiler 23000 may convert various neural network models and graphs transmitted from the adaptation layer 21000 into a constant intermediate representation (IR). The converted IR may be a preset expression that is easy to handle in the backend compiler 24000 later.

The IR of the front-end compiler 23000 may be preliminarily optimized at the graph level. In addition, the front-end compiler 23000 may finally generate an IR through an operation of converting the layout into a hardware-optimized layout.

The backend compiler 24000 optimizes the IR converted by the frontend compiler 23000 and converts it into a binary file so that the runtime driver can use it. The backend compiler 24000 may generate optimized code by dividing a job into a scale suitable for hardware details.

The compute library 22000 may store template operations designed in a form suitable for hardware among various operations. The compute library 22000 provides the backend compiler 24000 with several template operations that require hardware to generate optimized codes.

The runtime driver 25000 may perform continuous monitoring during operation to drive the neural network device according to some embodiments of the present invention. Specifically, it may be responsible for executing interfaces of neural network devices.

The backend module 30000 may include an application specific integrated circuit (ASIC) 31000, a field programmable gate array (FPGA) 32000, and a C-model 33000. The ASIC 31000 may refer to a hardware chip determined according to a predetermined design method. The FPGA 32000 may be a programmable hardware chip. The C-model (33000) may refer to a model implemented by imitating hardware on software.

The backend module 30000 may perform various tasks and derive results using binary codes generated through the compiler stack 20000 .

18 is a block diagram for explaining multi-processes of a neural processing system according to some embodiments of the present invention.

Referring to FIG. 18 , the neural processing system may create several processes (Prs1 to Prs3) in an operating system (OS). At this time, although three processes are shown in FIG. 18, the present embodiment is not limited thereto. That is, the number of processes may be 2 or less or 4 or more.

Each of the processes (Prs1 to Prs3) can all be defined by the same context. At this time, the first process (Prs1), the second process (Prs2), and the third process (Prs3) may be separately allocated system resources from the operation system (OS). Each of the first process (Prs1), the second process (Prs2), and the third process (Prs3) may be contexts loaded into memory and executed, that is, instances of programs.

In principle, each process runs in a separate address space, and resources may not be shared between each process. However, the first process (Prs1), the second process (Prs2), and the third process (Prs3) of this embodiment may be executed in the same address space according to the type of data. Of course, it can be executed in different address spaces depending on the type of data.

The first process Prs1, the second process Prs2, and the third process Prs3 may have virtual memory areas. Each virtual memory area may individually have a code area, a data area, a heap area, and a stack area. In this case, the code area may mean a space in which program functions written by a user are stored, and the data area may mean a space in which data used by a program, such as a global variable or a static variable, is stored. The heap area may refer to a memory area used whenever a programmer needs it. In addition, the stack area may refer to a space for temporarily storing addresses and data (local variables, parameters, return values, etc.) to be returned after execution of a called function.

In this case, the virtual memory area may be shared by the first process Prs1, the second process Prs2, and the third process Prs3, or may be independent of each other. This will be explained in more detail later.

The first process Prs1 may include a first thread and a second thread. In this case, a thread may be a unit of a flow executed within a process. That is, the first thread and the second thread may perform tasks in parallel while sharing process resources. In FIG. 18, each of the first process (Prs1), the second process (Prs2), and the third process (Prs3) shows two threads, but the present embodiment is not limited thereto. That is, the number of threads of each process may be different from each other, and may be 1 or less or 3 or more.

Each thread can be allocated a separate stack area. Each thread can share a code area, a data area, and a heap area.

19 is a block diagram illustrating address conversion of a neural processing system according to some embodiments of the present invention.

Referring to FIG. 19 , the processing unit 160 may issue an instruction including a logical address LA for a task. In this case, the logical address LA may be converted into a physical address PA by the memory management unit MMU.

The logical address LA is a virtual address and may be different from the physical address PA, which is an actual address of the physical memory PM. The logical address LA may be mapped with the physical address PA. As the logical address LA is used, all addresses of the physical memory PM do not have to be paged at once, but can be paged only when necessary. Also, space of the physical memory PM may be saved by mapping a plurality of logical addresses LA to one physical address PA.

The memory management unit (MMU) may include a translation lookaside buffer 114 and a shared page table (SPT). The conversion lookaside buffer 114 may be included in the aforementioned LSU 110 of FIG. 8 . The shared page table (SPT) may be located in at least one external memory, that is, the off-chip memory 30 of FIG. 2 , the shared memory 2000 of FIG. 3 , and the L2 shared memory 2000 of FIG. 5 . However, this embodiment is not limited thereto.

As described above, the conversion lookaside buffer 114 may include information for converting a recently used logical address LA into a physical address PA. Accordingly, when the processing unit 160 issues an instruction including the logical address LA, it first checks whether the same logical address LA as the logical address LA of the issued instruction is stored in the conversion lookaside buffer 114. You can check.

If the same logical address as the logical address (LA) of the issued instruction is stored in the translation lookaside buffer 114, this is a situation where the translation lookaside buffer 114 is hit (TLB hit), and the translation lookaside buffer In step 114, the logical address LA can be quickly converted to the physical address PA. That is, the physical address PA is acquired through the first path R1, and the instruction can be accessed to the physical address PA of the physical memory PM.

The physical memory PM may include at least one physical page Pg0 to Pg(n−1). Although not all parts of the physical memory PM are paged, parts for the physical addresses PA mapped by the logical addresses LA may exist as physical pages Pg0 to Pg(n-1). All of the physical pages Pg0 to Pg(n-1) may be allocated with the same size. The physical memory PM may be at least one of the off-chip memory 30 of FIG. 2 , the shared memory 2000 of FIG. 3 , and the L2 shared memory 2000 of FIG. 5 , but is not limited thereto.

The memory management unit (MMU) may include a shared page table (SPT). The shared page table (SPT) may include conversion information of physical addresses (PAs) for all logical addresses (LAs). That is, the shared page table (SPT) may include mapping information for the logical address (LA) not stored in the translation lookaside buffer 114 .

If the same logical address as the logical address (LA) of the issued instruction is not stored in the translation lookaside buffer 114, this is a situation where the translation lookaside buffer 114 misses (TLB miss), and the translation index The logical address LA may be converted to the physical address PA by the shared page table SPT instead of the buffer 114 . That is, the physical address PA is acquired through the second path R2, and the physical address PA of the physical memory PM can be accessed by the instruction.

In this case, the shared page table (SPT) may be maintained the same even if the process is switched. That is, in general, each process uses a different memory address and, accordingly, a different page table, but the neural processing apparatus according to the present embodiment can use only one shared page table even when process is switched.

Since the conversion lookaside buffer 114 stores recently accessed addresses, access to the frequently accessed physical address PA of the physical memory PM can be quickly guided.

However, in a general device, while different processes use different page tables, when process switching occurs, access to the physical address PA of the process used just before may become meaningless. That is, the hit rate, that is, the hit rate of the conversion lookaside buffer may be significantly lowered.

In contrast, the neural processing apparatus according to the present embodiment uses the shared page table (SPT) and performs access through the same shared page table (SPT) even when process switching occurs, so that the heat rate of the translation lookaside buffer 114 is maintained. Efficient memory access can proceed.

FIG. 20 is a block diagram illustrating allocation of a physical page of the physical memory of FIG. 19 .

19 and 20, the shared page table (SPT) includes a page table entry (PTE) recording the start position of the physical address (PA) and mapping type data (Md) for each logical address (LA). ) may be included. PTE (PTE) may be used to calculate a physical address (PA) corresponding to the logical address (LA).

The mapping type data Md may include information about the mapping type of the logical address LA. The mapping type data Md may be, for example, data indicating whether the mapping type is a shared type or a private type.

The sharing type may be an address where data shared by all processes in a multi-process environment is stored. Data stored in the shared type address may be stored in the shared physical page Psh in the physical memory PM. The shared physical page Psh may be a physical page shared by first to fourth processes when there are four processes of a plurality of processes, for example, first to fourth processes.

Since the shared physical page (Psh) stores data shared by each process in one physical page, it is possible to maximize storage efficiency by reducing the number of physical pages required by the number of processes to one.

That is, data used by each process may be stored in the shared physical page (Psh). In this case, the shared page table (SPT) allows all processes to access the shared physical page (Psh) of the same physical address (PA). In this case, the physical address (PA) of the shared physical page (Psh) may be the same as the value of PTE (PTE).

The private type may be an address for dedicated data not shared by each process in a multi-process environment. Data stored in the address of the private type may be stored in the private physical page Ppr in the physical memory PM. In the private physical page Ppr, the first to fourth processes may access different dedicated pages.

That is, each process can store data in one private physical page (Ppr). In this case, as many private physical pages (Ppr) as the number of processes, that is, as many as m may exist. That is, for example, if there are 4 processes, there may be 4 private physical pages (Ppr). That is, m may be the number of processes. In this case, the private physical pages Ppr may be arranged to be consecutive to each other.

Accordingly, for the private physical page Ppr, the physical address PA may be calculated by adding an offset according to each process ID to the value of the PTE (PTE). For example, a first process calculates a physical address (PA) according to a PTE (PTE) value, and a second process adds an offset equal to the size of a physical page to the PTE (PTE) value to obtain a physical address (PA). can be calculated. In the third process, the physical address PA may be calculated by adding an offset equal to twice the size of the physical page to the PTE value. Similarly, in the fourth process, the physical address PA may be calculated by adding an offset three times the size of the physical page to the PTE value.

21 is a block diagram illustrating allocation of a physical page of a physical memory of a neural processing system according to some embodiments of the present invention.

Referring to FIG. 21 , the mapping type data Md may be, for example, data indicating whether the mapping type is a shared type, a private type, or an intermediate type.

The intermediate type may be an address for data shared by some processes in a multi-process environment. Data stored in the intermediate type address may be stored in an intermediate physical page Pitm in the physical memory PM. In the intermediate physical page (Pitm), some of the first to fourth processes may share the same page. For example, the first and third processes may share an intermediate physical page (Pitm) with each other, and the second and fourth processes may share an intermediate physical page (Pitm) with each other.

In this case, the first and second processes may be performed simultaneously, so that they do not share the same intermediate physical page (Pitm). Similarly, the third and fourth processes may be performed simultaneously with each other, so that they do not share the same intermediate physical page (Pitm) with each other.

That is, processes sharing the same intermediate physical page (Pitm) may be processes that do not operate concurrently with each other.

In this case, concurrently operating processes operating concurrently may store data in different intermediate physical pages (Pitm). In this case, as many intermediate physical pages (Pitm) as the number of simultaneous operating processes, that is, as many as k, may exist.

That is, for example, if the first and second processes are processes operating concurrently with each other, and the third and fourth processes are processes operating concurrently with each other, k may be two. Accordingly, two intermediate physical pages (Pitm) may be allocated. At this time, the intermediate physical pages (Pitm) may be arranged to be continuous with each other.

Accordingly, for the intermediate physical page Pitm, the physical address PA may be calculated by adding the final offset according to each process ID to the value of the PTE (PTE). The final offset may be an offset that concurrently operating processes have the same value as each other. For example, if the first and second processes are processes operating concurrently with each other, and the third and fourth processes are processes operating concurrently with each other, then the final offsets for the first and second processes are 0, and the third and fourth processes are concurrently operating. The final offset for p may be the size of the intermediate physical page (Pitm).

For example, the first process and the third process calculate a physical address (PA) according to the PTE (PTE) value, and the second process and the fourth process calculate an offset equal to the size of the physical page in the PTE (PTE) value. By adding them, the physical address PA may be calculated.

In the case of the intermediate physical page (Pitm), a buffer width equal to the number of processes not concurrently operating may be formed, and a process depth equal to the number of concurrently operating processes (k) may be formed.

22 is a block diagram illustrating allocation of a physical page of a physical memory of a neural processing system according to some embodiments of the present invention.

Referring to FIG. 22 , the mapping type data Md may be, for example, data indicating whether the mapping type is a private type or an intermediate type.

That is, a physical page shared by all processes does not exist, but a private physical page (Ppr), which is a physical page dedicated to individual processes, and an intermediate physical page (Pitm) shared by some processes may exist in the physical memory (PM). there is.

Although there is no physical page shared by all processes, the intermediate physical page (Pitm) can share data among some processes to increase data storage efficiency, and the conversion index buffer (114) using the shared page table (SPT). ) can also maintain a high heat rate.

FIG. 23 is a conceptual diagram for explaining mapping type data of FIG. 19 .

Referring to FIG. 23 , mapping type data Md may be expressed as one of a shared type, a private type, and an intermediate type. In this case, the shared type may be, for example, an address where context data shared by all one context is stored.

Unlike this, in the case of a private type, it may be, for example, an address where input/output data of each process is stored. Finally, in the case of the intermediate type, it may be an address where data that can be shared with some other processes is stored.

24 is a conceptual diagram for explaining a deep learning operation performed by a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 24, the artificial neural network model 40000, as an example of a machine learning model, is a statistical learning algorithm implemented based on the structure of a biological neural network or an algorithm thereof in machine learning technology and cognitive science. is the structure that runs

In the artificial neural network model 40000, as in a biological neural network, nodes, which are artificial neurons that form a network by combining synapses, repeatedly adjust the weight of synapses, so that between the correct output corresponding to a specific input and the inferred output By learning to reduce the error of , it is possible to represent a machine learning model having problem solving ability. For example, the artificial neural network model 40000 may include an arbitrary probability model, a neural network model, and the like used in artificial intelligence learning methods such as machine learning and deep learning.

A neural processing apparatus according to some embodiments of the present invention may implement the form of the artificial neural network model 40000 to perform calculations. For example, the artificial neural network model 40000 may receive an input image and output information about at least a part of an object included in the input image.

The artificial neural network model 40000 is implemented as a multilayer perceptron (MLP) composed of multi-layer nodes and connections between them. The artificial neural network model 40000 according to this embodiment may be implemented using one of various artificial neural network model structures including MLP. As shown in FIG. 25, the artificial neural network model 40000 includes an input layer 41000 that receives input signals or data 40100 from the outside, and an output layer that outputs output signals or data 40200 corresponding to the input data. (44000), which is located between the input layer 41000 and the output layer 44000, receives signals from the input layer 41000, extracts characteristics, and delivers n (where n is a positive integer) to the output layer 44000. It is composed of hidden layers (42000 to 43000). Here, the output layer 44000 receives signals from the hidden layers 42000 to 43000 and outputs them to the outside.

The learning method of the artificial neural network model (40000) includes a supervised learning method that learns to be optimized for problem solving by inputting a teacher signal (correct answer), and an unsupervised learning method that does not require a teacher signal. ) way.

The neural processing apparatus may directly generate learning data for learning the artificial neural network model 40000 through simulation. In this way, a plurality of output variables corresponding to a plurality of input variables are matched in the input layer 41000 and the output layer 44000 of the artificial neural network model 40000, respectively, and the input layer 41000, hidden layers 42000 to 43000 and By adjusting synapse values between nodes included in the output layer 44000, learning can be performed so that a correct output corresponding to a specific input can be extracted. Through this learning process, the characteristics hidden in the input variables of the artificial neural network model 40000 can be identified, and the nodes of the artificial neural network model 40000 can reduce the error between the output variable calculated based on the input variable and the target output. You can adjust the synaptic value (or weight) between them.

25 is a conceptual diagram illustrating learning and reasoning operations of a neural network of a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 25 , in a training phase, a plurality of training materials (TD) may be forwarded to an artificial neural network model (NN) and then forwarded again. Through this, weights and biases of each node of the artificial neural network model (NN) are tuned, and through this, learning can be performed so that more and more accurate results can be derived. In this way, through the training phase, the artificial neural network model (NN) may be converted into a learned neural network model (NN_T).

In the inference phase, new data ND may be input to the learned neural network model NN_T again. The learned neural network model NN_T may derive result data RD through already learned weights and biases by taking new data ND as an input. For the result data RD, it may be important which learning material TD was used in the training phase and how many of the learning materials TD were used.

Hereinafter, a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention will be described with reference to FIGS. 19, 20, 26, and 27 . Parts overlapping with the above-described embodiment are omitted or simplified.

26 is a flowchart illustrating a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention, and FIG. 27 is a flowchart illustrating the step of accessing the physical memory of FIG. 26 .

Referring to FIG. 26 , the first process accesses the memory by referring to the shared page table (S100).

Specifically, referring to FIG. 19 , the processing unit 160 may issue an instruction including a logical address LA for a task. In this case, the logical address LA may be converted into a physical address PA by the memory management unit MMU.

The memory management unit (MMU) may include a translation lookaside buffer 114 and a shared page table (SPT). The memory management unit (MMU) may include a shared page table (SPT). The shared page table (SPT) may include conversion information of physical addresses (PAs) for all logical addresses (LAs). That is, the shared page table (SPT) may include mapping information for the logical address (LA) not stored in the translation lookaside buffer 114 .

If the same logical address as the logical address (LA) of the issued instruction is not stored in the translation lookaside buffer 114, this is a situation where the translation lookaside buffer 114 misses (TLB miss), and the translation index The logical address LA may be converted to the physical address PA by the shared page table SPT instead of the buffer 114 . That is, the physical address PA is acquired through the second path R2, and the physical address PA of the physical memory PM can be accessed by the instruction.

Again, referring to FIG. 26, it is determined whether there is process switching (S200). If so, the second process accesses the memory by referring to the shared page table (S200). If not, the procedure ends.

Specifically, referring to FIG. 19 , the shared page table (SPT) may remain the same even if a process is switched. That is, in general, each process uses a different memory address and, accordingly, a different page table, but the neural processing apparatus according to the present embodiment can use only one shared page table even when process is switched.

Hereinafter, with reference to FIGS. 20 and 27, step S100 or step S300 of FIG. 26 will be described in detail.

Referring to FIG. 27, the PTE value and mapping type are checked from the shared page table (S310).

Specifically, referring to FIGS. 19 and 20, the shared page table (SPT) includes a page table entry (PTE) recording the start position of the physical address (PA) for each logical address (LA) and a mapping type. It may include data Md. PTE (PTE) may be used to calculate a physical address (PA) corresponding to the logical address (LA). The mapping type data Md may include information about the mapping type of the logical address LA.

Again, referring to FIG. 27, it is determined whether the mapping type is a shared type (S320). If so, the physical page is accessed according to the PTE value (S330).

Specifically, referring to FIGS. 19 and 20 , the mapping type data Md may be, for example, data indicating whether the mapping type is a shared type or a private type. The sharing type may be an address where data shared by all processes in a multi-process environment is stored.

The shared page table (SPT) allows all processes to access the shared physical page (Psh) of the same physical address (PA). In this case, the physical address (PA) of the shared physical page (Psh) may be the same as the value of PTE (PTE).

Referring again to FIG. 27 , when the mapping type is not the shared type, the physical page is accessed by correcting the PTE value with the process ID as an offset (S340).

If the mapping type is not a shared type, it may be a private type. The private type may be an address for dedicated data not shared by each process in a multi-process environment. Private physical pages Ppr may exist as many as the number of processes, that is, as many as m. That is, for example, if there are 4 processes, there may be 4 private physical pages (Ppr). The private physical page (Ppr) may calculate the physical address (PA) by adding an offset according to each process ID to the value of the PTE (PTE).

Hereinafter, a method of using a shared page table in a neural processing apparatus according to some embodiments of the present invention will be described with reference to FIGS. 19, 21, and 28. Parts overlapping with the above-described embodiment are omitted or simplified.

28 is a flowchart illustrating a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 28 , steps S310 , S320 , S330 , and S340 may be the same as those of FIG. 27 . In step S320, if the mapping type is not a shared type, it is determined whether the mapping type is a private type (S335). If so, the same process as step S340 of FIG. 27 may be performed.

If not, the final offset according to the process ID is calculated (S350), and the PTE value is corrected with the final offset to access the physical page (S360).

Specifically, referring to FIGS. 19 and 21 , the mapping type data Md may be, for example, data indicating whether the mapping type is a shared type, a private type, or an intermediate type. Accordingly, when the mapping type is neither a shared type nor a private type, it may be an intermediate type.

In the intermediate physical page (Pitm), the physical address (PA) may be calculated by adding a final offset according to each process ID to the value of the PTE (PTE). The final offset may be an offset that concurrently operating processes have the same value as each other. For example, if the first and second processes are processes operating concurrently with each other, and the third and fourth processes are processes operating concurrently with each other, then the final offsets for the first and second processes are 0, and the third and fourth processes are concurrently operating. The final offset for p may be the size of the intermediate physical page (Pitm).

For example, the first process and the third process calculate a physical address (PA) according to the PTE (PTE) value, and the second process and the fourth process calculate an offset equal to the size of the physical page in the PTE (PTE) value. By adding them, the physical address PA may be calculated.

Hereinafter, a method of using a shared page table in a neural processing apparatus according to some embodiments of the present invention will be described with reference to FIGS. 19, 20, and 29 . Parts overlapping with the above-described embodiment are omitted or simplified.

29 is a flowchart illustrating a method of allocating a physical page in a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 29, it is determined whether the mapping type is a shared type (S1100), and if so, one physical page is allocated (S1200).

Specifically, referring to FIGS. 19 and 20 , the sharing type may be an address at which data shared by all processes in a multi-process environment is stored. Since the shared physical page (Psh) stores data shared by each process in one physical page, it is possible to maximize storage efficiency by reducing the number of physical pages required by the number of processes to one.

Referring again to FIG. 29 , if the mapping type is not a shared type, contiguous physical pages as many as the number of processes are allocated (S1300).

Specifically, referring to FIGS. 19 and 20 , the mapping type data Md may be, for example, data indicating whether the mapping type is a shared type or a private type. Accordingly, when the mapping type is not a shared type, it may be a private type.

Private physical pages Ppr may exist as many as the number of processes, that is, as many as m. That is, for example, if there are 4 processes, there may be 4 private physical pages (Ppr). That is, m may be the number of processes. In this case, the private physical pages Ppr may be arranged to be consecutive to each other.

Hereinafter, a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention will be described with reference to FIGS. 19, 21, and 30 . Parts overlapping with the above-described embodiment are omitted or simplified.

30 is a flowchart illustrating a method of allocating a physical page in a method of using a shared page table of a neural processing apparatus according to some embodiments of the present invention.

Referring to FIG. 30 , steps S1100, S1200, and S1300 may be the same as those of FIG. 29 . In step S1100, if the mapping type is not a shared type, it is determined whether the mapping type is a private type (S1250). If so, it may proceed in the same way as step S1300 of FIG. 29 .

If not, a final offset according to the process ID is calculated (S1400), and consecutive physical pages as many as the number of concurrently operating processes are allocated (S1500).

Specifically, referring to FIGS. 19 and 21 , the mapping type data Md may be, for example, data indicating whether the mapping type is a shared type, a private type, or an intermediate type. Accordingly, when the mapping type is neither a shared type nor a private type, it may be an intermediate type.

Intermediate physical pages (Pitm) may exist as many as the number of simultaneous operating processes, that is, as many as k.

That is, for example, if the first and second processes are processes operating concurrently with each other, and the third and fourth processes are processes operating concurrently with each other, k may be two. Accordingly, two intermediate physical pages (Pitm) may be allocated. At this time, the intermediate physical pages (Pitm) may be arranged to be continuous with each other.

The above description is merely an example of the technical idea of the present embodiment, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but to explain, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of this embodiment should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of rights of this embodiment.

Claims (20)

A neural processing apparatus including at least one neural processor, a shared memory shared by the at least one neural processor, and a global interconnection for exchanging data between the at least one neural processor and the shared memory,
a processing unit included in the neural processor and providing a logical address;
a memory management unit that receives the logical address and converts it into a physical address; and
Including a physical memory accessible by the physical address,
The memory management unit includes a shared page table that has conversion information between the logical address and the physical address and is shared by at least one process;
The physical memory includes a plurality of intermediate physical pages;
The logical address mapping type includes an intermediate type shared by some of the at least one process, a private type not shared by the at least one process, and a shared type shared by the at least one process;
The logical address of the intermediate type is
When the at least one process is performed simultaneously, different intermediate physical pages among the plurality of intermediate physical pages are accessed;
Accessing the same intermediate physical page among the plurality of intermediate physical pages when the at least one process is not performed at the same time.
Neural Processing Unit.
According to claim 1,
The shared page table,
A PTE (Page Table Entry) recording the start position of the physical memory corresponding to the logical address;
Includes mapping type data recording the mapping type of the logical address;
The mapping type data is data for any one of the shared type, the private type, and the intermediate type.
Neural Processing Unit.
delete According to claim 1,
The physical memory further includes a shared physical page,
The logical address of the shared type is such that all of the at least one process accesses the same shared physical page in the physical memory.
Neural Processing Unit.
According to claim 4,
In the shared physical page, context data of a context to which the at least one process belongs is stored.
Neural Processing Unit.
According to claim 1,
The physical memory further includes a plurality of private physical pages,
In the private type logical address, each of the at least one process accesses a different private physical page among the plurality of private physical pages in the physical memory.
Neural Processing Unit.
According to claim 6,
The plurality of private physical pages are arranged consecutively with each other,
Neural Processing Unit.
According to claim 6,
In the plurality of private physical pages, input/output data of each of the at least one process is stored.
Neural Processing Unit.
delete delete delete According to claim 1,
The plurality of intermediate physical pages are arranged consecutively.
Neural Processing Unit.
According to claim 1,
The memory management unit further comprises a translation lookaside buffer for storing a physical address for a recently used logical address.
Neural Processing Unit.
According to claim 13,
The conversion lookaside buffer,
When a logical address identical to the stored logical address is received, converting the stored logical address into the physical address;
When a logical address that is not identical to the stored logical address is received, converting the received logical address into a physical address by referring to the shared page table.
Neural Processing Unit.
According to claim 13,
The neural processor,
comprising at least one neural core;
The neural core,
the processing unit;
an L0 memory for transmitting and receiving data to and from the processing unit;
And a load store unit for loading data into the L0 memory and transmitting data from the L0 memory.
Neural Processing Unit.
According to claim 15,
wherein the load store unit includes the translation lookaside buffer;
Neural Processing Unit.
delete delete delete delete

Images