WO2021196160A1

WO2021196160A1 - Data storage management apparatus and processing core

Info

Publication number: WO2021196160A1
Application number: PCT/CN2020/083208
Authority: WO
Inventors: 罗飞; 王维伟
Original assignee: 北京希姆计算科技有限公司
Priority date: 2020-04-03
Filing date: 2020-04-03
Publication date: 2021-10-07
Also published as: CN115380292A

Abstract

A data storage management apparatus and a processing core. The apparatus comprises: at least two random access memories (RAM); a control unit, receiving an instruction, generating and sending a control signal according to the instruction (S101); and a direct memory access controller (DMAC), achieving access of data in the RAM according to the control signal (S102). The data storage management apparatus receives and responds to an instruction sent from an external processing unit, and data is read from an external storage unit, so that data required for executing a program can be directly read from the data storage management apparatus when the external processing unit executes the program, and the external processing unit does not need to fetch a number from the external storage unit by means of a Cache, thereby eliminating the decrease of computing efficiency due to a Cache access failure, and improving the controllability of program efficiency.

Description

Data storage management device and processing core

Technical field

The invention relates to the technical field of processing cores, in particular to a data storage management device and a processing core.

Background technique

With the development of science and technology, human society is rapidly entering the era of intelligence. The important feature of the intelligent age is that people have more and more types of data, the amount of data they can obtain is larger and larger, and the requirements for the speed of data processing are getting higher and higher.

The chip is the cornerstone of data processing, and it fundamentally determines the ability of people to process data. From the perspective of application areas, there are two main routes for chips: one is a general chip route, such as CPU, etc., which can provide great flexibility, but the effective computing power is relatively low when processing algorithms in a specific field; the other is a dedicated chip Routes, such as TPU, can exert high effective computing power in some specific fields, but they have poor processing capabilities or even unable to handle flexible and versatile fields.

Due to the wide variety and huge amount of data in the intelligent era, the chip is required to have extremely high flexibility, capable of processing different fields and rapidly changing algorithms, and extremely strong processing capabilities, which can quickly process extremely large and rapidly increasing data. quantity.

Summary of the invention

The invention provides a data storage management device and a processing core, which can eliminate the decrease in calculation efficiency caused by Cache access failure, and improve the controllability of program efficiency.

The first aspect of the present invention provides a data storage management device, including: at least two random access memories RAM; a control unit for receiving instructions, generating and sending control signals according to the instructions; direct memory access controller DMAC , Used to realize the access to the data in the random access memory RAM according to the control signal.

The data storage management device provided by the embodiment of the present invention receives and responds to instructions sent from an external processing unit, and reads data from the external storage unit, so that the external processing unit can directly read from the data storage management device when executing a program To read the data required to execute the program, the external processing unit does not need to fetch the data from the external storage unit through the high-speed Cache, which eliminates the decrease in computing efficiency caused by the Cache access failure, and improves the controllability of the program efficiency.

Preferably, a direct memory access controller DMAC is configured to implement access to data in the RAM according to the control signal, including: the direct memory access controller DMAC is configured to send data from an external storage unit according to the control signal Read data in the control signal, and store the data in the random access memory RAM indicated by the control signal; or the direct memory access controller DMAC is used to obtain data from the control signal according to the control signal. The indicated random access memory RAM reads data and stores the data in an external storage unit.

Preferably, the number of the random access memory RAM indicated by the control signal is one or more.

Preferably, all the addresses of the random access memory RAM and the addresses of the external storage unit are uniformly addressed. Or, the addresses of all the random access memory RAMs are uniformly addressed.

Preferably, the access address range of the direct memory access controller DMAC is an address segment of the random access memory RAM and an address segment of an external storage unit.

According to another aspect of the present invention, there is also provided a processing core, including a processing unit, a storage unit, and the storage management device provided in the first aspect; the processing unit is configured to send instructions, and the instructions are used to instruct the The storage management device realizes the access to the data in the storage unit; the processing unit is also used to read the data required for executing the program from any random access memory RAM.

Preferably, the instructions include a fetch instruction and a storage instruction; the processing unit is used to send a fetch instruction, and the fetch instruction is used to instruct the data storage management device to fetch data from the storage unit and store the data Stored in the random access memory RAM indicated by the fetch instruction.

Preferably, the direct memory access controller DMAC is configured to send a storage completion signal after completing the fetch instruction; the processing unit is configured to issue a new fetch instruction according to the storage completion signal, and send a new fetch instruction from the The data is read from the random access memory RAM indicated by the fetch instruction.

Preferably, at the same point in time, the processing unit and the direct memory access controller DMAC access different random access memory RAMs.

Preferably, at the same time, the processing unit and the direct memory access controller DMAC access different random access memory RAMs, including: the at least two random access memory RAMs include a first random access memory RAM. Fetch the memory RAM and the second random access memory RAM; at the first time, the processing unit reads the first data from the first random access memory RAM, and the direct memory access controller DMAC sends the second random access memory to the The second data retrieved from the storage unit is written in the access memory RAM; at the second time, the processing unit reads the second data from the second random access memory RAM, and the direct The memory access controller DMAC writes the third data retrieved from the storage unit into the first random access memory RAM.

Preferably, the first random access memory RAM is a first group of random access memory RAM including a plurality of RAMs, and the second random access memory RAM is a second group of random access memory RAM including a plurality of random access memories RAM .

When the first random access memory RAM is the first random access memory RAM and the second random access memory RAM is the second random access memory RAM group, at the same time, the processing unit and the direct memory access controller DMAC are respectively Each random access memory RAM in a group is accessed, or, at the same time, the processing unit and the direct memory access controller DMAC access RAMs belonging to two groups.

According to a third aspect of the present invention, there is provided a chip including one or more processing cores provided in the second aspect.

According to a fourth aspect of the present invention, a card board is provided, which includes one or more chips provided in the third aspect.

According to a fifth aspect of the present invention, there is provided an electronic device including one or more cards provided in the fourth aspect.

According to a sixth aspect of the present invention, there is provided a data storage management method. A control unit receives an instruction, generates and sends a control signal according to the instruction; the direct memory access controller DMAC realizes the control of the random memory according to the control signal. Access to the data in the memory RAM.

According to a seventh aspect of the present invention, there is provided an electronic device, including: a memory for storing computer-readable instructions; and one or more processors for running the computer-readable instructions so that the processor The method for realizing any of the aforementioned data storage management in the sixth aspect at runtime.

According to an eighth aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute any of the aforementioned sixth aspects Describe the method of data storage management.

According to a ninth aspect of the present invention, there is provided a computer program product, which includes computer instructions, and when the computer instructions are executed by a computing device, the computing device can execute any of the data storage in the sixth aspect. Methods of management.

Description of the drawings

FIG. 1 is a schematic diagram of reading data in a processing core in the prior art;

Figure 2 is a schematic structural diagram of a data storage management device according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a processing core according to an embodiment of the present invention;

4 is a schematic diagram of the structure of a neural network according to an embodiment of the present invention;

Fig. 5 is a schematic structural diagram of a processing core according to an embodiment of the present invention.

Fig. 6 is a sequence diagram of neural network calculation performed by a processing core according to an embodiment of the present invention;

Fig. 7 is a schematic flowchart of a data storage management method according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary, and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concept of the present invention.

In neural network calculations, multi-core or many-core chips are often used. Here, the cores in the multi-core architecture all have a certain degree of independent processing capability, and have a relatively large internal storage space for storing their own programs, data, and weights.

The play of the basic computing power of a single core determines the ability of the entire chip to compute neural networks. The performance of the basic computing power of a single-core is determined by the ideal computing power and storage access efficiency of the single-core computing unit.

Different storage units have different access speeds. Generally speaking, the speed of accessing registers is the fastest, which usually takes a few hundred ps (picoseconds) for one access; followed by Static Random Access Memory (SRAM), which generally takes a few hundred ps to a few ns ( In the range of nanoseconds); again is the memory unit, that is, Double Data Rate Synchronous Dynamic Access Memory (DDR SDRAM). Generally, it takes tens to hundreds of ns for one access; the last is through Other memories accessed by the IO port, such as hard disks, have a slow access speed, generally in the ms (millisecond) level.

In the case of neural network processing, the general concern is the access of the processing unit to the memory unit. As we all know, the speed of the processing unit is very fast, and its main frequency is generally several hundred MHz (megahertz) to several GHz (gigahertz), that is, ps to ns level, and the access speed of the memory unit is tens of ns level, both There is a big difference in speed. How to solve the speed difference between the processing unit and the memory access, and effectively utilize the computing power of the processing unit, is a difficult point in modern CPU design.

Figure 1 is a schematic diagram of reading data in a processing core.

As shown in Figure 1, in the processing core, a high-speed cache is inserted between the processing unit (PU) and the storage unit memory. The PU accesses the Memory in a hierarchical and indirect manner, that is, the PU directly accesses the Cache. PU accesses Memory indirectly through Cache. Cache is a mapping of Memory, and its content is a subset of the memory content. The Cache has no independent organization space, and the address of the Cache is the same as the address of the accessed memory.

For example, when the PU is executing the program, it reads some data from the Memory through the Cache, that is, the Cache saves this part of the data. When the PU needs to use this part of the data again in a short time, the PU will directly call it from the Cache .

However, for the program executed by the PU, the Cache is transparent and has no functional significance, that is, the program cannot access the Cache alone, that is, the program thinks that the PU has retrieved data from the memory, but in fact the PU is called from the Cache. Fetched data.

The above scheme has the following shortcomings:

(1) Due to the huge amount of parameters and data used in neural network calculations, which usually far exceed the capacity of the Cache, the measures taken by the Cache to reduce the access failure rate based on the temporal and spatial local characteristics of the data will not be able to Realize, thereby greatly reducing the computing power of the processing unit.

(2) The Cache circuit is complicated, which leads to the difficulty of chip design and the higher cost of the chip.

The data storage management device provided by an embodiment of the present application will be described in detail below. In the description of the present invention, it should be noted that the terms "first", "second", "third", and "fourth" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

Fig. 2 is a schematic structural diagram of a data storage management device according to an embodiment of the present invention.

As shown in FIG. 2, the data storage management device includes: at least two random access memories RAM, a control unit, and a direct memory access controller (DMAC).

Preferably, the data storage management device may be arranged in the processing core.

Among them, at least two RAMs include RAM_0, RAM_1...RAM_N. The data storage management device has at least two RAMs, and each RAM can be accessed independently and in parallel.

Optionally, the storage capacity of all RAMs can be the same or different.

The control unit is used for receiving instructions, generating and sending control signals C_DMAC according to the instructions. Among them, the instruction is sent by the processing unit PU located outside the data storage management device.

DMAC is used to realize the access to the data in the RAM according to the control signal.

In one embodiment, the DMAC is used to implement access to data in RAM according to a control signal, including: DMAC is used to read data from an external storage unit Memory according to the control signal, and store the data in In the RAM indicated by the control signal.

In one embodiment, the DMAC is used to implement access to data in RAM according to a control signal, including: DMAC is used to read data from the RAM indicated by the control signal according to the control signal, and The data is stored in an external memory.

Preferably, the number of RAM indicated by the control signal is one or more.

Preferably, the access addresses of all RAMs are addressed uniformly. More preferably, the access addresses of the RAMs are programmed continuously to reduce the complexity of program control.

In a preferred embodiment, the addresses of all RAMs are uniformly addressed with the addresses of the external storage unit.

For example, the data storage management device contains two RAMs, namely RAM_0 and RAM_1. Among them, the address of RAM_0 is 0000H-0FFFH, the address of RAM_1 is 1000H-1FFFH, and the address of the external memory is 2000H-FFFFH.

Further preferably, the access address of the DMAC is a full address range, specifically all RAM address segments and external memory address segments.

The access address of PU is the address segment of all RAM.

In one embodiment, after the DMAC reads data from the external memory according to the control signal and stores the data in the RAM indicated by the control signal, it sends a storage completion signal to the external processing unit, and the storage is completed The signal is used to prompt the external processing unit to read data from the RAM that has just completed storage.

Fig. 3 is a schematic structural diagram of a processing core according to an embodiment of the present invention.

As shown in Fig. 3, the processing core includes a processing unit PU, a storage unit memory, and the data storage management device provided in the above-mentioned embodiment.

The PU is used to send instructions, and the instructions are used to instruct the data storage management device to implement access to the data in the memory.

The instructions include fetch instructions and store instructions.

The processing unit is configured to send instructions, where the instructions are used to instruct the data storage management device to implement access to the data in the memory, including:

The PU is used to send a fetch instruction, and the fetch instruction is used to instruct the data storage management device to read data from the memory and store the data in the RAM indicated by the fetch instruction. Preferably, when the data is stored in the RAM indicated by the instruction, the DAMC sends a storage complete signal to the PU.

In one embodiment, the PU is also used to read data needed to execute the program from any RAM.

Preferably, the PU is used to issue a new fetch instruction every time after receiving a storage completion signal sent by the DMAC, and read data from the RAM that has just completed storage.

Specifically, when the PU receives the storage completion signal sent by the DMAC, it issues a new fetch instruction, and then reads the data from the RAM that has just completed the storage, so that the DMAC reads the data from the memory according to the new fetch instruction And stored in the corresponding RAM, and the PU fetches the data from the RAM that has just completed the storage to execute the program in parallel, which improves the efficiency of the operation.

Of course, after receiving the storage completion signal sent by the DMAC, the PU can first read the data from the RAM that has just completed storage, and then issue a new fetch instruction.

In one embodiment, at the same point in time, the PU and the DMAC access different RAMs.

Specifically, the at least two RAMs include a first RAM and a second RAM. At the first time, the PU reads the first data from the first RAM, and the DMAC writes the second data retrieved from the memory into the second RAM. At the second time, the PU reads the second data from the second RAM, and the DMAC writes the third data retrieved from the memory into the first RAM.

Optionally, PU and DMAC can also access the same RAM at the same time, and RAM responds to PU and DMAC serially.

Optionally, if the RAM is a dual-port RAM, the PU and DMAC can also access the same RAM at the same time, and the dual-port RAM responds to the PU and DMAC in parallel.

It should be noted that the above-mentioned first RAM may be a first group of RAMs including a plurality of RAMs, and the second RAM may be a second group of RAMs including a plurality of RAMs.

Optionally, the number of RAMs in the first group of RAMs may be the same as or different from the number of RAMs in the second group.

When the first RAM is the first RAM group and the second RAM is the second RAM group, at the same time, the PU and DMAC can respectively access each RAM in a group, or at the same time, the PU and DMAC accesses belong to two groups. RAM for each group.

Specifically, at the first time, the PU reads the first data from the first group of RAM, and the DMAC writes the second data retrieved from the memory to the second group of RAM; at the second time, the PU reads from the second group of RAM The second data is read in the DMAC, and the DMAC writes the third data retrieved from the memory to the first group of RAM.

Optionally, when the first RAM is the first RAM group and the second RAM is the second RAM group, the processing unit or the DMAC may also time-sharing access to each RAM in the same group.

In the processing core provided by the embodiment of the present invention, the PU fetches data from the RAM and the DMAC stores the data stored in the memory in the RAM can be processed in parallel, which can further improve the computing power of the processing core, and is more suitable for neural network operations. In addition, there is no need to design a complex Cache circuit in the processing core, which saves the cost of the processing core and reduces the difficulty of chip design. Moreover, because there is no need to design a Cache circuit in the processing core, the processing unit does not need to fetch data from external memory through the high-speed cache. The decrease in computing efficiency caused by Cache access failure is eliminated, and the processing core can directly call data from the RAM of the storage management device, which improves the controllability of program efficiency.

Fig. 4 is a schematic structural diagram of a neural network according to an embodiment of the present invention.

As shown in Figure 4, the neural network has two layers, the output of the first layer is used as the input of the second layer, and the output of the second layer is the output of the entire neural network.

Fig. 5 is a schematic structural diagram of a processing core according to an embodiment of the present invention. The processing core shown in FIG. 5 is used to realize the calculation of the neural network shown in FIG. 4.

As shown in Figure 5, the processing core includes a data storage management device, a processing unit, and a storage unit.

Among them, the data storage management device includes RAM_0, RAM_1, DAMC and a control unit.

Assuming that the parameters and data of each layer of neural network are less than the capacity of a single block of RAM, that is, RAM_0 and RAM_1 can hold the parameters and data calculated by the next layer of neural network.

Set at the same point in time, PU and DMAC access different RAMs, so that the PU execution program and DMAC can store data in parallel, thereby optimizing calculation and storage efficiency. For example, at the first time, PU accesses RAM_0, and DAMC accesses RAM_1. At the second time point, PU accesses RAM_1, DAMC accesses RAM_0, and so on.

In a specific embodiment, the PU sends the instruction lls_dis, the control unit receives the instruction, and generates a control signal C_DMAC and sends it to the DMAC. The DMAC reads the data indicated by the instruction from the memory according to the instruction, and stores the data in the instruction indicated by the instruction. In RAM_0, when the DMAC has finished storing the data, it sends a storage completion signal to the PU; the PU receives the storage completion signal and issues a new instruction to instruct the DMAC to read new data from memory and store it in RAM_1, and then Read data from RAM_0. When the DMAC stores the data in RAM_1, it sends a signal that the storage is complete. The PU issues an instruction again and then reads the data from RAM_1. The instruction issued again instructs the DMAC to read new data from memory and store it in RAM_0. In this way, DMAC storage data and PU read data realize parallel processing, so that both calculation and storage can maximize efficiency.

Fig. 6 is a sequence diagram of a neural network calculation performed by a processing core according to an embodiment of the present invention.

As shown in Figure 6, at t0, when the PU reads data from RAM_0, that is, RAM_0 is occupied by the PU executing the program of the first layer of the neural network, and the DMAC writes the data read from memory to RAM_1. ; At t1, when RAM_1 is occupied by the program of the second layer of the neural network executed by the PU, at the same time the DMAC writes the data read from memory to RAM_0. By setting the difference between the PU and DMAC access addresses, there is no need for the PU to fetch data from the memory, which can reduce the complexity of the program and increase the computing power of the processing core.

It should be understood that the program executed by the PU at t2 can be set to be the same as the program executed at t1, that is, the calculation of the first layer of the same neural network is also performed at t2, or the program executed by the PU at t2 can be set It is different from the program executed at t1, and the present invention is not limited to this.

According to an embodiment of the present invention, there is provided a chip including one or more processing cores provided in the foregoing embodiments.

According to an embodiment of the present invention, a card board is provided, which includes one or more chips provided in the foregoing embodiments.

According to an embodiment of the present invention, there is provided an electronic device, including one or more of the card boards provided in the foregoing embodiments.

FIG. 7 is a data storage management method provided by an embodiment of the present invention. The method includes: step S101-step S102;

Wherein, in step S101, the control unit receives an instruction, generates and sends a control signal according to the instruction.

Step S102, the direct memory access controller DMAC realizes the access to the data in the RAM according to the control signal.

The DMAC implements the access to the data in the RAM according to the control signal, including: the DMAC reads data from the external memory according to the control signal, and stores the data in the RAM indicated by the control signal.

The DMAC implements the access to the data in the RAM according to the control signal, including: the DMAC reads data from the RAM indicated by the control signal according to the control signal, and stores the data in an external memory.

It is understandable that the data storage management method provided in this embodiment is executed by the data storage management device provided in the foregoing embodiment of the present invention, and therefore, the same features are not described here.

An embodiment of the present invention provides a schematic flowchart of a method for processing core processing data.

The method includes: step S201-step S202,

Step S201, the processing unit sends a fetch instruction;

Step S202: The data storage management device reads data from the storage unit according to the fetch instruction, and stores the data in the RAM of the data storage management device indicated by the fetch instruction.

In a preferred embodiment, the data storage management device sends a storage completion signal after storing the data in the RAM of the data storage management device indicated by the fetch instruction.

Each time the processing unit receives a storage completion signal sent by the DMAC, it issues a new fetch instruction, and reads data from the RAM indicated by the fetch instruction just completed.

In one embodiment, at the same point in time, the processing unit and the direct memory access controller DMAC access different said RAMs.

Specifically, at the first time, the processing unit reads the first data from the first RAM, and the DMAC writes the second data retrieved from the memory into the second RAM. At the second time, the PU reads the second data from the second RAM, and the DMAC writes the third data retrieved from the memory into the first RAM.

It is understandable that the method for processing core processing data provided in this embodiment is executed by the processing core provided in the foregoing embodiment of the present invention, and therefore, the same features are not described here.

According to an embodiment of the present invention, there is provided an electronic device, including: a memory for storing computer-readable instructions; and one or more processors for running the computer-readable instructions so that the processor The method of data storage management of the foregoing embodiment is implemented at runtime.

According to one embodiment of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the data storage management method of the foregoing embodiment .

According to an embodiment of the present invention, a computer program product is provided, which includes computer instructions, and when the computer instructions are executed by a computing device, the computing device can execute the data storage management method of the foregoing embodiment.

It should be understood that the foregoing specific embodiments of the present invention are only used to exemplarily illustrate or explain the principle of the present invention, and do not constitute a limitation to the present invention. Therefore, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundary of the appended claims, or equivalent forms of such scope and boundary.

Claims

A data storage management device, characterized in that it comprises:

At least two random access memory RAMs;

The control unit is configured to receive instructions, and generate and send control signals according to the instructions;

The direct memory access controller DMAC is used to implement the access to the data in the random access memory RAM according to the control signal.
The device according to claim 1, wherein the direct memory access controller DMAC is configured to implement access to data in the random access memory RAM according to the control signal, comprising:

The direct memory access controller DMAC is configured to read data from an external storage unit according to the control signal, and store the data in the random access memory RAM indicated by the control signal; or

The direct memory access controller DMAC is configured to read data from the random access memory RAM indicated by the control signal according to the control signal, and store the data in an external storage unit.
The device according to claim 1 or 2, wherein the number of the random access memory RAM indicated by the control signal is one or more.
The device according to any one of claims 1-3, wherein all the addresses of the random access memory RAM and the addresses of the external storage unit are uniformly addressed; or, all the random access memories are The address of the memory RAM is uniformly addressed.
The device according to any one of claims 1 to 4, wherein the access address range of the direct memory access controller DMAC is all the address segments of the random access memory RAM and addresses of external storage units part.
A processing core, characterized by comprising a processing unit, a storage unit, and the data storage management device according to any one of claims 1-5;

The processing unit is configured to send an instruction, the instruction being used to instruct the data storage management device to implement access to the data in the storage unit;

The processing unit is also used to read data required for executing the program from any random access memory RAM.
The processing core according to claim 6, wherein:

The instructions include fetch instructions and store instructions;

The processing unit is configured to send a fetch instruction, and the fetch instruction is used to instruct the data storage management device to fetch data from the storage unit and store the data in the random access indicated by the fetch instruction Memory RAM.
The processing core according to claim 7, wherein the direct memory access controller (DMAC) is configured to send a storage completion signal after completing the fetch instruction;

The processing unit is configured to issue a new fetch instruction according to the storage completion signal, and read data from the random access memory RAM indicated by the fetch instruction.
The processing core according to any one of claims 6-8, wherein at the same point in time, the processing unit and the direct memory access controller DMAC access different random access memory RAMs.
7. The processing core according to claim 7, wherein the processing unit and the direct memory access controller DMAC access different random access memory RAMs at the same time, comprising:

The at least two random access memory RAMs include a first random access memory RAM and a second random access memory RAM;

At the first time, the processing unit reads first data from the first random access memory RAM, and the direct memory access controller DMAC writes to the second random access memory RAM from the The second data retrieved from the storage unit;

At the second time, the processing unit reads the second data from the second random access memory RAM, and the direct memory access controller DMAC writes the slave data to the first random access memory RAM. The third data retrieved from the storage unit.