WO2020001665A2

WO2020001665A2 - On-chip cache and integrated chip

Info

Publication number: WO2020001665A2
Application number: PCT/CN2019/112380
Authority: WO
Inventors: 张乾龙
Original assignee: 华为技术有限公司
Priority date: 2019-10-21
Filing date: 2019-10-21
Publication date: 2020-01-02
Also published as: WO2020001665A3; CN114556335A

Abstract

An on-chip cache and an integrated chip, used for solving the problems in the prior art of low page storage space utilisation and wasted on-chip cache storage space. The on-chip cache comprises: a storage unit, used for storing a first page, the first page comprising a first portion data unit and a second portion data unit, first tag information corresponding to the first portion data unit having a first priority, second tag information corresponding to the second portion data unit having a second priority, the second priority being lower than the first priority.

Description

On-chip cache and integrated chip

Technical field

The present application relates to the field of chip technology, and in particular, to an on-chip cache and an integrated chip.

Background technique

At present, with the development of manufacturing processes and technologies, the on-chip cache capacity of processors is getting larger and larger, and the implementation media of on-chip caches are becoming more and more diverse, such as static random-access memory (SRAM) and enhanced dynamics. The random access memory (enhanced dynamic random access memory, eDRAM) can achieve a cache of 128MB or more, and the 3D packaged SRAM can further increase the storage density of ordinary SRAM to achieve greater storage capacity and access bandwidth. 2.5D or 3D packaged DRAM (ie, on-chip memory) as on-chip cache capacity can reach 16GB, and is expected to further increase. The larger the cache capacity, the greater the impact of its management efficiency on processor performance. In the embodiments of the present application, both the on-chip memory and the on-chip cache refer to a 2.5D or 3D packaged DRAM.

Specifically, the access bandwidth problem of the memory system has become one of the main reasons hindering the improvement of processor performance. On-chip large-capacity cache is a new technology proposed by the industry to solve the access bandwidth problem of memory systems. The large-capacity on-chip cache can package memory dies on-chip through through silicon via (TSV) technology, thereby achieving the effect of improving the access bandwidth of the memory system. Taking 2.5D or 3D packaged DRAM as an example, some data show that the bandwidth of on-chip memory can reach 4 to 8 times of the off-chip double-rate dynamic random access memory (DDR) DRAM (DDR DRAM).

Generally, on-chip caches can work in three modes: cache mode, flat mode, and mixed mode. When working in cache mode, the on-chip cache can be used as a cache for off-chip memory; when working in flat mode, the on-chip cache is used as ordinary memory; when working in mixed mode, some on-chip caches are used as off-chip memory caches, and some on-chip caches are used as ordinary RAM. This application improves the scenario where the on-chip cache is used as the cache of off-chip memory.

When the on-chip cache is used as the cache of off-chip DRAM, the concept of Footprint cache is proposed. The design idea is: Cache allocation is based on page granularity, but only the footprint granularity data is stored in the on-chip cache as needed. Data that is not accessed does not enter the on-chip cache to save bandwidth for accessing off-chip memory. As shown in FIG. 1, it is a schematic diagram of a page stored in an on-chip cache. In FIG. 1, each small box represents a footprint (for example, it can be a block). The gray-filled footprint stores data, and the white-filled footprint is blank, and no data is stored. That is, the gray-filled footprint can store frequently accessed data on a page, and the data corresponding to the white-filled portion of the page is accessed less frequently, so it is not stored in the on-chip cache.

With the solution shown in FIG. 1, although the access efficiency of the memory system can be improved and unnecessary waste of memory bandwidth is reduced, the utilization of page storage space is not high, resulting in waste of on-chip cache storage space.

Summary of the invention

The embodiments of the present application provide an on-chip cache and an integrated chip, which are used to solve the problems of low utilization of page storage space and waste of on-chip cache storage space in the prior art.

In a first aspect, an embodiment of the present application provides an on-chip cache, including: a storage unit configured to store a first page. The first page includes a first part of the data unit and a second part of the data unit. The tag information has a first priority, and the second tag information corresponding to the second part of the data unit has a second priority, and the second priority is lower than the first priority.

With the on-chip cache provided in the first aspect, the data unit stored in the first page can be divided into two parts, and each part of the data unit corresponds to one tag information. In the off-chip memory, one tag information corresponds to one page in the off-chip memory, that is, in the on-chip cache provided in the first aspect, each part of the data unit is used to store a data unit of one page in the off-chip memory. The on-chip cache provided in the first aspect can store data units that were originally stored in two pages in off-chip memory to the first page. In the prior art, the data units stored in one page in the on-chip cache are all from the same page in off-chip memory, that is, the data units stored in the on-chip cache correspond to one tag information, and the data provided in the first aspect The first page of the on-chip cache stores two data units corresponding to the tag information. That is to say, the on-chip cache provided in the first aspect can compress and store data originally stored in two pages to one page, thereby saving the storage space of the on-chip cache.

In a possible design, the storage unit is further configured to store index information of the first page, where the index information of the first page includes index information of the first part of the data unit and index information of the second part of the data unit.

With the above solution, the data unit stored in the first page can be indexed according to the index information of the first page.

Specifically, the index information of the first part of the data unit may include first tag information and first valid bit information, and the index information of the second part of the data unit may include second tag information and second valid bit information.

With the above solution, the priority of the first part of the data unit and the valid data unit in the first page can be determined by the index information of the first part of the data unit; the priority of the second part of the data unit can be determined by the index information of the second part of the data unit And valid data units on the first page.

In a possible design, the on-chip cache may further include: a storage controller, configured to set a priority of the second tag information to a first priority, and set the first tag information or a The priority of the third tag information is set to the second priority.

With the above solution, the priorities of the first part of the data unit and the second part of the data unit can be reversed.

In a possible design, the first page further includes a third part of the data unit, and the fourth tag information corresponding to the third part of the data unit has a third priority, and the third priority is lower than the second priority.

With the above solution, multiple partial storage units can be stored in the on-chip cache, and each partial storage unit has a different priority.

In a second aspect, an embodiment of the present application provides an integrated chip, including: an on-chip cache for storing a first page, where the first page includes a first part data unit and a second part data unit, and a first part corresponding to the first part data unit The tag information has a first priority, and the second tag information corresponding to the second part of the data unit has a second priority, and the second priority is lower than the first priority.

With the integrated chip provided in the second aspect, the first part of the data unit corresponding to the first tag information and the second part of the data unit corresponding to the second tag information can be stored in the first page. Compared with the prior art solution of storing only one data unit corresponding to one tag information on one page, the integrated chip provided in the second aspect can be used to compress and store the data that was originally stored in multiple pages in the on-chip cache China to one page. Pages, thereby saving on-chip cache storage space.

In a possible design, the on-chip cache is further configured to store index information of the first page, where the index information of the first page includes index information of the first part of the data unit and index information of the second part of the data unit.

Specifically, the index information of the first part of the data unit includes first tag information and first valid bit information, and the index information of the second part of the data unit includes second tag information and second valid bit information.

In a possible design, the integrated chip provided in the second aspect further includes: a processor for sending a first access instruction, the first access instruction for requesting access to the first data unit, and the first data unit corresponds to the first The tag information; the on-chip cache is further configured to: determine that the first data unit is stored in the on-chip cache according to the first significant bit information; and send the first data unit to the processor.

With the above solution, when the processor requests to access the data unit corresponding to the first tag information, if the data unit is stored in the on-chip cache, the data unit is directly returned to the processor without having to obtain it from off-chip memory. This improves data access efficiency.

In a possible design, the processor is further configured to send a second access instruction, the second access instruction is used to request access to the second data unit, and the second data unit corresponds to the second tag information; the on-chip cache is further configured to: Determine that the second data unit is stored in the on-chip buffer according to the second significant bit information; and send the second data unit to the processor.

With the above solution, when the processor requests to access the data unit corresponding to the second tag information, if the data unit is stored in the on-chip cache, the data unit is directly returned to the processor without having to obtain it from off-chip memory. This improves data access efficiency.

In a possible design, the on-chip cache is further configured to determine whether to set the priority of the second tag information to the first priority according to the number of times the second data unit is accessed in a unit time, and set the first tag The priority of the information or the third tag information having the first priority is set to the second priority.

With the above solution, the priorities of the second label information and the first label information can be replaced.

In a possible design, the processor is further configured to: send a third access instruction, the third access instruction is used to request access to the third data unit, and the third data unit corresponds to the second tag information; the on-chip cache is further configured to: It is determined that the third data unit is not stored in the on-chip cache according to the second significant bit information; the processor is further configured to: read the third data unit from the off-chip memory; the on-chip cache is further configured to: store the third data unit.

With the above solution, when the processor requests to access the data unit corresponding to the second tag information, if the data unit is not stored in the on-chip cache, the corresponding data unit is read from the off-chip memory, and the on-chip cache stores the data unit storage.

Specifically, when the on-chip cache stores the third data unit, it is specifically configured to store the third data unit in the first page or the second page.

Among them, the second page and the first page are data units on the same Way in the Data Array.

Further, the on-chip cache is further configured to: set the valid position corresponding to the third data unit in the second significant bit information to be valid; set the priority of the second tag information to the first priority, and set the first tag information Or the priority of the fourth tag information having the first priority is set to the second priority.

The second tag information can be regarded as the victim role of the first tag information. When the data unit corresponding to the second tag information is not stored in the on-chip cache, and the corresponding storage location of the data unit is occupied by the first tag information, then When the corresponding data is read into the on-chip cache, the second tag information having the second priority needs to be upgraded to the first priority. At this time, one of the tag information having the first priority in the Tag Array must be downgraded to the second priority. This downgraded tag information may be the first tag information or the fourth tag information.

In addition, the integrated chip provided in the second aspect may further include: a tag buffer, configured to store the first tag information and the second tag information.

With the above solution, when the processor issues an access instruction, it can first look up the corresponding information in the tag buffer to determine whether the data accessed by the processor is stored in the on-chip cache, thereby improving data access efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an on-chip cache storage page provided by the prior art; FIG.

2 is a schematic structural diagram of a first integrated chip according to an embodiment of the present application;

3 is a schematic diagram of a page in an on-chip cache provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a label array and a data array according to an embodiment of the present application; FIG.

5 is a schematic structural diagram of a second integrated chip according to an embodiment of the present application;

6 is a schematic structural diagram of an on-chip cache according to an embodiment of the present application;

7 is a schematic structural diagram of a third integrated chip according to an embodiment of the present application;

8 is a schematic flowchart of a first priority replacement process according to an embodiment of the present application;

9 is a schematic flowchart of a second priority replacement process according to an embodiment of the present application;

10 is a schematic diagram of a data access process according to an embodiment of the present application;

FIG. 11 is a schematic flowchart of a third priority replacement process according to an embodiment of the present application.

detailed description

In the following, the application scenarios of the embodiments of the present application are first introduced.

The embodiment of the present application can be applied to the integrated chip shown in FIG. 2. The integrated chip includes a processor, a memory controller (MC) and an on-chip cache. In addition, the integrated chip is also connected to off-chip memory. Among them, the processor is used to initiate data access requests and perform data processing; the memory controller is used to control the data interaction between the processor and the on-chip cache, and between the processor and off-chip memory; the off-chip memory stores a large amount of data, and the on-chip memory The cache can be regarded as an on-chip cache outside the chip.

In the embodiment of the present application, the on-chip cache is used as a cache of off-chip memory. In other words, the on-chip cache stores some data in off-chip memory. After the processor issues an access instruction, if the data accessed by the processor is stored in the on-chip cache, the data is directly returned from the on-chip cache; if the data accessed by the processor is not stored in the on-chip cache, it needs to be accessed from off-chip The data is accessed internally, and the retrieved data is stored in the on-chip cache for the next access hit.

In the on-chip cache, the cache space is allocated by the cache for pages. A page can be divided into multiple data units. In the embodiment of the present application, the on-chip cache can store the page that has been previously processed by the processor. The accessed data unit is for subsequent re-access by the processor; that is, not all data units in the page may be cached in the on-chip cache.

Specifically, the data unit stored in one page of the on-chip cache may be divided into multiple parts, and each part of the data unit corresponds to one tag information (Tag). Among them, each part of the data unit is used to store a data unit of a page in the off-chip memory. In other words, the on-chip cache can store data units that were originally stored in multiple pages in off-chip memory to one page of the on-chip cache. Taking the data unit stored in one page of the on-chip cache as an example, the schematic diagram of one page in the on-chip cache can be shown in FIG. 3. In FIG. 3, one page of the on-chip cache stores a data unit corresponding to the first tag information (Tag1) and a data unit corresponding to the second tag information (Tag2). The data unit corresponding to Tag1 is stored in one page in off-chip memory, and the data unit corresponding to Tag2 is stored in another page in off-chip memory. In the prior art, the data units stored in a page in the on-chip cache are all from the same page in the off-chip memory, that is, the data units stored in the on-chip cache correspond to a tag. In the embodiment of the present application, A page in the on-chip cache stores data units corresponding to multiple tags. That is, in the embodiment of the present application, data originally stored in multiple pages in the off-chip memory may be compressed and stored in one page of the on-chip cache, thereby saving the storage space of the on-chip cache.

In the on-chip cache, in addition to pages, index information of the pages is also stored. Specifically, the index information of the page can be stored in a tag array, and the page can be stored in a data array. The index information of the pages in the tag array corresponds to the pages in the data array. As the name suggests, the Tag Array and Data Array are stored in the form of an array. Take the label array as an example, each label array can be an m * n array, and each element in the array is the index information of the page. Specifically, the row vectors in the m * n array can be called Set, and the column vectors in the m * n array can be called Way. In addition, in the embodiment of the present application, if each row vector includes N elements, this storage structure may be referred to as an N-way group connection. For example, the on-chip cache can use a four-way set of connected storage structures, that is, each row vector contains four elements, and in the label array, one of the four elements contained in each row vector represents the index of a page Information. In the data array, one of the four elements contained in each row vector represents a page.

A specific example of a tag array and a data array can be shown in FIG. 4. As can be seen from Figure 4, the elements in the TagArray correspond to the elements in the DataArray one-to-one, and the elements in the TagArray are used to indicate the index information of the corresponding pages in the DataArray. In the example in FIG. 4, the on-chip cache adopts a four-way group connection method. In practical applications, the on-chip cache can also use an eight-way group connection method and other architectures, which are not limited in the embodiment of the present application.

The following describes the index information of the pages stored in the label array. As mentioned above, in the embodiment of the present application, one page of the on-chip cache stores data units in multiple pages of off-chip memory, that is, the data unit stored in one page of the on-chip cache can be divided into multiple section. Each part of the data unit corresponds to a set of index information, and multiple sets of index information constitute the index information of a page cached in the slice. For example, the data unit stored in the on-chip cache is divided into two parts, A and B, then the index information of the page includes two sets of index information, one set of index information is used for index A, and the other set of index information is used for index B. section. Specifically, each set of index information may include the foregoing tag information (Tag). In addition to the tag information (Tag), it can also include the following information: Overall valid bit information (Valid), which is used to indicate whether the entire tag is valid. If the tag is invalid, all data units corresponding to the tag are not accessible; least recently used (least Recently used (LRU) information is used to indicate the least recently used data unit; Dirty Bits information is used to indicate whether the data of the data unit stored in the on-chip cache is dirty. If Dirty Bits is some The bit is set to 0, indicating that the corresponding data unit is clean data. When replacement occurs, it can be invalid without writing back to off-chip memory. Otherwise, if some bits of Dirty Bits are set to 1, the corresponding dirty data needs to be written when replacement occurs. Back to off-chip memory. Valid bits information is used to indicate valid data units in the page, that is, data is stored in valid data units in the page, and data is not stored in invalid data units. For example, in the example in FIG. 3, blank The valid bit information corresponding to the grid is invalid, and no data is stored in it. By looking up the ValidBits information, you can determine whether the data unit accessed by the processor is stored in the on-chip cache. For example, the processor initiates an access instruction requesting access to a certain data unit corresponding to Tag1. By searching the TagArray in the on-chip cache, the index information of Tag1 (which can be called a Tag1 hit) can be found. At this time, further judgment and processing can be performed. The valid bit information corresponding to the data unit that the device requests to access. If it is valid, it means that the data unit is stored in the on-chip cache. If it is invalid, it means that the data unit is not stored in the on-chip cache and needs to be accessed off-chip memory. Of course, in determining whether the data unit is stored in the on-chip cache and whether it can be accessed, the foregoing information such as the overall valid bit information (Valid) and dirty bit (Dirty Bits) information must also be considered. This example is only for the Valid The use of Bits information is introduced, and other details will not be repeated here.

It should be noted that, in each set of index information, the corresponding significant bit information cannot be valid at the same time. This is because, in the embodiment of the present application, data units in multiple pages in off-chip memory are stored in one page in the on-chip cache, and a page in the off-chip memory has the same storage space as a page in the on-chip cache. , Then for a storage location for storing a data unit in a page of the on-chip cache, if the storage location is used to store a data unit corresponding to Tag1 (in the index information of Tag1, the storage location corresponds to Valid Bits is valid), then this storage location cannot be used to store the data unit corresponding to Tag2 (in the index information of Tag2, the Valid Bits corresponding to this storage location is invalid). Taking a page shown in FIG. 3 as an example, the storage position of the second row and the first column is used to store the data unit corresponding to Tag1, and then the storage location cannot simultaneously store the data unit corresponding to Tag2. Reflected in ValidBits, the fifth bit in ValidBits corresponding to Tag1 is valid, and the fifth bit in ValidBits corresponding to Tag2 is invalid.

In addition, each set of index information can also include access bit information (Reference bits), which is used to record the historical information of which data units have been accessed. When the data unit corresponding to the tag is no longer stored in the on-chip cache, it needs to be written back to off-chip memory. At this time, it is necessary to write the access bit information back to the off-chip memory storage. In this case, when the data unit corresponding to the tag is stored in the on-chip cache again, the pre-fetch of the data unit can be performed through the access bit information.

As mentioned earlier, since a page stores data units corresponding to multiple tags in the embodiment of the present application, for each tag in the corresponding page, there is a set of the above including Tag, Valid, LRU, Dirty Bits, Valid Bits and Reference Bits index information. In FIG. 4, a page includes data units corresponding to two tags as an example, and the index information of the page includes two groups. Specifically, among the two tags, a tag with a higher priority is called a Prime Tag, and a tag with a lower priority is called a Sub Tag. The index information of the page includes index information corresponding to Prime Tag and index information corresponding to Sub Tag.

In addition, in the example of FIG. 4, the data unit corresponding to the tag (Prime tag) with a higher priority can be stored in the on-chip cache preferentially. As mentioned above, for a storage location for storing a data unit in a page of the on-chip cache, the storage location can only store a data unit corresponding to a tag, and in determining which storage location is used to store which tag corresponds to When it comes to data units, the issue of storage priority is involved. For example, if a data unit corresponding to a SubTag wants to be stored in the on-chip cache, and the storage location of the data unit is occupied by a data unit corresponding to a PrimeTag, then the data unit corresponding to the SubTag cannot be stored in In the on-chip cache; for example, if a data unit corresponding to the Prime Tag is to be stored in the on-chip cache, and the storage location of the data unit is occupied by the data unit corresponding to the Sub Tag, then the data unit corresponding to the Prime Tag is This storage location can be occupied, and the data unit corresponding to the SubTag is kicked out of the on-chip cache.

In addition, in the integrated chip shown in FIG. 2, the memory controller may further include a tag buffer (Tag buffer, TB), which is used to cache part of the data in the Tag Array stored in the on-chip cache to the TB (in order to save The storage space of the memory controller considers that only part of the data of the TagArray is stored to the TB, and not all of the data of the TagArray is stored to the TB), thereby improving the access speed of the page index information. Specifically, for the specific content of the Tag Array stored in the TB, please refer to the related description of the Tag Array stored in the on-chip cache, which is not repeated here.

In the case where part of the data in the TagArray is stored in the TB, after the processor issues an access instruction, it can first determine whether the accessed data is stored in the on-chip cache by looking up the TB. If the accessed data is determined to be stored in the on-chip cache by looking up the TB, you can access the data directly from the on-chip cached Data Array without searching the Tag Array in the on-chip cache, thereby improving the access to the index information of the page. Speed; if it is determined by searching TB that the accessed data is not stored in the on-chip cache, there are two cases, the first case is that the accessed data is not actually stored in the on-chip cache, and the second case is that the TB is only Store some of the data in the Tag Array cached on-chip, and the index information corresponding to the accessed data is not stored in the TB. At this time, you need to go to the Tag Array in the on-chip cache to look up to determine that the access belongs to the two above. Which of these situations.

It is worth noting that the data stored in the TB may be newer than the data stored in the on-chip cache, so when a data update occurs in the TB, the updated data needs to be written back to the on-chip cache. In addition, in order to ensure that the data in the TB is correct, when the page index information in the on-chip cache or off-chip memory changes, the data in the TB must be updated synchronously.

Similar to the Tag Array stored in the on-chip cache, the Tag Array stored in the TB is also stored in the form of an array. For example, it can be an m * n array, and each element in the array is the index information of the page. Specifically, the row vectors in the m * n array can be called Set, and the column vectors in the m * n array can be called Way. In addition, if each row vector includes N elements, this storage structure of TB can be referred to as an N-way group connection. Exemplarily, the TB may adopt a four-way group connected storage structure, that is, each row vector contains four elements.

Exemplarily, in the integrated chip provided in the embodiment of the present application, data stored in the TB and the on-chip cache may be as shown in FIG. 5. In the example in FIG. 5, the on-chip cache and the off-chip memory share a memory controller. In practical applications, the on-chip cache and the off-chip memory can also be controlled by two controllers, which are not limited in this application.

In addition, in the TB storage array shown in FIG. 5, the meaning of the tag information (Tag) is the same as that of the Prime Tag or the Sub Tag in the Tag Array of the on-chip cache, and the value (Value) represents the Tag Array of the on-chip cache Valid, LRU, Valid Bits, Dirty Bits, and Reference Bits. It should be noted that, in the example of Figure 5, in order to simplify the management of TB, the status of PrimeTag and SubTag is equal from TB, so each item (Tag + Value) in TB is targeted to a PrimeTag or a SubTag. Instead of recording a pair of PrimeTag and SubTag at the same time as the on-chip cache. Therefore, one element in the on-chip cached TagArray corresponds to two sets of Tag + Value stored in TB. Each set of Tag + Value can be regarded as one element in the storage array of TB, so it is not difficult to see that In the example in FIG. 5, the TB is a four-way group connected storage structure, that is, each row vector includes four elements.

In the example of FIG. 5, a single cache set includes four page index information, each page index information occupies 64B of storage space, and is used to store index information of a 4KB page. . In addition, historical data information is inserted after the page index information, which is used to record the relevant information of the page that was kicked out of the on-chip cache (including the tag information Tag, the count value of the number of visits, and the historical footprint of the visit), so that The next time the page is stored in the on-chip cache, the relevant information of the page can be obtained directly.

Specifically, the information included in the page index information in FIG. 5 is basically the same as the example in FIG. 4. The difference is that the example in FIG. 5 further includes a role flip bit. As mentioned earlier, multiple tags corresponding to a page are prioritized. The data unit corresponding to a tag with a higher priority can be stored in the on-chip cache first. In a specific example, the priority of a PrimeTag is higher than that of a SubTag. Priority, but in some cases (for example, the frequency of access to the data corresponding to the SubTag) needs to be replaced by the priority. At this time, in order to avoid the overhead caused by data dump, you can use this role to flip the bit. Instructions. For example, when the role rollover bit is 0 (that is, by default), the PrimeTag has a higher priority; when the role rollover bit is 1, the SubTag has a higher priority.

It should be noted that, in the examples in the embodiments of the present application, the on-chip cache is integrated on the chip (that is, the on-chip cache is located inside the integrated chip) as an example. In practical applications, an off-chip cache may also be provided outside the integrated chip. The off-chip cache adopts the same storage structure design as the on-chip cache provided in the embodiment of the present application and implements the same implementation as the on-chip cache provided in the embodiment of the present application. Function, the off-chip cache should also be regarded as falling within the protection scope of the embodiment of the present application.

In addition, in the integrated chip, the MC can be used as the controller of the on-chip cache and off-chip memory to realize the data interaction between the processor and the on-chip cache and off-chip memory. In actual applications, the on-chip cache can also be separately configured with a storage controller to control data access in the on-chip cache. For example, the storage controller in the on-chip cache can access the data unit corresponding to the tag. Statistics can be used to determine whether the foregoing priority replacement is required (SubTag is promoted to PrimeTag, which can also be referred to as SubTag upgrade).

In order to solve the problem of wasted storage space of large-capacity caches proposed in the background, an embodiment of the present application provides an on-chip cache and an integrated chip, which intends to optimize the storage efficiency of the above-mentioned large-capacity cache system to further improve cache management efficiency. In the embodiment of the application, a 2.5D or 3D packaged on-chip cache is taken as an example for introduction. In addition, the embodiment of the present application is also applicable to on-chip caches made of other media (such as SRAM, 3D-SRAM, and eDRAM mentioned in the background art).

Hereinafter, embodiments of the present application will be described in detail with reference to the drawings.

An embodiment of the present application provides an on-chip cache. As shown in FIG. 6, the on-chip cache 600 includes a storage unit 601, which is configured to store a first page. The first page includes a first part of the data unit and a second part of the data unit. The first tag information corresponding to the first part of the data unit has a first priority, the second tag information corresponding to the second part of the data unit has a second priority, and the second priority is lower than the first priority.

It is not difficult to see that the data unit stored in the first page can be divided into two parts, and each part of the data unit corresponds to a tag information (Tag). In off-chip memory, a tag corresponds to a page in off-chip memory, that is, in on-chip cache 600, each part of the data unit is used to store a page of data units in off-chip memory. On-chip cache 600 can The data unit originally stored in the two pages in the off-chip memory is stored in the first page. In the prior art, the data units stored in a page in the on-chip cache are all from the same page in the off-chip memory, that is, the data units stored in the on-chip cache correspond to a tag. In the embodiment of the present application, One page (that is, the first page) of the on-chip cache 600 stores data units corresponding to multiple tags. In other words, the on-chip cache 600 shown in FIG. 6 can compress and store data originally stored in two pages to one page, thereby saving the storage space of the on-chip cache.

For example, a possible structure of the first page may be shown in FIG. 3, where Tag1 may be regarded as the first tag information, and Tag2 may be regarded as the second tag information.

Here, the first priority and the second priority are briefly described. It can be understood that in the off-chip memory, the data corresponding to Tag1 can occupy the storage space of an entire page, and the data corresponding to Tag2 can also occupy the storage space of an entire page. However, due to the limited storage space of the on-chip cache, the on-chip cache may store only part of the data in the off-chip memory (such as frequently accessed data or data that has been accessed), then the data unit corresponding to Tag1 It may be only partially stored in the first page, as is the data unit corresponding to Tag2. This will cause a problem. For a certain storage location in the first page (for example, the storage location of the first row and the first column in FIG. 3), if Tag1 in the off-chip memory corresponds to the data unit stored in that location and Tag2 corresponds to The data units stored in this position all want to be stored in the first page of the on-chip cache. At this time, there must be a priority order to determine whether the data unit corresponding to Tag1 is stored in this location, or Tag2 is corresponding. The data unit is stored in this location. In the embodiment of the present application, the priority of the first tag information (Tag1) is set higher than the priority of the second tag information (Tag2). When encountering the above situation, the data unit corresponding to Tag1 is preferentially stored in the on-chip cache. in.

In addition, the storage unit 601 is further configured to store index information of the first page. Specifically, the index information of the first page includes index information of the first part of the data unit and index information of the second part of the data unit. Exemplarily, in the example of FIG. 4, the first half of the index information of the page is PrimeTag + Valid + LRU + ValidBits + DirtyBit + ReferenceBits can be regarded as the index information of the first part of the data unit, and the second half of SubTag + Valid + LRU + Valid Bits + Dirty Bits + Reference Bits can be regarded as the index information of the second part of the data unit.

Specifically, the index information of the first part of the data unit may include first tag information (such as PrimeTag) and first valid bit information (for example, Valid bits), and the index information of the second part of the data unit may include second tag information (for example, SubTag) And second significant bit information (for example, Valid Bits).

The first significant bit information and the second significant bit information are described in detail below. Taking the first significant bit information as an example, the first significant bit information may be used to indicate a valid data unit in the first page. Still taking FIG. 4 as an example, in the example of FIG. 4, the first page includes 4 * 16 data units, then the first significant bit information can be represented by 64b data. If the corresponding position stores the first part of the data unit, the The corresponding bit field in the first significant bit information is set to valid (for example, it can be set to 1). For example, in the 64 positions of the first page, the first position stores the data unit corresponding to Tag1, then the 64b The first position in the first significant bit information is valid. The meaning of the second significant bit information is similar to that of the first significant bit information, and details are not described herein again.

In addition, the on-chip cache 600 may further include a storage controller 602, configured to set the priority of the second tag information to the first priority, and set the first tag information or the third tag information having the first priority to The priority is set to the second priority.

As mentioned above, in order to select and manage the storage of the first page, the first priority of the first part of the data unit is higher than the second priority of the second part of the data unit. However, in some cases (for example, the second part of the data unit is frequently accessed or the number of accesses is high), priority replacement is required. At this time, the priority of the second tag information needs to be set to the first priority. The priority of the first tag information or the third tag information having the first priority is set as the second priority. In this case, the priority of the second label information is higher than the priority of the first label information, and then the data unit corresponding to the second label information can be stored in the on-chip cache preferentially.

Specifically, when the priority replacement is performed, the replacement may be performed by using a role flip bit as in the example of FIG. 5, and the index information of the first part of the data unit and the index information of the second part of the data unit may be stored in a conventional manner. Relocation.

It should be noted that in the description of the embodiments of the present application, the first page stores two data units (the first data unit and the second data unit) as an example. In practical applications, the on-chip cache also It is possible to store more than two data units corresponding to the tag information, as long as the priority of the data unit of each part is reasonably limited. For example, the foregoing first page may further include a third part of the data unit, and the fourth tag information corresponding to the third part of the data unit has a third priority, and the third priority is lower than the second priority.

Using the on-chip cache shown in FIG. 6, the first part of the data unit corresponding to the first tag information and the second part of the data unit corresponding to the second tag information can be stored in the first page. Compared with the prior art solution of storing only the data unit corresponding to one tag information on one page, the on-chip cache 600 shown in FIG. 6 can be used to compress and store data originally stored in multiple pages to one page. So as to save the storage space of the on-chip cache.

Based on the same inventive concept, an embodiment of the present application further provides an integrated chip. Referring to FIG. 7, the integrated chip 700 includes an on-chip cache 701. The on-chip cache 701 is used to store a first page. The first page includes a first part of the data unit and a second part of the data unit. The first tag information corresponding to the first part of the data unit has a first priority, and the first part of the second part of the data unit corresponds to the first The two-label information has a second priority, and the second priority is lower than the first priority.

Specifically, the integrated chip 700 may be a system on chip (SoC).

In addition, the on-chip cache 701 is further configured to store index information of the first page, where the index information of the first page includes index information of the first part of the data unit and index information of the second part of the data unit. The index information of the first part of the data unit includes the first tag information and the first significant bit information, and the index information of the second part of the data unit includes the second label information and the second significant bit information.

Similarly, the first page further includes a third part of the data unit, and the fifth tag information corresponding to the third part of the data unit has a third priority, and the third priority is lower than the second priority.

For a related description of the above-mentioned on-chip cache 701, reference may be made to the related introduction in the on-chip cache 600 shown in FIG. 6, and details are not described herein again.

The integrated chip 700 may further include: a processor 702, configured to send a first access instruction, where the first access instruction is used to request access to the first data unit, and the first data unit corresponds to the first tag information; the on-chip cache 701 is further configured to: : Determine that the first data unit is stored in the on-chip buffer 701 according to the first valid bit information; and send the first data unit to the processor 702.

That is, when the processor 702 requests to access the data unit corresponding to the first tag information, if the data unit is stored in the on-chip cache 701, the data unit is directly returned to the processor 702 without having to be retrieved from off-chip memory. Access, thereby improving data access efficiency.

In a possible design, the processor 702 is further configured to send a second access instruction, the second access instruction is used to request access to the second data unit, and the second data unit corresponds to the second tag information; the on-chip cache 701 is also used Yu: Determine that the second data unit is stored in the on-chip buffer 701 according to the second significant bit information; and send the second data unit to the processor 702.

That is, when the processor 702 requests to access the data unit corresponding to the second tag information, if the data unit is stored in the on-chip cache 701, the data unit is directly returned to the processor 702 without having to be retrieved from off-chip memory. Access, thereby improving data access efficiency.

Further, if the processor 702 accesses a data unit corresponding to the second tag information, the on-chip cache 701 is further configured to determine whether to prioritize the second tag information according to the number of times the second data unit is accessed in a unit time. The priority is set to the first priority, and the priority of the first label information or the third label information having the first priority is set to the second priority.

As mentioned above, in some cases, the priorities of the second tag information and the first tag information may be replaced. Then, after determining that the processor 702 accesses the data unit corresponding to the second tag information, the on-chip cache 701 may determine whether to perform the foregoing priority replacement according to the number of times the data unit corresponding to the second tag information is accessed in a unit time.

Exemplarily, if the access count of the second tag information exceeds 20% of the minimum value of the access count of the first priority tag information in the same set (taking a 4-way group connection as an example, including four pages in the same set), then Perform the above priority replacement process. When replacing, the tag information with the first priority in the same Set is selected for replacement.

In another possible design, the processor 702 is further configured to send a third access instruction, the third access instruction is used to request access to a third data unit, and the third data unit corresponds to the second tag information; the on-chip cache 701 also Used for: determining that the third data unit is not stored in the on-chip cache 701 according to the second significant bit information; the processor 702 is further configured to read and store the third data unit from the off-chip memory; and the on-chip cache 701 is further used for A third data unit is stored.

That is, when the processor 702 requests to access the data unit corresponding to the second tag information, if the data unit is not stored in the on-chip cache 701, the corresponding data unit is read from the off-chip memory, and the on-chip cache 701 will The data unit is stored.

Specifically, when the on-chip cache 701 stores the third data unit, the third data unit may be stored in the first page or the second page. Among them, the second page and the first page are data units on the same Way in the Data Array.

Further, the on-chip cache 701 is further configured to: make the effective position corresponding to the third data unit in the second significant bit information valid; set the priority of the second label information to the first priority, and set the first label The priority of the information or the fourth tag information having the first priority is set as the second priority.

In the embodiment of the present application, the second tag information may be regarded as the victim role of the first tag information. When the data unit corresponding to the second tag information is not stored in the on-chip cache 701, and the corresponding storage location of the data unit Occupied by the first tag information, when the corresponding data is read into the on-chip cache 701, the second tag information having the second priority needs to be upgraded to the first priority. At this time, one of the tag information having the first priority in the Tag Array must be downgraded to the second priority. This downgraded tag information may be the first tag information or the fourth tag information.

It is not difficult to see that when managing the first page, the problem of priority replacement is often involved. Regardless of whether the priority replacement is determined based on the number of times accessed in a unit time, or the priority replacement is performed because the access to the second tag information is not hit in the on-chip cache 701, for the second tag information (SubTag) , Can be called SubTag Promote (upgrade) process.

When performing SubTag promotion, it can be the priority replacement in the same way (such as the priority replacement of the first tag information and the second label information), or the priority replacement in different Ways (such as the first tag information and the third Priority replacement of the label information, or priority replacement of the first label information and the fourth label information).

When performing SubTag promotion, try to select PrimeTags in the same set in the same set as much as possible. If you select other ways in the same set, additional memory access operations will be introduced, but the advantage is that the PrimeTags being replaced are the least recently used. Conversely, if only the PrimeTags in the same way in the same set are selected for replacement, there may be a case where the replaced PrimeTags are not the least recently used.

In addition, if the priority is replaced in different Ways, the replaced tag information with the first priority may be in the same Way as the tag information with the second priority cached in the movie, or it may be a different Way. inside.

When performing SubTag Promote, in addition to priority replacement, operations involving significant bit information are also involved. The following uses two specific examples as examples to introduce the SubTag Promote process.

1. The replaced tag information with the first priority is the same as the tag information with the second priority cached in the movie.

A specific example of the SubTag Promote in the same Way can be shown in Figure 8. The processor accesses the data unit corresponding to the fifth significant bit in the second significant bit information, and the data unit is not stored in the on-chip cache, then the data unit is read from the off-chip DRAM. It should be noted that When fetching data from the external DRAM, the data units corresponding to other valid bit information in the second tag information must be retrieved together (that is, data prefetching is performed based on historical access information). After the corresponding data is retrieved, the fifth valid position in the second significant bit information needs to be valid, and the priority of the second tag information is upgraded to the first priority. For this reason, the tag information with the first priority (hereinafter referred to as Tag1) is selected for replacement. The valid bit information corresponding to Tag1 is 10010010, and the valid bits corresponding to the second priority tag information (hereinafter referred to as Tag2) in the same Way. The information is 01100001. After the priority of the second tag information is upgraded to the first priority, the data corresponding to Tag2 is kicked out of the on-chip cache and stored in the off-chip DRAM; Tag1 is downgraded to the second priority because the first valid bit and the fourth The data corresponding to each valid bit conflicts with the upgraded second tag information. Because Tag1 has a lower priority, the corresponding data unit in Tag1 is stored in the off-chip DRAM. After the completion of the SubTag Promote, the valid bit information is shown in Figure 8.

It can be seen that in the example of FIG. 8, the replaced Tag 2 is in the same Way as the tag 1 that is kicked out of the on-chip cache.

2. The replaced tag information with the first priority is different from the tag information with the second priority cached in the movie.

A specific example of SubTag and Promote in different Ways can be shown in Figure 9. The processor accesses the data unit corresponding to the fifth significant bit in the second significant bit information, and the data unit is not stored in the on-chip cache, then the data unit is read from the off-chip DRAM. It should be noted that When fetching data from the external DRAM, the data units corresponding to other significant bit information in the second tag information must be retrieved together. After the corresponding data is retrieved, the fifth valid position in the second significant bit information needs to be valid, and the priority of the second tag information is upgraded to the first priority. To do this, select the tag information with the first priority Tag1 (assuming it is in Way1, that is, in column vector 1), then at this time, you also need to select a tag information with the second priority to kick out the on-chip cache Suppose it is in Way3, that is, in column vector 3.)

In the example in FIG. 9, after the fifth valid position in the second significant bit information is valid, Tag1 with the first priority in Way1 is selected for replacement, and then the corresponding information of Tag1 can be found in other locations in the on-chip cache for storage. The part of Tag2 corresponding information that does not conflict with the updated second significant bit information can be retained, and the conflicting part is stored in the off-chip DRAM.

In order to continue to store Tag1 in the on-chip cache, find Tag4 with the second priority in Way3 for replacement, so all the corresponding information of Tag4 can be stored in the off-chip DRAM. Then, compare Tag1 with Tag3 with the first priority in Way3. The non-conflicting part with Tag3 can be kept in the on-chip cache, and the conflicting part with Tag3 is stored in the off-chip DRAM.

In addition, in the embodiment of the present application, the integrated chip 700 may further include a tag buffer 703 for storing the first tag information and the second tag information. The tag buffer 703 may be located in the MC.

That is, the tag buffer 703 may further store index information of pages stored in the on-chip cache. In addition to storing tag information, information such as valid bit information and dirty bit information can also be stored. In this case, after the processor issues an access instruction, the corresponding information may be first searched in the tag buffer 703 to determine whether the data accessed by the processor is stored in the on-chip cache, thereby improving data access efficiency.

It should be noted that the data in the tag buffer 703 may be newer than the data in the on-chip cache 701. Therefore, when data update occurs in the tag buffer 703, the updated data needs to be written back to the on-chip cache 701. In addition, when the page index information in the on-chip cache 701 or the off-chip memory changes, the data in the tag buffer 703 must be updated synchronously.

It is not difficult to see that in the integrated chip 700, the judgment of the storage location of the accessed data may occur in the tag buffer (TB), the on-chip cache (DC, that is, DRAM cache), and the off-chip DRAM. In a specific example, the data access flow of the processor may be as shown in FIG. 10.

When a memory access request is missed in the Last Level Cache (LLC), the request is sent to the TB in the memory controller. If the tag information (Tag) is hit, then Continue to determine whether the corresponding valid bit in the corresponding valid bit information (valid bits) in the TB is 1 (valid = 1 is valid, valid = 0 is invalid), if it is 1, it means that the data is in DC , Continue to access the DC. It is determined in the DC whether the currently accessed address hits the first tag information (PrimeTag) or the second tag information (SubTag) (PrimeTag and SubTag are not distinguished in TB). If it is PrimeTag (TB hit, DC PrimeTag hit, valid) Then it is determined that the data is in the DC, and the DC is directly accessed to obtain the data, and the related count of the number of times that the PrimeTag is accessed is updated. If the access is a SubTag (TB, Hit, DC, SubTag, hit, valid), it is determined that the data is in the DC, and the DC is directly accessed to obtain the data, update the SubTag count, and determine whether a second tag information upgrade (SubTag promotion) is required.

If you visit the TB and find that the tag hits and the valid bit = 0, it means that the page corresponding to the tag is in the DC, but the corresponding footprint (that is, data) is not in the DC (DC hit, data miss, that is, false miss). After querying the DC, confirm whether the PrimeTag or SubTag is accessed. If it is PrimeTag (TB hit, PrimeTag, not valid), because PrimeTag has a high priority, you can directly return data from off-chip DRAM and update the corresponding Valid bits in DC and TB to 1. At this time, the valid bits corresponding to the same set of SubTags need to be cleared in both DC and TB. If it is a SubTag (TB, Hit, DC, SubTag, Hit, not valid), then access the off-chip DRAM to obtain data, and decide whether to perform SubTag promotion.

If a miss occurs when accessing the TB, it is impossible to determine whether the data is in the DC at this time, because the TB miss only represents that the Tag Array in the DC is not in the TB at this time. Continue to visit the DC to confirm whether a real hit (that is, a tag hit and valid bit = 1 can also be referred to as a full hit in this application). If a real hit occurs, continue to distinguish whether it is a PrimeTag or a SubTag. If the PrimeTag hits (TB miss, DC (Prime Tag hit, valid), return the data from the DC, and write the corresponding Tag and valid bits back to the TB. If the SubTag hits (TB miss, DC subTag Hit, valid), return the data from the DC, and at the same time Tag and valid bits are written back to TB.

If a miss occurs when accessing the TB, and a tag hit of the DC occurs but valid = 0 (that is, invalid), a DC only tag hit occurs. At this time, it is necessary to continue to determine whether the PrimeTag hits. If a DC primaryTag hit (TB miss, DC PrimeTag hit, not valid), access the off-chip DRAM to obtain data, and update the valid bits in the DC and the Tag and valid bits in the TB. If a DC SubTag hit (TB miss, DC SubTag hit, not valid) accesses the off-chip DRAM to obtain data, and decides whether to perform a SubTag promotion.

If a miss occurs when accessing the TB, and the DC is also a real miss (TB miss, DC real miss), which meets the original PrimeTag replacement standard, a PrimeTag replacement occurs.

Specifically, the process of upgrading the second tag information (SubTag) can be as shown in FIG. 11. First determine whether to perform SubTag promotion according to the access process and set conditions. If not, it is only necessary to update the access count of the second tag information (SubTag). If SubTag promotion is performed, it is judged whether the replaced first tag information (PrimeTag) and SubTag are in the same column vector (way). If they are in the same way, the roles of PrimeTag and SubTag can be directly reversed. At the same time, The data corresponding to the valid bit that conflicts with the SubTag in the PrimeTag is written back to the chip. If the PrimeTag and the SubTag that are replaced are not in the same way, you can perform the SubTag promotion according to the method shown in Figure 8 and Figure 9 above. More details.

Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. In this way, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these changes and variations.

Claims

An on-chip cache is characterized in that it includes:

A storage unit, configured to store a first page, where the first page includes a first partial data unit and a second partial data unit, and the first tag information corresponding to the first partial data unit has a first priority and the second partial The second tag information corresponding to the data unit has a second priority, and the second priority is lower than the first priority.
The on-chip cache according to claim 1, wherein the storage unit is further configured to:

The index information of the first page is stored, and the index information of the first page includes the index information of the first partial data unit and the index information of the second partial data unit.
The on-chip cache according to claim 2, wherein the index information of the first partial data unit includes the first tag information and the first significant bit information, and the index information of the second partial data unit includes the The second tag information and the second significant bit information are described.
The on-chip cache according to any one of claims 1 to 3, further comprising:

A storage controller, configured to set the priority of the second label information to the first priority, and set the priority of the first label information or the third label information having the first priority to all The second priority is described.
The on-chip cache according to any one of claims 1 to 4, wherein the first page further includes a third partial data unit, and fourth tag information corresponding to the third partial data unit has a third priority. Level, the third priority is lower than the second priority.
An integrated chip, comprising:

An on-chip cache is used to store a first page, where the first page includes a first partial data unit and a second partial data unit, and the first tag information corresponding to the first partial data unit has a first priority, and the second The second tag information corresponding to some data units has a second priority, and the second priority is lower than the first priority.
The integrated chip according to claim 6, wherein the on-chip cache is further configured to:

The index information of the first page is stored, and the index information of the first page includes the index information of the first partial data unit and the index information of the second partial data unit.
The integrated chip according to claim 7, wherein the index information of the first partial data unit includes the first tag information and the first significant bit information, and the index information of the second partial data unit includes the Second tag information and second significant bit information.
The integrated chip according to claim 8, further comprising:

A processor, configured to send a first access instruction, where the first access instruction is used to request access to a first data unit, and the first data unit corresponds to the first tag information;

The on-chip cache is also used for:

Determining that the first data unit is stored in the on-chip cache according to the first significant bit information;

Sending the first data unit to the processor.
The integrated chip according to claim 9, wherein the processor is further configured to:

Sending a second access instruction, where the second access instruction is used to request access to a second data unit, and the second data unit corresponds to the second tag information;

The on-chip cache is also used for:

Determining that the second data unit is stored in the on-chip cache according to the second significant bit information;

Sending the second data unit to the processor.
The integrated chip according to claim 10, wherein the on-chip cache is further configured to:

Determining whether to set the priority of the second tag information to the first priority according to the number of times the second data unit is accessed in a unit time, and the first tag information may have the first priority The priority of the third tag information is set to the second priority.
The integrated chip according to any one of claims 9 to 11, wherein the processor is further configured to:

Sending a third access instruction, where the third access instruction is used to request access to a third data unit, and the third data unit corresponds to the second tag information;

The on-chip cache is also used for:

Determine that the third data unit is not stored in the on-chip cache according to the second significant bit information;

The processor is further configured to:

Reading the third data unit from an off-chip memory;

The on-chip cache is also used for:

The third data unit is stored.
The integrated chip according to claim 12, wherein the on-chip cache is specifically configured to: when storing the third data unit:

The third data unit is stored in the first page or the second page.
The integrated chip according to claim 12 or 13, wherein the on-chip cache is further configured to:

Valid a valid position corresponding to the third data unit in the second valid bit information;

Setting the priority of the second label information as the first priority, and setting the priority of the first label information or the fourth label information having the first priority as the second priority.
The integrated chip according to any one of claims 6 to 14, further comprising:

A tag buffer is configured to store the first tag information and the second tag information.
The integrated chip according to any one of claims 6 to 15, wherein the first page further includes a third partial data unit, and the fifth tag information corresponding to the third partial data unit has a third priority , The third priority is lower than the second priority.