CN116578409A - Method, system and medium for identifying and migrating memory hot page - Google Patents

Abstract

The application provides a method, a system and a medium for identifying and migrating a memory hot page, which relate to the technical field of computer system structures and comprise the following steps: step S1: initializing a memory node; step S2: recording the access frequency of the memory page; step S3: counting the heat distribution of each memory page to obtain a counting result; step S4: analyzing the statistical result; step S5: dynamically adjusting system parameters according to analysis; step S6: and screening the memory hot page, and migrating the memory hot page to a memory node. The application can well reduce the average delay of memory access.

Description

Method, system and medium for identifying and migrating memory hot page

Technical Field

The application relates to the technical field of computer system structures, in particular to a method, a system and a medium for identifying and migrating a memory hot page.

Background

Non-coherent memory access (Non Uniform Memory Access, NUMA) is a design in which computer memory is divided into memory nodes in a multi-process environment, where the access time of a memory node depends on the relative position between the node and the processor, which accesses a local memory node faster than a remote memory node.

The current mainstream operating systems all support NUMA architecture to varying degrees. In recent years, emerging memory devices, such as a pride persistent memory, have been developed, and the unit density of these physical memory devices is higher than that of a conventional DRAM memory, and the memory capacity is also larger at the same cost, but the access speed is slower than that of the conventional DRAM.

In the current technology implementation, usually, under the NUMA framework, the DRAM memory device is set as a local memory node, and the high-capacity slow memory device is added into the system memory in a hot plug manner and is set as a remote memory node under the NUMA framework. The existing mechanism of the operating system can identify remote memory node access, and migrate a memory page positioned at the remote memory node to a local node, so that the memory access is performed at the local node as much as possible, and the memory access is accelerated.

In order to improve the overall access performance of the memory, the memory pages need to be placed in layers according to the access heat. The existing mechanism of NUMA can not perform good balance optimization on memory distribution, and a new mechanism is needed to well screen out pages with higher access heat in a slow memory node, migrate hot pages to a fast node and keep cold pages in the slow node. Meanwhile, a mechanism is needed to control the page migration rate, so that space is reserved for page migration and application of new memory pages, and memory jolt is reduced. None of the other existing solutions meets these requirements well.

NUMA: non Uniform Memory Access non-coherent memory access is a design of dividing a computer memory into several memory nodes in a multi-process environment, wherein the access time of a memory node depends on the relative position between the node and a processor.

DAX: direct Access, a Direct Access mechanism, allows users to bypass the system cache Direct Access block device and NVDIMM device.

HOTPLUG: the hot plug, an operating system function, can access newly added equipment into the system bus, and under the support of the system driver, the equipment supporting memory access can be added into the system memory in the form of new memory nodes.

NVDIMM: non Volatile Dual In-line Memory Module, a nonvolatile dual inline memory module, a physically persistent storage device, can be used as a cost-effective memory device replacement.

Disclosure of Invention

Aiming at the defects in the prior art, the application provides a method, a system and a medium for identifying and migrating a memory hot page.

According to the method, the system and the medium for identifying and migrating the memory hot page provided by the application, the scheme is as follows:

in a first aspect, a method for identifying and migrating a hot page of a memory is provided, where the method includes:

step S1: initializing a memory node;

step S2: recording the access frequency of the memory page;

step S3: counting the heat distribution of each memory page to obtain a counting result;

step S4: analyzing the statistical result;

step S5: dynamically adjusting system parameters according to analysis;

step S6: and screening the memory hot page, and migrating the memory hot page to a memory node.

Preferably, the step S1 includes:

step S1.1: dividing a memory area from a memory device;

step S1.2: formatting the memory area by using a DAX file system;

step S1.3: creating corresponding memory nodes for the formatted memory areas by using a hot plug technology;

step S1.4: adding the newly created memory node into a memory node list, establishing a mapping relation between the newly created memory node and a target node of hot page migration, and initializing a heat statistics module and a page table scanning task of the newly created memory node.

Preferably, the step S2 includes:

step S2.1: after initializing the memory node information, scanning page tables of all memory nodes, setting protection marks for scanned virtual pages, and recording scanning time in a description structure of the mapped memory pages;

step S2.2: triggering a page protection mechanism when a user process accesses a virtual page with a set protection mark, and enabling the system to fall into a kernel mode;

step S2.3: and (3) estimating the access frequency of the memory page corresponding to the virtual page in the last period according to the time difference between the current time and the scanning time recorded in the step S2.1, and eliminating the protection mark.

Preferably, the step S3 includes: and (3) sampling and analyzing the heat of the used memory page every other statistical period by each process according to the page access frequency information recorded in the step S2, wherein the sampling and analyzing comprises the following steps:

step S3.1: uniformly sampling the virtual address space of the process, and recording the starting time of the sampling period;

step S3.2: setting a sampling mark and a protection mark for the sampled memory page, and establishing a periodic record table according to the virtual address sequence;

step S3.3: resetting a sampling mark and a protection mark for a memory page sampled before a statistical period;

step S3.4: for the page sampled before two statistical periods, the data in the periodic record table are summarized to a collection table;

step S3.5: and deleting the summarized periodic record table, and releasing the space occupied by the periodic record table.

Preferably, the step S3.1 includes:

acquiring system time and recording the starting time of the scanning period;

determining a random range S= {0,1,2, …, y-1}, and extracting a random integer x in S by using a random function; if the random integer x extracted at this time is the same as the random integer x' obtained last time, re-extracting a different integer in the range;

the method comprises the steps of scanning a memory region VMA linked list of a process, and dividing a memory page contained in a memory region into a plurality of continuous page groups according to a certain size for the memory region;

for each page group, randomly mapping the initial memory address of the page group into S to obtain a mapping value y;

if the mapping value y is the same as the random integer x, adding the pages in the memory group into the memory page set of the current sampling;

skipping if the mapping value y is not the same as the random integer x;

the step S3.2 includes:

creating and initializing a new red-black tree as a periodic record table, wherein a Key Key stores a virtual address of a memory page, a Value stores an access interval of the memory page corresponding to the virtual page, and the Key comprises additional information of a memory node number to which the memory page belongs;

setting sampling marks and protection marks for the virtual pages extracted at this time in batches according to the memory area;

establishing a corresponding record item for a memory page corresponding to the virtual page extracted at the time, wherein the default value of the access interval is set as the maximum statistical period interval;

inserting the newly established record item into a record table;

when step S2.3 is carried out, a periodic record table of a corresponding process is inquired, if the memory page accessed at this time is found to have a corresponding entry in the record table of the current statistics period, the value of the entry is set as the access frequency measured in step S2.3, and the statistics mark is eliminated;

the step S3.3 includes:

taking out a record table of the last statistics period, and searching a corresponding virtual page and a memory area for each table item according to the virtual address stored in the key;

resetting the access mark and the protection mark of the searched virtual page;

when step S2.3 is carried out, a periodic record table of a corresponding process is inquired, if the memory page accessed at this time is found to have a corresponding entry in the record table of the previous statistical period, the value of the entry is set to be the maximum value of the access frequency and the current value of the entry measured at the step S2.3 at this time, and the statistical mark is eliminated;

the step S3.4 includes:

the access frequency is uniformly divided into a plurality of continuous intervals, and the record table is divided into a statistical unit according to the memory nodes and the frequency intervals;

traversing records in the record table, and finding out a corresponding statistical unit in the collection table;

the number of pages of each statistical unit, the mean and variance of the access frequency are counted.

Preferably, the step S4 includes:

step S4.1: for each memory node to be migrated, inquiring the corresponding target memory node;

step S4.2: establishing two scanning pointers of a hot page pointer and a cold page pointer in a collection table, wherein the hot page pointer scans a statistic unit list corresponding to a memory node to be migrated, and the scanning direction is from hot to cold; the cold page pointer scans a statistic unit list corresponding to the target memory node, and the scanning direction is from cold to hot;

step S4.3: scanning the collection table until the access frequency pointed by the cold page pointer is higher than the access frequency pointed by the hot page pointer;

step S4.4: recording the number of pages swept by the cold and hot pointers and the average value of the access frequency pointed by the cold and hot pointers at the end;

the step S4.2 includes:

the hot page pointer points to the statistics unit corresponding to the maximum value of the access frequency, and the cold page pointer points to the statistics unit corresponding to the minimum value of the access frequency;

if the number of the residual pages of the statistical unit pointed by the hot page pointer is larger than the number of the residual pages corresponding to the cold page pointer, the cold page pointer moves, the number of the residual pages corresponding to the hot page pointer is subtracted from the number of the residual pages corresponding to the cold page pointer, and the scanned number of the pages is updated;

if the number of the residual pages of the statistical unit pointed by the hot page pointer is smaller than the number of the residual pages corresponding to the cold page pointer, the hot page pointer moves, the number of the residual pages corresponding to the cold page pointer is reduced by the number of the residual pages corresponding to the hot page pointer, and the scanned number of the pages is updated;

and when the process is finished, calculating the specific access frequency according to the residual page quantity proportion of the statistical unit pointed by the cold and hot pointer, the average value and the variance of the access frequency.

Preferably, the step S5 includes:

step S5.1: according to the result obtained in the step S4, calculating a heat threshold and a migration rate by combining the sampling rate;

step S5.2: updating the calculated result into a collection table of each memory node;

step S5.3: collecting the statistics results of the tables in the time sequence updated by each process and eliminating the outdated statistics results according to the LRU strategy;

step S5.4: and (3) carrying out weighted average on the statistical results in the collection table according to the number of pages, and updating two system parameters, namely the heat threshold value and the migration rate of the memory node.

Preferably, the step S6 filters the hot pages of the memory based on the hot threshold and the migration rate generated in the step S5, including:

step S6.1: comparing the recent page access frequency with the magnitude of the heat threshold, and judging that the page is cold if the frequency is lower than the heat threshold; if the frequency is not lower than the heat threshold, judging that the page is hot;

step S6.2: if the page is judged to be a hot page, judging to be a transferable page if the number of page migration in the statistical period does not reach the maximum number specified by the migration rate; otherwise, judging whether the page is a transferable page according to the strategy;

if a conservative policy is selected, the page is determined to be a non-migratable page; if the aggressive strategy is selected, the heat threshold is improved, if the page heat is higher than the improved threshold, the page is judged to be a transferable page, otherwise, the page is judged to be an non-transferable page;

step S6.3: and migrating the memory data of the transferable pages, updating the position mapping relation between the memory data and the virtual pages, and completing the page migration process.

In a second aspect, a system for identifying and migrating a hot page of memory is provided, the system comprising:

and the heat calculating module is used for: calculating access frequency by using page table scanning and page access time, so as to estimate the recent access heat of the memory page;

heat screening system: the method is used for judging whether the memory page accessed at the time should be migrated;

and a heat statistics module: the method is used for counting the heat distribution condition of the memory pages of each memory node;

and a heat analysis module: sorting and analyzing the heat statistics results of the memory nodes;

and a summarizing module: summarizing the heat analysis results of each process on the memory nodes, and calculating the reference value of the heat screening system parameters;

and a migration module: and transferring the screened memory pages to the target memory node by calling other system interfaces.

In a third aspect, a computer readable storage medium storing a computer program is provided, which when executed by a processor implements steps in the method of memory hot page identification and migration.

Compared with the prior art, the application has the following beneficial effects:

1. aiming at the problem of memory data cold and hot layering under a NUMA architecture, the method and the device provided by the application can well reduce the average delay of memory access;

2. the application can be adapted to the memory management sub-module in the kernel of the operating system, optimizes the support of the module to the novel memory equipment based on the NVDIMM technology, and has good market prospect and application value.

Other advantages of the present application will be set forth in the description of specific technical features and solutions, by which those skilled in the art should understand the advantages that the technical features and solutions bring.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, given with reference to the accompanying drawings in which:

FIG. 1 is a schematic diagram of the overall module of the device in an embodiment of the application;

FIG. 2 is a schematic diagram of a hot page screening process in an embodiment of the present application;

FIG. 3 is a schematic diagram of a heat statistics module according to an embodiment of the present application;

FIG. 4 is a schematic illustration of a thermal analysis flow in an embodiment of the application;

fig. 5 is a schematic diagram of a collection module in an embodiment of the application.

Detailed Description

The present application will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the present application, but are not intended to limit the application in any way. It should be noted that variations and modifications could be made by those skilled in the art without departing from the inventive concept. These are all within the scope of the present application.

The embodiment of the application provides a method for identifying and migrating a hot page of a memory, which is shown by referring to fig. 1, and specifically comprises the following steps:

step S1: initializing the memory node.

The step S1 specifically includes:

step S1.1: dividing a memory area from a memory device;

step S1.2: formatting the memory area by using a DAX file system;

step S1.3: creating corresponding memory nodes for the formatted memory areas by using a memory hot plug technology; the memory hot plug technology can access newly added equipment into a system bus, and the equipment supporting memory access can be added into a system memory in the form of a new memory node under the support of a system driver.

Step S1.4: adding the newly created memory node into a memory node list, establishing a mapping relation between the newly created memory node and a target node of hot page migration, and initializing a heat statistics module and a page table scanning task of the newly created memory node.

Step S2: the access frequency of the memory pages is recorded.

The step S2 specifically includes:

step S2.1: after initializing the memory node information, scanning page tables of all memory nodes, setting protection marks for scanned virtual pages, and recording scanning time in a description structure of the mapped memory pages;

step S2.2: triggering a page protection mechanism when a user process accesses a virtual page with a set protection mark, and enabling the system to fall into a kernel mode;

step S2.3: and (3) estimating the access frequency of the memory page corresponding to the virtual page in the last period according to the time difference between the current time and the scanning time recorded in the step S2.1, and eliminating the protection mark.

Step S3: and counting the heat distribution of each memory page to obtain a counting result.

Sampling and analyzing the heat of the used memory page every other statistical period by each process according to the page access frequency information recorded in the step S2, wherein the step S3 specifically comprises the following steps:

step S3.1: uniformly sampling the virtual address space of the process, and recording the starting time of the sampling period;

acquiring system time and recording the starting time of the scanning period;

determining a random range S= {0,1,2, …, y-1}, and extracting a random integer x in S by using a random function; if the random integer x extracted at this time is the same as the random integer x' obtained last time, re-extracting a different integer in the range;

the method comprises the steps of scanning a memory region VMA linked list of a process, and dividing a memory page contained in a memory region into a plurality of continuous page groups according to a certain size for the memory region;

for each page group, randomly mapping the initial memory address of the page group into S to obtain a mapping value y;

if the mapping value y is the same as the random integer x, adding the pages in the memory group into the memory page set of the current sampling;

if the mapping value y is not the same as the random integer x, skipping.

Step S3.2: setting a sampling mark and a protection mark for the sampled memory page, and establishing a periodic record table according to the virtual address sequence;

creating and initializing a new red-black tree as a periodic record table, wherein a Key Key stores a virtual address of a memory page, a Value stores an access interval of the memory page corresponding to the virtual page, and the Key comprises additional information of a memory node number to which the memory page belongs;

setting sampling marks and protection marks for the virtual pages extracted at this time in batches according to the memory area;

establishing a corresponding record item for a memory page corresponding to the virtual page extracted at the time, wherein the default value of the access interval is set as the maximum statistical period interval;

inserting the newly established record item into a record table;

and when step S2.3 is carried out, the periodic record table of the corresponding process is inquired, if the memory page accessed at the present time has corresponding items in the record table of the current statistical period, the value of the items is set as the access frequency measured in step S2.3, and the statistical mark is eliminated.

Step S3.3: and resetting the sampling mark and the protection mark for the memory page sampled before one statistical period.

Taking out a record table of the last statistics period, and searching a corresponding virtual page and a memory area for each table item according to the virtual address stored in the key;

resetting the access mark and the protection mark of the searched virtual page;

when step S2.3 is carried out, a periodic record table of a corresponding process is inquired, if the memory page accessed at this time is found to have a corresponding entry in the record table of the previous statistical period, the value of the entry is set to be the maximum value of the access frequency and the current value of the entry measured at the step S2.3 at this time, and the statistical mark is eliminated;

step S3.4: for the page sampled before two statistical periods, the data in the periodic record table are summarized to a collection table;

the access frequency is uniformly divided into a plurality of continuous intervals, and the record table is divided into a statistical unit according to the memory nodes and the frequency intervals;

traversing records in the record table, and finding out a corresponding statistical unit in the collection table;

the number of pages of each statistical unit, the mean and variance of the access frequency are counted.

Step S3.5: and deleting the summarized periodic record table, and releasing the space occupied by the periodic record table.

Step S4: and analyzing the statistical result.

The step S4 specifically includes: step S4.1: for each memory node to be migrated, inquiring the corresponding target memory node;

step S4.2: establishing two scanning pointers of a hot page pointer and a cold page pointer in a collection table, wherein the hot page pointer scans a statistic unit list corresponding to a memory node to be migrated, and the scanning direction is from hot to cold; the cold page pointer scans the statistic unit list corresponding to the target memory node, and the scanning direction is from cold to hot.

Step S4.2 comprises:

the hot page pointer points to the statistics unit corresponding to the maximum value of the access frequency, and the cold page pointer points to the statistics unit corresponding to the minimum value of the access frequency;

if the number of the residual pages of the statistical unit pointed by the hot page pointer is larger than the number of the residual pages corresponding to the cold page pointer, the cold page pointer moves, the number of the residual pages corresponding to the hot page pointer is subtracted from the number of the residual pages corresponding to the cold page pointer, and the scanned number of the pages is updated;

if the number of the residual pages of the statistical unit pointed by the hot page pointer is smaller than the number of the residual pages corresponding to the cold page pointer, the hot page pointer moves, the number of the residual pages corresponding to the cold page pointer is reduced by the number of the residual pages corresponding to the hot page pointer, and the scanned number of the pages is updated;

and when the process is finished, calculating the specific access frequency according to the residual page quantity proportion of the statistical unit pointed by the cold and hot pointer, the average value and the variance of the access frequency.

Step S4.3: scanning the collection table until the access frequency pointed by the cold page pointer is higher than the access frequency pointed by the hot page pointer;

step S4.4: recording the number of pages swept by the cold and hot pointers and the average value of the access frequency pointed by the cold and hot pointers at the end;

step S5: and dynamically adjusting system parameters according to the analysis.

The step S5 includes:

step S5.1: according to the result obtained in the step S4, calculating a heat threshold and a migration rate by combining the sampling rate;

step S5.2: updating the calculated result into a collection table of each memory node;

step S5.3: collecting the statistics results of the tables in the time sequence updated by each process and eliminating the outdated statistics results according to the LRU strategy;

step S5.4: and (3) carrying out weighted average on the statistical results in the collection table according to the number of pages, and updating two system parameters, namely the heat threshold value and the migration rate of the memory node.

Step S6: and screening the memory hot page, and migrating the memory hot page to a memory node.

The step S6 of screening the hot pages of the memory based on the hot threshold and the migration rate generated in the step S5 includes:

step S6.1: comparing the recent page access frequency with the magnitude of the heat threshold, and judging that the page is cold if the frequency is lower than the heat threshold; if the frequency is not lower than the heat threshold, judging that the page is hot;

step S6.2: if the page is judged to be a hot page, judging to be a transferable page if the number of page migration in the statistical period does not reach the maximum number specified by the migration rate; otherwise, judging whether the page is a transferable page according to the strategy;

if a conservative policy is selected, the page is determined to be a non-migratable page; if the aggressive strategy is selected, the heat threshold is improved, if the page heat is higher than the improved threshold, the page is judged to be a transferable page, otherwise, the page is judged to be an non-transferable page;

step S6.3: and migrating the memory data of the transferable pages, updating the position mapping relation between the memory data and the virtual pages, and completing the page migration process.

The application also provides a memory hot page identification and migration system, which can be realized by executing the flow steps of the memory hot page identification and migration method, namely, a person skilled in the art can understand the memory hot page identification and migration method as a preferred implementation mode of the memory hot page identification and migration system.

And the heat calculating module is used for: calculating access frequency by using page table scanning and page access time, so as to estimate the recent access heat of the memory page;

heat screening system: the method is used for judging whether the memory page accessed at the time should be migrated;

and a heat statistics module: the method is used for counting the heat distribution condition of the memory pages of each memory node;

and a heat analysis module: sorting and analyzing the heat statistics results of the memory nodes;

and a summarizing module: summarizing the heat analysis results of each process on the memory nodes, and calculating the reference value of the heat screening system parameters;

and a migration module: and transferring the screened memory pages to the target memory node by calling other system interfaces.

Next, the present application will be described in more detail.

Referring to fig. 1, a system for identifying and migrating a hot page of a memory specifically includes the following:

and the heat calculation module calculates the access frequency by using the page table scanning and the time of page access so as to estimate the recent access heat of the memory page.

Heat screening system: the module is used for judging whether the memory page accessed at the time should be migrated.

And the heat statistics module is used for counting the heat distribution condition of the memory pages of each memory node.

And the heat analysis module is used for sorting and analyzing the heat statistical results of the memory nodes.

And the summarizing module is used for summarizing the heat analysis results of each process on the memory nodes and calculating the reference value of the heat screening system parameters.

And the migration module migrates the screened memory pages to the target memory node by calling other system interfaces.

As shown in fig. 2, when a user process accesses a memory page, if a corresponding virtual page has a protection flag, the user process falls into a kernel mode, and the device records the system time at that time. By calculating the time difference between the time when the page is provided with the protection mark and the time, the frequency of the user accessing the memory page can be estimated, and the access heat of the page can be measured in units of the access time frequency.

Ideally, for a slow memory node to be migrated and its target fast memory node, there is a boundary of page warmth, pages with warmth higher than the boundary should be placed at the fast memory node, and pages with warmth lower than the boundary should be placed at the slow memory node. When the memory pages between two nodes have uneven heat and cold, hotter pages in the slow memory nodes should be screened out and migrated to the fast memory nodes, and meanwhile, migration rate is limited to prevent memory jolt. After the system is initialized, the heat boundary and the migration rate are set as default values, and are continuously adjusted in the running process.

When hot page screening is performed, the system compares the heat of the memory page with a heat threshold value: if the threshold value is lower, the page is not screened as a hot page; if the threshold is higher, the system checks the number of the pages which have been migrated in the last period of time and calculates the migration rate. If the limiting rate is not reached, the hot page is screened, and if the limiting rate is reached, the system further judges according to a screening strategy: if a conservation strategy is adopted, the page is not screened as a hot page; if the aggressive strategy is adopted, the heat threshold value is multiplied by a coefficient and then is compared, if the heat threshold value is still higher than the threshold value, the heat page is screened, and otherwise, the heat page is not screened.

In order to observe the heat distribution condition of the memory pages of each memory node and adjust the limitation of the heat threshold and the migration rate in real time according to the condition, the self-adaptive adjustment method based on heat statistics in the system can be used.

As shown in fig. 3, the system time is divided into a plurality of statistical periods, each period is divided into two stages of scanning and collecting, each process maintains a period collection table created by the last plurality of periods, and the period collection tables perform steps of creating, collecting sampling information of the current stage, sorting and recycling and the like in a pipeline form.

When each period starts, a plurality of pages are uniformly selected as sampling pages through a random algorithm, the sampling pages are simultaneously provided with a protection mark and a sampling mark, corresponding entries are created in a collection table, information such as access intervals, numbers of memory nodes and the like is recorded, and default values of the access intervals are set as the maximum value of the statistical period. When the page is accessed, the calculated interval is updated into the corresponding entry of the record table. If the statistical table is not created in the scanning stage of the statistical period, the maximum value of the current result and the historical result is removed.

And after a plurality of periods of the periodic collection table, finishing the collection task, and finishing the collection result into the analysis table. And converting the access interval in each entry of the collection table into access frequency, finding out the corresponding analysis unit in the analysis table according to the access frequency and the memory node to which the access interval belongs, and updating the information such as the page number of the unit, the average value and the variance of the access frequency.

As shown in fig. 4, at the beginning of each statistical period, extraction calculation is performed on the analysis table of the last statistical period. The method comprises the steps of setting a hot page pointer to point to a memory unit with highest representing frequency on a memory node to be migrated and setting a cold page pointer to point to a memory unit with lowest representing frequency on a target memory node one at a time. Each cycle compares the number of remaining pages of two pointed analysis units, moves the pointer representing the small number to the next memory unit, and subtracts the difference between the number of pages of the analysis unit with the large number of remaining memory pages. When the access frequency pointed by the hot page pointer is not more than that pointed by the cold page pointer, the cycle is terminated, the number of pages scanned by the two pointers at the moment and the access frequency pointed by the cycle at the time of termination are recorded, and the reference values of the heat threshold and the migration rate are calculated according to the number of pages scanned by the two pointers at the moment and the access frequency pointed by the cycle at the time of termination.

The analysis results of each process are summarized through a collection module, as shown in fig. 5, after a process obtains the reference values of the heat threshold and the migration rate of a certain memory node and a target memory node thereof, the process submits the results to the collection module of the memory node, and the collection module firstly compares the usage ratio of the process to the memory node, and skips if the usage ratio is lower. The collecting module submits the results meeting the requirements to the result buffer, and if the result buffer reaches the preset capacity, the outdated result furthest updated from the last time is eliminated, so that the instantaneity and the accuracy of the final result are ensured. The result cache can carry out comprehensive calculation according to the submitted results of all the processes, and the calculation results are updated in real time when the cache content changes, so that the parameter values such as the heat threshold value, the migration rate limit and the like of the hot page screening module are updated. The behavior of the screening module can be controlled more accurately by dynamically adjusting the system parameters, so that the memory page distribution is obviously improved at a higher speed through memory migration, the average memory access delay is reduced, and the overall performance is improved.

The embodiment of the application provides a method, a system and a medium for identifying and migrating memory hot pages, which aim at the problem of cold and hot layering of memory pages in hybrid memory equipment, can well improve the physical distribution of the memory pages among memory nodes, enable the pages with higher access heat to be mapped to the memory nodes with higher access speed as much as possible, reduce the average delay of memory access, optimize the overall performance of a memory system, and have good market prospect and application value.

Those skilled in the art will appreciate that the application provides a system and its individual devices, modules, units, etc. that can be implemented entirely by logic programming of method steps, in addition to being implemented as pure computer readable program code, in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Therefore, the system and various devices, modules and units thereof provided by the application can be regarded as a hardware component, and the devices, modules and units for realizing various functions included in the system can also be regarded as structures in the hardware component; means, modules, and units for implementing the various functions may also be considered as either software modules for implementing the methods or structures within hardware components.

The foregoing describes specific embodiments of the present application. It is to be understood that the application is not limited to the particular embodiments described above, and that various changes or modifications may be made by those skilled in the art within the scope of the appended claims without affecting the spirit of the application. The embodiments of the application and the features of the embodiments may be combined with each other arbitrarily without conflict.

Claims (10)

1. A method for identifying and migrating a hot page of memory, comprising:

step S1: initializing a memory node;

step S2: recording the access frequency of the memory page;

step S3: counting the heat distribution of each memory page to obtain a counting result;

step S4: analyzing the statistical result;

step S5: dynamically adjusting system parameters according to analysis;

step S6: and screening the memory hot page, and migrating the memory hot page to a memory node.

2. The method for identifying and migrating a hot page of a memory according to claim 1, wherein the step S1 comprises:

step S1.1: dividing a memory area from a memory device;

step S1.2: formatting the memory area by using a DAX file system;

step S1.3: creating corresponding memory nodes for the formatted memory areas by using a hot plug technology;

step S1.4: adding the newly created memory node into a memory node list, establishing a mapping relation between the newly created memory node and a target node of hot page migration, and initializing a heat statistics module and a page table scanning task of the newly created memory node.

3. The method for identifying and migrating a hot page of a memory according to claim 1, wherein the step S2 comprises:

step S2.1: after initializing the memory node information, scanning page tables of all memory nodes, setting protection marks for scanned virtual pages, and recording scanning time in a description structure of the mapped memory pages;

step S2.2: triggering a page protection mechanism when a user process accesses a virtual page with a set protection mark, and enabling the system to fall into a kernel mode;

step S2.3: and (3) estimating the access frequency of the memory page corresponding to the virtual page in the last period according to the time difference between the current time and the scanning time recorded in the step S2.1, and eliminating the protection mark.

4. The method for identifying and migrating a hot page of a memory according to claim 3, wherein the step S3 comprises: and (3) sampling and analyzing the heat of the used memory page every other statistical period by each process according to the page access frequency information recorded in the step S2, wherein the sampling and analyzing comprises the following steps:

step S3.1: uniformly sampling the virtual address space of the process, and recording the starting time of the sampling period;

step S3.2: setting a sampling mark and a protection mark for the sampled memory page, and establishing a periodic record table according to the virtual address sequence;

step S3.3: resetting a sampling mark and a protection mark for a memory page sampled before a statistical period;

step S3.4: for the page sampled before two statistical periods, the data in the periodic record table are summarized to a collection table;

step S3.5: and deleting the summarized periodic record table, and releasing the space occupied by the periodic record table.

5. The method for identifying and migrating a hot page of a memory according to claim 4, wherein the step S3.1 comprises:

acquiring system time and recording the starting time of the scanning period;

determining a random range S= {0,1,2, …, y-1}, and extracting a random integer x in S by using a random function; if the random integer x extracted at this time is the same as the random integer x' obtained last time, re-extracting a different integer in the range;

the method comprises the steps of scanning a memory region VMA linked list of a process, and dividing a memory page contained in a memory region into a plurality of continuous page groups according to a certain size for the memory region;

for each page group, randomly mapping the initial memory address of the page group into S to obtain a mapping value y;

if the mapping value y is the same as the random integer x, adding the pages in the memory group into the memory page set of the current sampling;

skipping if the mapping value y is not the same as the random integer x;

the step S3.2 includes:

creating and initializing a new red-black tree as a periodic record table, wherein a Key Key stores a virtual address of a memory page, a Value stores an access interval of the memory page corresponding to the virtual page, and the Key comprises additional information of a memory node number to which the memory page belongs;

setting sampling marks and protection marks for the virtual pages extracted at this time in batches according to the memory area;

establishing a corresponding record item for a memory page corresponding to the virtual page extracted at the time, wherein the default value of the access interval is set as the maximum statistical period interval;

inserting the newly established record item into a record table;

when step S2.3 is carried out, a periodic record table of a corresponding process is inquired, if the memory page accessed at this time is found to have a corresponding entry in the record table of the current statistics period, the value of the entry is set as the access frequency measured in step S2.3, and the statistics mark is eliminated;

the step S3.3 includes:

taking out a record table of the last statistics period, and searching a corresponding virtual page and a memory area for each table item according to the virtual address stored in the key;

resetting the access mark and the protection mark of the searched virtual page;

when step S2.3 is carried out, a periodic record table of a corresponding process is inquired, if the memory page accessed at this time is found to have a corresponding entry in the record table of the previous statistical period, the value of the entry is set to be the maximum value of the access frequency and the current value of the entry measured at the step S2.3 at this time, and the statistical mark is eliminated;

the step S3.4 includes:

the access frequency is uniformly divided into a plurality of continuous intervals, and the record table is divided into a statistical unit according to the memory nodes and the frequency intervals;

traversing records in the record table, and finding out a corresponding statistical unit in the collection table;

the number of pages of each statistical unit, the mean and variance of the access frequency are counted.

6. The method for identifying and migrating a hot page of a memory according to claim 1, wherein the step S4 comprises:

step S4.1: for each memory node to be migrated, inquiring the corresponding target memory node;

step S4.2: establishing two scanning pointers of a hot page pointer and a cold page pointer in a collection table, wherein the hot page pointer scans a statistic unit list corresponding to a memory node to be migrated, and the scanning direction is from hot to cold; the cold page pointer scans a statistic unit list corresponding to the target memory node, and the scanning direction is from cold to hot;

step S4.3: scanning the collection table until the access frequency pointed by the cold page pointer is higher than the access frequency pointed by the hot page pointer;

step S4.4: recording the number of pages swept by the cold and hot pointers and the average value of the access frequency pointed by the cold and hot pointers at the end;

the step S4.2 includes:

the hot page pointer points to the statistics unit corresponding to the maximum value of the access frequency, and the cold page pointer points to the statistics unit corresponding to the minimum value of the access frequency;

if the number of the residual pages of the statistical unit pointed by the hot page pointer is larger than the number of the residual pages corresponding to the cold page pointer, the cold page pointer moves, the number of the residual pages corresponding to the hot page pointer is subtracted from the number of the residual pages corresponding to the cold page pointer, and the scanned number of the pages is updated;

if the number of the residual pages of the statistical unit pointed by the hot page pointer is smaller than the number of the residual pages corresponding to the cold page pointer, the hot page pointer moves, the number of the residual pages corresponding to the cold page pointer is reduced by the number of the residual pages corresponding to the hot page pointer, and the scanned number of the pages is updated;

and when the process is finished, calculating the specific access frequency according to the residual page quantity proportion of the statistical unit pointed by the cold and hot pointer, the average value and the variance of the access frequency.

7. The method for identifying and migrating a hot page of a memory according to claim 1, wherein the step S5 comprises:

step S5.1: according to the result obtained in the step S4, calculating a heat threshold and a migration rate by combining the sampling rate;

step S5.2: updating the calculated result into a collection table of each memory node;

step S5.3: collecting the statistics results of the tables in the time sequence updated by each process and eliminating the outdated statistics results according to the LRU strategy;

step S5.4: and (3) carrying out weighted average on the statistical results in the collection table according to the number of pages, and updating two system parameters, namely the heat threshold value and the migration rate of the memory node.

8. The method for identifying and migrating a hot page of a memory according to claim 5, wherein the step S6 of screening the hot page of the memory based on the hot threshold and the migration rate generated in the step S5 comprises:

step S6.1: comparing the recent page access frequency with the magnitude of the heat threshold, and judging that the page is cold if the frequency is lower than the heat threshold; if the frequency is not lower than the heat threshold, judging that the page is hot;

step S6.2: if the page is judged to be a hot page, judging to be a transferable page if the number of page migration in the statistical period does not reach the maximum number specified by the migration rate; otherwise, judging whether the page is a transferable page according to the strategy;

if a conservative policy is selected, the page is determined to be a non-migratable page; if the aggressive strategy is selected, the heat threshold is improved, if the page heat is higher than the improved threshold, the page is judged to be a transferable page, otherwise, the page is judged to be an non-transferable page;

step S6.3: and migrating the memory data of the transferable pages, updating the position mapping relation between the memory data and the virtual pages, and completing the page migration process.

9. A system for hot page identification and migration in a memory, comprising:

and the heat calculating module is used for: calculating access frequency by using page table scanning and page access time, so as to estimate the recent access heat of the memory page;

heat screening system: the method is used for judging whether the memory page accessed at the time should be migrated;

and a heat statistics module: the method is used for counting the heat distribution condition of the memory pages of each memory node;

and a heat analysis module: sorting and analyzing the heat statistics results of the memory nodes;

and a summarizing module: summarizing the heat analysis results of each process on the memory nodes, and calculating the reference value of the heat screening system parameters;

and a migration module: and transferring the screened memory pages to the target memory node by calling other system interfaces.

10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor performs the steps in the method for hot page identification and migration of memory of any one of claims 1 to 8.