CN115878300A - Management method and device of heterogeneous storage system - Google Patents

Management method and device of heterogeneous storage system Download PDF

Info

Publication number
CN115878300A
CN115878300A CN202111134158.4A CN202111134158A CN115878300A CN 115878300 A CN115878300 A CN 115878300A CN 202111134158 A CN202111134158 A CN 202111134158A CN 115878300 A CN115878300 A CN 115878300A
Authority
CN
China
Prior art keywords
virtual memory
storage medium
memory page
memory pages
allocated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111134158.4A
Other languages
Chinese (zh)
Inventor
张旭舟
黎亮
杨昕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202111134158.4A priority Critical patent/CN115878300A/en
Publication of CN115878300A publication Critical patent/CN115878300A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The application discloses a management method and a device of a heterogeneous storage system, wherein the method comprises the following steps: acquiring M characteristic values corresponding to M virtual memory pages respectively, wherein the ith characteristic value in the M characteristic values is determined by the state information of the ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \ 8230; based on M characteristic values, allocating a storage medium to each virtual memory page in M virtual memory pages from N different storage media, and updating a mapping relationship of the ith virtual memory page by using the storage medium allocated to the ith virtual memory page when the storage medium allocated to the ith virtual memory page is different from the storage medium to which the ith virtual memory page is mapped. By adopting the method and the device, the storage space required by the process can be dynamically configured based on the characteristics of the storage medium, so that the characteristics (performance, power consumption and the like) of the heterogeneous storage system can meet the requirements of users in different scenes.

Description

Management method and device of heterogeneous storage system
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for managing a heterogeneous storage system.
Background
In the computer structure, the storage system provides data and instruction access function for the computer, and different types of storage media in the storage system usually have different access speeds, power consumptions, capacities and time delays. Storage media of different nature contained in a computer constitute a heterogeneous storage system of the computer. For a heterogeneous storage system, the memory of a computer program is reasonably and dynamically allocated on the storage device of the computer by using the principle of locality of the program, so that the purposes of saving cost and improving performance can be achieved.
The Memory management Technology of the heterogeneous Memory system in the prior art mainly includes Least Recently Used (LRU) mechanism of Linux and Intel Memory Drive Technology (IMDT).
The memory management technology in the prior art uniformly allocates and manages different storage media in the heterogeneous storage system, and cannot meet the requirements of users on different characteristics (such as performance, power consumption and the like) of the heterogeneous storage system in different scenes.
Disclosure of Invention
The embodiment of the application provides a management method and a management device for a heterogeneous storage system, which can dynamically configure a storage space required by a process based on the characteristics of a storage medium, so that the characteristics (performance, power consumption and the like) of the heterogeneous storage system can meet the requirements of users in different scenes.
In a first aspect, the present application provides a method for managing a heterogeneous storage system, where the heterogeneous storage system includes N different storage media, and the method includes: obtaining M characteristic values corresponding to M virtual memory pages respectively, wherein an ith characteristic value in the M characteristic values is determined by state information of an ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \8230;, M; allocating a storage medium to each virtual memory page in the M virtual memory pages from the N different storage media based on the M characteristic values; when the storage medium allocated to the ith virtual memory page is different from the storage medium to which the ith virtual memory page is mapped, updating the mapping relationship of the ith virtual memory page by using the storage medium allocated to the ith virtual memory page.
The M virtual memory pages may be virtual memory pages corresponding to one process or virtual memory pages corresponding to multiple processes.
In view of technical effects, since the characteristic value corresponding to the virtual memory page is determined by the state information of the virtual memory page, the characteristic value of each virtual memory page may reflect the comprehensive characteristics of each virtual memory page. In different scenarios, the corresponding characteristic value of each virtual memory page may be determined based on different manners, that is, different characteristics of the virtual memory page may be reflected when the characteristic value corresponding to the virtual memory page is determined in different manners. Therefore, the characteristic value of the virtual memory page can be determined in a corresponding mode based on the requirement of the user on the characteristics of the heterogeneous storage system, and different storage media are allocated to the virtual memory page so as to meet the specific requirement of the user. For example, the characteristic values are determined in different ways, and then when different storage media are allocated to the M virtual memory pages based on the characteristic values, the heterogeneous storage system after the subsequent mapping relationship update can have the best read-write performance, the lowest power consumption, or the largest number of processes at the same time.
In a possible implementation manner, the state information of the ith virtual memory page includes frequency of accessing the ith virtual memory page and power consumption generated by operating a storage medium to which the ith virtual memory page is mapped.
The access frequency refers to the number of times that a physical memory page corresponding to a virtual memory page is accessed in a unit time. The above power consumption refers to power consumption generated when the storage medium is operated per unit time.
In view of technical effects, since the characteristic value of each virtual memory page is determined by the corresponding state information such as access frequency and power consumption, the characteristic value may reflect the frequency with which each virtual memory page is accessed and the power consumption of the storage medium having a mapping relationship with each virtual memory page, and after determining the characteristic value based on different manners and allocating the corresponding storage medium to the virtual memory page based on the characteristic value, the heterogeneous storage system may meet a specific requirement of a user, for example, performance is optimal or power consumption is lowest.
In a possible implementation manner, the storage media allocated to P virtual memory pages in the M virtual memory pages are the same, the P virtual memory pages correspond to P characteristic values with consecutive values among the M characteristic values, and P is a positive integer smaller than or equal to M.
In view of technical effects, the present application may allocate the same storage medium to virtual memory pages with continuous characteristic values, that is, allocate the same storage medium to virtual memory pages with similar characteristics, thereby effectively satisfying the principle of program spatial locality.
In a feasible implementation manner, the M virtual memory pages include Q virtual memory pages, where the Q virtual memory pages correspond to Q characteristic values with consecutive values among the M characteristic values, and a minimum value among the P characteristic values is greater than a maximum value among the Q characteristic values; the storage medium to which the P virtual memory pages are allocated is a first storage medium among the N different storage media, and the storage medium to which the Q virtual memory pages are allocated is a second storage medium among the N different storage media; wherein the sum of P and Q is less than or equal to M, and N is less than or equal to M.
In view of technical effects, in the embodiment of the present application, different storage media may be allocated to virtual memory pages with different characteristics based on characteristic values, and the same storage media may be allocated to virtual memory pages with similar characteristics, that is, performance of different storage media in a heterogeneous storage system is utilized to the greatest extent, so as to achieve performance improvement of a computer device.
In a possible implementation manner, when the size of the characteristic value is positively correlated to how many times the virtual memory page is accessed, the read-write speed of the first storage medium is greater than or equal to the read-write speed of the second storage medium.
From the technical effect, when the storage media are allocated to the virtual memory pages by taking the running performance of the computing device as an index, the method and the device can allocate the storage media with high read-write speed to the virtual memory pages which are frequently accessed, allocate the storage media with low read-write speed to the virtual memory pages which are frequently accessed, and further utilize the performance of different storage media in heterogeneous storage media to the greatest extent, so that the computing device achieves the best performance.
In a possible implementation manner, when the size of the characteristic value is positively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, the power consumption of the storage medium to which the ith virtual memory page is allocated is less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
From the technical effect, when the power consumption of the computing device is used as an index to allocate the storage medium for the virtual memory page, the power consumption of the storage medium allocated for the virtual memory page in the present application may be less than or equal to the power consumption of the storage medium mapped to the virtual memory page, so as to achieve an effect of reducing the power consumption of the entire heterogeneous storage system and the computing device.
In a possible implementation manner, the M virtual memory pages correspond to a first process, and the method further includes: acquiring K characteristic values corresponding to K virtual memory pages, respectively, where the K virtual memory pages correspond to a second process, each characteristic value in the K characteristic values is determined by state information of a corresponding virtual memory page, and K is a positive integer; based on the K feature values, allocating a storage medium to each of the K virtual memory pages from the N different storage media, where the storage medium allocated to C virtual memory pages in the K virtual memory pages is the same as the storage medium allocated to D virtual memory pages in the M virtual memory pages, the C virtual memory pages correspond to C feature values with continuous values among the K feature values, the D virtual memory pages correspond to D feature values with continuous values among the M feature values, a ratio of C to K is equal to a ratio of D to M, and C and D are positive integers; when the storage medium allocated to the jth virtual memory page in the K virtual memory pages is different from the storage medium to which the jth virtual memory page is mapped, updating a mapping relationship of the jth virtual memory page, where j =1, \\ 8230;, K, by using the storage medium allocated to the jth virtual memory page.
From the technical effect, for M virtual memory pages and K virtual memory pages corresponding to the first process and the second process respectively, the virtual memory pages in the M virtual memory pages and the K virtual memory pages in the same proportion can be divided into the same storage medium, it is avoided that all the virtual memory pages corresponding to the same process are divided into the storage medium with high read-write speed, thereby affecting the states of other processes, that is, the operating states of two processes (or a plurality of different processes) can be kept consistent through the distribution mode in the embodiment of the present application, thereby the process with the largest number of simultaneous operations on the computing device is enabled, at this time, when the computing device is a server device, corresponding services can be provided for more users at the same time, and thereby the server efficiency is improved.
It should be understood that, in the embodiment of the present application, the first process and the second process are only described as an example, and when multiple processes exist in a computing device at the same time, allocation of storage media corresponding to virtual memory pages may be performed in the same proportion manner as described above, which is not described herein again.
In a second aspect, an embodiment of the present application provides a management apparatus for a heterogeneous storage system, where the heterogeneous storage system includes N different storage media, where N is an integer greater than or equal to 2, and the apparatus includes: an obtaining unit, configured to obtain M feature values corresponding to M virtual memory pages, where an ith feature value of the M feature values is determined by state information of an ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \8230;, M; an allocating unit, configured to allocate, based on the M characteristic values, a storage medium for each virtual memory page in the M virtual memory pages from the N different storage media; an updating unit, configured to update the mapping relationship of the ith virtual memory page by using the storage medium to which the ith virtual memory page is allocated when the storage medium to which the ith virtual memory page is allocated is different from the storage medium to which the ith virtual memory page is mapped.
In a possible implementation manner, the state information of the ith virtual memory page includes frequency of access to the ith virtual memory page and power consumption generated by operating a storage medium to which the ith virtual memory page is mapped.
In a feasible implementation manner, the allocated storage media of P virtual memory pages in the M virtual memory pages are the same, where the P virtual memory pages correspond to P distinct values that have consecutive values among the M distinct values, and P is a positive integer smaller than or equal to M.
In a feasible implementation manner, the M virtual memory pages include Q virtual memory pages, where the Q virtual memory pages correspond to Q characteristic values with consecutive values among the M characteristic values, and a minimum value among the P characteristic values is greater than a maximum value among the Q characteristic values; the storage medium to which the P virtual memory pages are allocated is a first storage medium of the N different storage media, and the storage medium to which the Q virtual memory pages are allocated is a second storage medium of the N different storage media; wherein the sum of P and Q is less than or equal to M, and N is less than or equal to M.
In a possible implementation manner, when the size of the characteristic value is positively correlated to how many times the virtual memory page is accessed, the read-write speed of the first storage medium is greater than or equal to the read-write speed of the second storage medium.
In a possible implementation manner, when the size of the characteristic value is positively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, the power consumption of the storage medium to which the ith virtual memory page is allocated is less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
In a feasible implementation manner, the M virtual memory pages correspond to a first process, and the obtaining unit is further configured to: acquiring K characteristic values corresponding to K virtual memory pages respectively, wherein the K virtual memory pages correspond to a second process, each characteristic value in the K characteristic values is determined by state information of the corresponding virtual memory page, and K is a positive integer; the allocation unit is further configured to: based on the K feature values, allocating a storage medium to each of the K virtual memory pages from the N different storage media, where the storage medium allocated to C virtual memory pages in the K virtual memory pages is the same as the storage medium allocated to D virtual memory pages in the M virtual memory pages, the C virtual memory pages correspond to C feature values with continuous values among the K feature values, the D virtual memory pages correspond to D feature values with continuous values among the M feature values, a ratio of C to K is equal to a ratio of D to M, and C and D are positive integers; the update unit is further configured to: when the storage medium allocated to the jth virtual memory page in the K virtual memory pages is different from the storage medium to which the jth virtual memory page is mapped, updating a mapping relationship of the jth virtual memory page, where j =1, \\ 8230;, K, by using the storage medium allocated to the jth virtual memory page.
In a third aspect, an embodiment of the present application provides a chip system, where the chip system includes at least one processor, a memory, and an interface circuit, where the memory, the interface circuit, and the at least one processor are interconnected by lines, and where the at least one memory has instructions stored therein; when the instructions are executed by the processor, the method of any one of the above first aspects is implemented.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having stored therein program instructions that, when executed on one or more processors, implement the method of any of the first aspects described above.
In a fifth aspect, the present application provides a computer program product, when the computer program product runs on a computer device, the method of any one of the above first aspects is implemented.
Drawings
The drawings used in the embodiments of the present application are described below.
FIG. 1 is a system architecture diagram of a computer device according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating a management method of a heterogeneous storage system according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a mapping relationship updating process in an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a heterogeneous storage system in an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a heterogeneous storage system management apparatus according to an embodiment of the present application;
fig. 6 is a schematic hardware structure diagram of a computer device in an embodiment of the present application.
Detailed Description
The embodiments of the present application are described below with reference to the drawings. The terms "first," "second," "third," and "fourth," etc. in the description and claims of this application and in the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The relevant terms in the embodiments of the present application are explained first:
(1) Procedure locality: including temporal locality and spatial locality. Temporal locality refers to the fact that if a certain instruction/data in a program is executed once, the instruction may be executed again shortly thereafter. Spatial locality refers to the fact that if some data is accessed, it is possible that data in the vicinity of this data in a storage area is subsequently accessed.
(2) Virtual memory page: in the memory management mechanism of an operating system, the operating system manages a basic management unit of a virtual memory space. The virtual memory pages correspond to physical memory pages on the storage device side.
(3) The heterogeneous storage system: a Memory system composed of Memory hardware with different performance (e.g. different parameters such as read/write speed and bandwidth) includes, for example, a Phase Change Memory (PCM), a Dynamic Random Access Memory (DRAM), a Storage Class Memory (SCM), and a magnetic disk.
Referring to fig. 1, fig. 1 is a schematic diagram of a system architecture of a computer device according to an embodiment of the present disclosure. As shown in fig. 1, a system architecture of a computer device 100 to which the embodiment of the present application is applicable includes an application layer 110 (user mode), an operating system 120 (kernel mode), and a heterogeneous storage system 130.
Optionally, the computer device 100 may be a mobile phone, a computer, a tablet, a server, a wearable device, or the like, which is not limited in this application.
Optionally, the application layer 110 may include an application layer 111 and an application framework layer 112. The application layer 111 may include a series of application packages, among other things. The application packages may include camera, gallery, calendar, phone, map, navigation, bluetooth, music, video, short message, virtual Machine (VM), etc. applications (also referred to as applications). The Application framework layer 112 provides an Application Programming Interface (API) and a Programming framework for the Application programs of the Application layer 111. The application framework layer includes a number of predefined functions.
Alternatively, operating system 120 may include a file system, a block layer, and a device driver (not shown in FIG. 1). The file system is used for managing and scheduling storage space of files, and provides a logical structure, a physical structure and a storage method of the files. The block layer is an interface for the file system to access the heterogeneous storage system 130, and is used to connect the file system and the device driver. The block layer may be divided into two layers: one is the bio layer and the other is the request layer. The block layer is used to encapsulate/decapsulate the associated request. The device driver may include a display driver, a camera driver, an audio driver, a sensor driver, and the like.
Alternatively, heterogeneous storage system 130 may include storage media of different storage capabilities.
The running of the program can be divided into a user mode and a kernel mode. When the program runs in the user mode, the processor can only access data in a part of the memory, and peripheral devices such as a hard disk, a network card and the like are not allowed to be accessed. When the program runs in the kernel mode, the processor may access all data in the memory, including peripheral devices such as a hard disk, a network card, and the like. At the same time, the processor may also switch itself from one program to another. Typically, the application programs in the application layer 110 run in a user mode and the operating system 120 runs in a kernel mode.
In the running process of the computer device 100, the application layer 110 initiates a memory state information scanning request to instruct the operating system 120 to acquire state information corresponding to multiple virtual memory pages (including access frequency of a physical memory page to which each virtual memory page is mapped and power consumption of a storage medium corresponding to each virtual memory page); distributing corresponding storage media for each virtual memory page based on the state information corresponding to each virtual memory page; and finally, when the storage medium to which the virtual memory page is allocated is different from the storage medium to which the virtual memory page is mapped, updating the mapping relation of the virtual memory page.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a management method of a heterogeneous storage system according to an embodiment of the present disclosure. The heterogeneous storage system includes N different storage media, N being an integer greater than or equal to 2. As shown in fig. 2, method 200 includes: step S210, step S220, and step S230.
Step S210: m characteristic values corresponding to M virtual memory pages are obtained, wherein the ith characteristic value of the M characteristic values is determined by state information of the ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \8230;, M.
The M virtual memory pages may be virtual memory pages corresponding to one process or virtual memory pages corresponding to multiple processes.
In a possible implementation manner, the state information of the ith virtual memory page includes frequency of accessing the ith virtual memory page and power consumption generated by operating a storage medium to which the ith virtual memory page is mapped.
Wherein each virtual memory page corresponds to one physical memory page in the storage medium to which it is mapped.
Wherein, the accessed frequency of the ith virtual memory page refers to: and in unit time, the frequency of access to the physical memory page corresponding to the ith virtual memory page is increased. The above power consumption means: and power consumption generated when the storage medium corresponding to the ith virtual memory page is operated in unit time.
In view of technical effects, since the characteristic value of each virtual memory page is determined by the corresponding state information such as access frequency, power consumption, and the like, the characteristic value may reflect the access frequency of each virtual memory page and the power consumption of the storage medium having a mapping relationship with each virtual memory page, and after the characteristic value is determined based on different manners and the corresponding storage medium is allocated to the virtual memory page based on the characteristic value, the heterogeneous storage system may meet a specific requirement of a user, for example, performance is optimal or power consumption is lowest.
Optionally, the obtaining manner of the state information corresponding to the M virtual memory pages may be as follows: in the application layer 110, one or more application programs may be specified by a user, that is, storage media need to be allocated to virtual memory pages corresponding to the one or more application programs; ko traverses the Page tables (Page Table Entry, PTE) corresponding to the one or more application programs, respectively, in the operating system 120, and then records an access flag bit (access bit, used for recording the number of times that a corresponding virtual memory Page is accessed) corresponding to each virtual memory Page in each Page Table into a process file (e.g.,/proc/PID/idle _ pages) corresponding to each Page Table; finally, a scanning period may be defined by a user in the application layer 110 to periodically scan the process files respectively corresponding to the one or more application programs, so as to obtain the frequency of accessing the virtual memory pages corresponding to the one or more application programs in the scanning period. Meanwhile, the address information of each virtual memory page may be obtained from the process file corresponding to the one or more application programs, where the address information of each virtual memory page includes a start address and a page length of each virtual memory page. In addition, the type of the storage medium to which each virtual memory page is mapped may be obtained from a process file corresponding to the one or more application programs, and then a power consumption of the storage medium to which each virtual memory page is mapped may be queried based on the type of the storage medium, where the power consumption may be a power consumption of the storage medium per unit time (e.g., milliwatt per minute (mW/min)) or a power consumption of the storage medium in a scanning cycle (mW/T, where T is a scanning cycle).
Alternatively, after the operating system 120 acquires the state information of each virtual memory page, the state information corresponding to each virtual memory page may be converted into a vma _ msg data structure shown in table 1, and sent to the application layer 110.
Table 1: data structure of corresponding state information of each virtual memory page
Figure BDA0003281905780000061
As shown in Table 1, the data structure of the corresponding state information for each virtual memory page may include the codes of lines 1-8 in Table 1. The line 2-line 5 codes indicate address information of the virtual memory page, including a start address (line 3 code) and a page length (line 4 code) of the virtual memory page. In the code of line 6, the character "count" indicates the number of times the virtual memory page is accessed in the scanning cycle, and in the code of line 7, the character "power" is the power consumption of the storage medium to which the virtual memory page is mapped.
It should be understood that table 1 is only one possible data structure of the status information of each virtual memory page, and those skilled in the art may also use other data structures to characterize the status information of each virtual memory page, which is not limited in this application.
Alternatively, the determination manner of the ith characteristic value may be defined by the user according to specific requirements. Specifically, when a user has a specific requirement on the heterogeneous storage system (for example, the requirement is optimal in performance, lowest in power consumption, or the maximum number of processes running simultaneously), the determination manner of the characteristic value of each virtual memory page may be defined according to the requirement.
Further, optionally, the user may assign a weight coefficient a to the number of times a each virtual memory page is accessed, and assign a weight coefficient B to the power consumption B of the storage medium to which each virtual memory page is mapped; and then calculating a characteristic value corresponding to each virtual memory page through formula (1).
Eigenvalue = a + B (1)
It should be understood that the user may dynamically configure the weighting factors a and b according to his specific requirements for the heterogeneous storage system.
Step S220: based on the M characteristic values, allocating a storage medium to each virtual memory page in the M virtual memory pages from the N different storage media.
Specifically, the M eigenvalues are sorted to obtain the sorted M eigenvalues. And dividing the sorted M characteristic values into E sets, wherein each set comprises at least one characteristic value, and when the number of the characteristic values in the set is more than 1, all the characteristic values in the set are continuous in value. Further, for any two sets of the E sets, the minimum value in one set is greater than the maximum value in the other set, or the maximum value in the one set is less than the minimum value in the other set. Wherein E is a positive integer and E is less than or equal to N.
The sorting mode of the M eigenvalues may be sorting from large to small, or sorting from small to large.
In a possible implementation manner, the storage media allocated to P virtual memory pages in the M virtual memory pages are the same, the P virtual memory pages correspond to P characteristic values with consecutive values among the M characteristic values, and P is a positive integer smaller than or equal to M.
Specifically, the P feature values having consecutive values may be feature values included in any one of the E sets. That is, in the E sets, the storage media allocated to the virtual memory pages corresponding to all the eigenvalues in each set are the same.
In view of technical effects, the embodiment of the present application may allocate the same storage medium to virtual memory pages with consecutive eigenvalues, that is, allocate the same storage medium to virtual memory pages with similar characteristics, so as to effectively satisfy the principle of program spatial locality.
Optionally, the process of dividing the sorted M eigenvalues into E sets may specifically be: and dividing the sorted M characteristic values into E sets based on the capacity of the virtual memory pages. Specifically, tables 2 and 3 list two set partitioning rules.
As shown in table 2, D1, D2, \8230andde are preset e capacity thresholds in the set partitioning rule 1, which is not limited in the present application. The set partitioning rule 1 shown in table 2 is such that: the sum of the capacity of the virtual memory pages corresponding to all the characteristic values in the set 1 is D1, the sum of the capacity of the virtual memory pages corresponding to all the characteristic values in the set 2 is D2, \ 8230, and the sum of the capacity of the virtual memory pages corresponding to all the characteristic values in the set E is De. The storage medium to which the virtual memory pages corresponding to all the characteristic values in the set 1 are allocated is a storage medium 1, the storage medium to which the virtual memory pages corresponding to all the characteristic values in the set 2 are allocated is a storage medium 2, \ 8230, and the storage medium to which the virtual memory pages corresponding to all the characteristic values in the set E are allocated is a storage medium E.
As shown in table 3, in the set partitioning rule 1, F2, \8230, and Fe are preset e capacity ratios, which is not limited in the present application. The set partitioning rule 2 shown in table 3 is such that: the proportion of the sum of the capacities of the virtual memory pages corresponding to all the characteristic values in the set 1 in the sum of the capacities of the M virtual memory pages is F1, the proportion of the sum of the capacities of the virtual memory pages corresponding to all the characteristic values in the set 2 in the sum of the capacities of the M virtual memory pages is F2, \ 8230;, and the proportion of the sum of the capacities of the virtual memory pages corresponding to all the characteristic values in the set E in the sum of the capacities of the M virtual memory pages is Fe. The storage medium to which the virtual memory pages corresponding to all the characteristic values in the set 1 are allocated is a storage medium 1, the storage medium to which the virtual memory pages corresponding to all the characteristic values in the set 2 are allocated is a storage medium 2, \ 8230, and the storage medium to which the virtual memory pages corresponding to all the characteristic values in the set E are allocated is a storage medium E.
In the set division rule 1 and the set division rule 2, the minimum value in the set 1 is greater than the maximum value in the set 2, the minimum value in the set 2 is greater than the maximum value in the set 3, \8230; or the maximum value in the set 1 is smaller than the minimum value in the set 2, the maximum value in the set 2 is smaller than the minimum value in the set 3, \8230, and the maximum value in the set E-1 is smaller than the minimum value in the set E, which is not limited in the application.
Table 2: set division rule 1 of sorted M characteristic values
Storage medium type Collection Capacity/GB
Storage medium 1 Set 1 D1
Storage medium 2 Set 2 D2
Storage medium E Set E De
Table 3: set division rule 2 of sorted M characteristic values
Type of storage medium Collection of Capacity fraction/%)
Storage medium 1 Set 1 F1
Storage medium 2 Set 2 F2
Storage medium E Set E Fe
In a feasible implementation manner, the M virtual memory pages include Q virtual memory pages, where the Q virtual memory pages correspond to Q consecutive characteristic values among the M characteristic values, and a minimum value among the P characteristic values is greater than a maximum value among the Q characteristic values; the storage medium to which the P virtual memory pages are allocated is a first storage medium of the N different storage media, and the storage medium to which the Q virtual memory pages are allocated is a second storage medium of the N different storage media; wherein the sum of P and Q is less than or equal to M, and N is less than or equal to M.
Specifically, the allocating a storage medium to each virtual memory page in the M virtual memory pages from the N different storage media based on the M characteristic values includes: e different storage media are respectively allocated to the E sets to which the M eigenvalues respectively belong, that is, the storage media allocated to the virtual memory pages corresponding to all eigenvalues in each set are the same, and the storage media allocated to the two eigenvalues arbitrarily belonging to different sets respectively correspond to the virtual memory pages are different.
The E sets comprise a first set and a second set, the first set comprises the P characteristic values, the second set comprises the Q characteristic values, and the minimum value in the first set is larger than the maximum value in the second set. In this case, the storage medium to which the P virtual memory pages corresponding to the P characteristic values are allocated is a first storage medium among the N storage media, and the storage medium to which the Q virtual memory pages corresponding to the Q characteristic values are allocated is a second storage medium among the N storage media. The first storage medium and the second storage medium are two different storage media among N storage media included in the heterogeneous storage system.
In view of technical effects, in the embodiment of the present application, different storage media may be allocated to virtual memory pages with different characteristics based on characteristic values, and the same storage media may be allocated to virtual memory pages with similar characteristics, that is, performance of different storage media in a heterogeneous storage system is utilized to the greatest extent, so as to achieve performance improvement of a computer device.
The rules for allocating storage media to virtual memory pages based on characteristic values in three different scenarios are described below:
scene one: requiring performance optimization of heterogeneous storage systems
Scene two: requiring power consumption minimization for heterogeneous memory systems
Scene three: it is desirable that a computer device containing a heterogeneous storage system can run the maximum number of processes simultaneously
In a first scenario, the M virtual memory pages may be virtual memory pages corresponding to one process or virtual memory pages corresponding to multiple processes. In a first scenario, the rule for allocating a storage medium to each virtual memory page is as follows:
in a possible implementation manner, when the size of the characteristic value is positively correlated with how many times the virtual memory page is accessed, the read-write speed of the first storage medium is greater than or equal to the read-write speed of the second storage medium.
The size of the characteristic value is positively correlated with the access frequency, specifically: the higher the frequency of accessing the virtual memory page, the larger the characteristic value corresponding to the virtual memory page. In this case, the weight coefficient a in the above formula (1) may be set to be larger than the weight coefficient b, for example, the weight coefficient a may be set to 0.99, and the weight coefficient b may be set to 0.01.
When the size of the characteristic value is positively correlated with the access frequency of the virtual memory pages, because the minimum value in the first set is greater than the maximum value in the second set, it indicates that the minimum access frequency of the P access frequencies of the P virtual memory pages corresponding to the P characteristic values in the first set is greater than the maximum access frequency of the Q access frequencies of the Q virtual memory pages corresponding to the Q characteristic values in the second set. At this time, in order to achieve the best performance of the heterogeneous storage system, the read-write speed of the allocated first storage medium may be greater than or equal to the read-write speed of the allocated second storage medium. For example, in this case, the first storage medium may be a DRAM, and the second storage medium may be an SCM, which is not limited in this application.
Optionally, when the size of the characteristic value is negatively related to how many access frequencies of the virtual memory page are, the read-write speed of the first storage medium is less than or equal to the read-write speed of the second storage medium. That is, the higher the frequency of accessing the virtual memory page, the smaller the characteristic value corresponding to the virtual memory page. In this case, the weight coefficient a in the above formula (1) may be set smaller than the weight coefficient b, for example, the weight coefficient a may be set to-0.99, and the weight coefficient b may be set to 0.01.
When the size of the eigenvalue is negatively correlated with the access frequency of the virtual memory pages, because the minimum value in the first set is greater than the maximum value in the second set, it indicates that the minimum access frequency of the P access frequencies of the P virtual memory pages corresponding to the P eigenvalues in the first set is greater than the maximum access frequency of the Q access frequencies of the Q virtual memory pages corresponding to the Q eigenvalues in the second set. At this time, in order to achieve the best performance of the heterogeneous storage system, the read-write speed of the allocated first storage medium may be made smaller than or equal to the read-write speed of the allocated second storage medium. For example, at this time, the first storage medium may be a Solid State Disk (SSD), and the second storage medium may be a DRAM, which is not limited in this application.
It should be understood that, for any two sets of the E sets, the storage media may be allocated according to a rule that the storage media are allocated to the feature values in the first set and the second set, and the description is omitted here.
From the technical effect, when the storage media are allocated to the virtual memory pages by taking the running performance of the computing device as an index, the method and the device can allocate the storage media with high read-write speed to the virtual memory pages which are frequently accessed, allocate the storage media with low read-write speed to the virtual memory pages which are frequently accessed, and further utilize the performance of different storage media in heterogeneous storage media to the greatest extent, so that the computing device achieves the best performance.
In a second scenario, the M virtual memory pages may be virtual memory pages corresponding to one process or virtual memory pages corresponding to multiple processes. In scenario two, the rule for allocating storage media to each virtual memory page is as follows:
in a possible implementation manner, when the size of the characteristic value is positively correlated with the power consumption of the storage medium corresponding to the virtual memory page, the power consumption of the storage medium allocated to the ith virtual memory page is less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
The size of the characteristic value is positively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, specifically: the higher the power consumption of the storage medium to which the virtual memory page is mapped is, the larger the characteristic value corresponding to the virtual memory page is. In this case, the weight coefficient a in the above formula (1) may be set smaller than the weight coefficient b, for example, the weight coefficient a may be set to 0.01 and the weight coefficient b may be set to 0.99.
When the size of the characteristic value is positively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, in order to achieve the lowest power consumption of the heterogeneous storage system, the first storage medium may be the storage medium to which the virtual memory page corresponding to the smallest characteristic value in the first set is mapped, and the second storage medium may be the storage medium to which the virtual memory page corresponding to the smallest characteristic value in the second set is mapped. By the allocation method, in the first set, for any eigenvalue except the minimum eigenvalue, the power consumption of the storage medium to which the virtual memory page corresponding to the any eigenvalue is allocated is less than the power consumption of the storage medium to which the virtual memory page corresponding to the any eigenvalue is mapped. That is, by the allocation method, the power consumption of the storage medium allocated to the ith virtual memory page of the M virtual memory pages may be less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
Optionally, when the size of the characteristic value is negatively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, the power consumption of the storage medium to which the ith virtual memory page is allocated is less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped. Specifically, the higher the power consumption of the storage medium to which the virtual memory page is mapped is, the smaller the characteristic value corresponding to the virtual memory page is. In this case, the weight coefficient a in the above formula (1) may be set to be larger than the weight coefficient b, for example, the weight coefficient a is set to 0.01 and the weight coefficient b is set to-0.99.
When the size of the characteristic value is negatively related to the power consumption of the storage medium to which the virtual memory page is mapped, in order to achieve the lowest power consumption of the heterogeneous storage system, the first storage medium may be the storage medium to which the virtual memory page corresponding to the largest characteristic value in the first set is mapped, and the second storage medium may be the storage medium to which the virtual memory page corresponding to the largest characteristic value in the second set is mapped. By the allocation method, in the first set, for any eigenvalue except the maximum eigenvalue, the power consumption of the storage medium to which the virtual memory page corresponding to the eigenvalue is allocated is less than the power consumption of the storage medium to which the virtual memory page corresponding to the eigenvalue is mapped. That is, through the foregoing allocation manner, the power consumption of the storage medium allocated to the ith virtual memory page in the M virtual memory pages may be less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
From the technical effect, when the storage medium is allocated to the virtual memory page by using the power consumption of the computing device as an index, the power consumption of the storage medium allocated to the virtual memory page in the present application may be less than or equal to the power consumption of the storage medium mapped to the virtual memory page, so as to achieve the effect of reducing the power consumption of the entire heterogeneous storage system and the computing device.
In a third scenario, the M virtual memory pages may be virtual memory pages corresponding to one process. A third scenario mainly describes a process of allocating corresponding storage media to virtual memory pages respectively corresponding to multiple processes at the same time, where a rule for allocating storage media to each virtual memory page is as follows:
in a feasible implementation manner, the M virtual memory pages correspond to a first process, and the method further includes: acquiring K characteristic values corresponding to K virtual memory pages respectively, wherein the K virtual memory pages correspond to a second process, each characteristic value in the K characteristic values is determined by state information of the corresponding virtual memory page, and K is a positive integer; based on the K feature values, allocating a storage medium to each of the K virtual memory pages from the N different storage media, where the storage medium allocated to C virtual memory pages in the K virtual memory pages is the same as the storage medium allocated to D virtual memory pages in the M virtual memory pages, the C virtual memory pages correspond to C feature values with continuous values among the K feature values, the D virtual memory pages correspond to D feature values with continuous values among the M feature values, a ratio of C to K is equal to a ratio of D to M, and C and D are positive integers; when the storage medium allocated to the jth virtual memory page in the K virtual memory pages is different from the storage medium to which the jth virtual memory page is mapped, updating the mapping relationship of the jth virtual memory page by using the storage medium to which the jth virtual memory page is allocated, wherein j =1, \ 8230;, K.
It should be understood that the determination manner of the K characteristic values corresponding to the K virtual memory pages respectively may be the same as the determination manner of each characteristic value in the M characteristic values in the foregoing embodiment, and is not described herein again.
In scenario three, the set division manner of the M eigenvalues and the set division manner of the K eigenvalues may be the same as set division rule 2 in table 3. The sorted M eigenvalues are divided into E sets: m1, M2, \8230, ME. The sorted K eigenvalues are divided into E sets: k1, K2, \ 8230and KE, and the specific division process is not described herein.
And the sorting rule of the M characteristic values is the same as the sorting rule of the K characteristic values.
After the set division is performed, the sets M1, M2, \ 8230, and ME are respectively allocated to different storage media, and specifically, the E sets may respectively allocate corresponding storage media to the E sets according to the allocation manner that the heterogeneous storage system has the best performance or the lowest power consumption based on the size of the feature value, which is not described herein again.
Further, in the 2E sets, all feature values in the set with the same number may be allocated to the same storage medium corresponding to the virtual memory page: that is, the storage media to which the virtual memory pages corresponding to the characteristic values in the set M1 and the set K1 are allocated are the same, the storage media to which the virtual memory pages corresponding to the characteristic values in the set M2 and the set K2 are allocated are the same, \8230, and the storage media to which the virtual memory pages corresponding to the characteristic values in the set ME and the set KE are allocated are the same.
The C eigenvalues and the D eigenvalues may be eigenvalues included in the set K1 and the set M1, or eigenvalues included in the set K2 and the set M2, \8230, or eigenvalues included in the set KE and the set ME, respectively. I.e. the ratio of C to K is equal to the ratio of D to M.
Specifically, the manner and the process for updating the jth virtual memory page mapping relationship may specifically refer to the following description of the process for updating the ith virtual memory page mapping relationship in step S230, and the updating process is the same, which is not described in detail herein.
It should be understood that the above embodiments only describe the allocation policy of the storage medium by taking two processes (i.e., the first process and the second process) as an example. When the storage media are allocated to the virtual memory pages corresponding to the two or more processes at the same time, any two processes of the two or more processes may adopt the same storage media allocation manner as the first process and the second process, so that the running state of each process is similar, and details are not described here.
From the technical effect, for M virtual memory pages and K virtual memory pages corresponding to the first process and the second process respectively, the virtual memory pages in the M virtual memory pages and the K virtual memory pages in the same proportion can be divided into the same storage medium, it is avoided that all the virtual memory pages corresponding to the same process are divided into the storage medium with high read-write speed, thereby affecting the states of other processes, that is, the operating states of two processes (or a plurality of different processes) can be kept consistent through the distribution mode in the embodiment of the present application, thereby the process with the largest number of simultaneous operations on the computing device is enabled, at this time, when the computing device is a server device, corresponding services can be provided for more users at the same time, and thereby the server efficiency is improved.
It should be understood that, in the embodiment of the present application, the first process and the second process are only described as an example, and when multiple processes exist in a computing device at the same time, allocation of storage media corresponding to virtual memory pages may be performed in the same proportion manner as described above, which is not described herein again.
In summary, by the storage medium allocation method of the virtual memory pages in the third scenario, the maximum number of processes can be simultaneously run on the computer device, and the running states of the processes are the same or close to each other. When the computing device is a server device, corresponding services can be provided for more users at the same time, and therefore the working efficiency of the server is improved.
Step S230: when the storage medium to which the ith virtual memory page is allocated is different from the storage medium to which the ith virtual memory page is mapped, updating the mapping relationship of the ith virtual memory page by using the storage medium to which the ith virtual memory page is allocated.
The storage medium to which the ith virtual memory page is mapped refers to a storage medium to which a physical memory page having a mapping relationship with the ith virtual memory page belongs.
Specifically, the updating the mapping relationship of the ith virtual memory page by using the storage medium allocated to the ith virtual memory page specifically includes: for an ith virtual memory page of the M virtual memory pages, when a storage medium allocated to the ith virtual memory page is different from a storage medium to which the ith virtual memory page is mapped, the storage medium allocated to the ith virtual memory page is set as a storage medium having a mapping relationship with the ith virtual memory page, that is, a physical memory page having a mapping relationship with the ith virtual memory page is located in the storage medium allocated in place of the ith virtual memory page.
It should be understood that, in the scenario three, for K virtual memory pages, an update process of any virtual memory page mapping relationship may be the same as an update process of the ith virtual memory page mapping relationship, and details are not described here again.
In view of technical effects, since the characteristic value corresponding to the virtual memory page is determined by the state information of the virtual memory page, the characteristic value of each virtual memory page may reflect the comprehensive characteristics of each virtual memory page. In different scenarios, the corresponding characteristic value of each virtual memory page may be determined based on different manners, that is, different characteristics of the virtual memory page may be reflected when the characteristic value corresponding to the virtual memory page is determined in different manners. Therefore, the characteristic value of the virtual memory page can be determined in a corresponding mode based on the requirements of the user on the characteristics of the heterogeneous storage system, and different storage media are distributed to the virtual memory page so as to meet the specific requirements of the user. For example, the characteristic values are determined in different ways, and then when different storage media are allocated to the M virtual memory pages based on the characteristic values, the heterogeneous storage system after the subsequent mapping relationship update can have the best read-write performance, the lowest power consumption, or the largest number of processes at the same time.
Table 4: code implementation for virtual memory page mapping relationship update
Figure BDA0003281905780000121
The implementation manner of the foregoing update process of the mapping relationship of the ith virtual memory page in the application layer 110 may be as follows:
first, when the storage medium to which the ith virtual memory page is allocated is different from the storage medium to which the ith virtual memory page is mapped, an update rule of the mapping relationship of the ith virtual memory page is generated, as shown in table 4.
Referring to table 4, table 4 illustrates a data structure of an update rule of a mapping relationship in code. As shown in table 4, the character "addr" indicates address information of the ith virtual memory page, including the start address and length information of the ith virtual memory page (corresponding to the 3 rd to 4 th line codes in the virtual memory page status information in table 1); the character "source" represents a storage medium to which the ith virtual memory page is mapped; the character "destination" refers to a storage medium allocated for the ith virtual memory page.
Taking a Linux operating system which is preinstalled with a function library for managing NUMA nodes and configured with SWAP partitions as an example, the following describes an updating process of a mapping relationship:
referring to fig. 3, fig. 3 is a schematic diagram illustrating a mapping relationship updating process according to an embodiment of the present disclosure. As shown in fig. 3, the update process includes:
step S310: and receiving an update rule of the mapping relation of the ith virtual memory page.
Step S320: and judging whether the storage medium allocated to the ith virtual memory page and the storage medium mapped to the ith virtual memory page are both storage media in the memory part.
Specifically, the determination is made based on the storage medium position to which the ith virtual memory page is mapped in the update rule and the position of the storage medium to which the ith virtual memory page is allocated, where whether both storage media are storage media included in the memory part in the heterogeneous storage system is determined, and if yes, step S321 is executed; if not, go to step S322.
Step S321: the move _ page interface of null is called to instruct the operating system 120 to update the mapping relationship. Specifically, when both the storage medium to which the ith virtual memory page is mapped and the storage medium to which the ith virtual memory page is allocated are storage media included in a memory portion of the heterogeneous storage system, a move _ page interface of numactl may be called in the application layer 110 to instruct the operating system 120 to update the mapping relationship.
Step S322: the operating system 120 is instructed to use the switch Swap function for map updates. Specifically, when one of the storage media to which the ith virtual memory page is mapped and the storage media to which the ith virtual memory page is allocated is a storage medium included in the memory portion in the heterogeneous storage system, and the other is a storage medium included in the external memory portion in the heterogeneous storage system, the operating system 120 may be instructed to use the Swap function to update the mapping relationship.
Step S330: and ending the updating process of the mapping relation.
For the memory portion and the external memory portion in the heterogeneous storage system, reference may be made to the description in fig. 4.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a heterogeneous storage system according to an embodiment of the present disclosure. As shown in fig. 4, heterogeneous storage system 130 may include a memory portion 131 and a memory portion 132.
The memory portion 131 refers to a storage medium directly accessible by a Central Processing Unit (CPU), the storage medium included in the memory portion 131 is a volatile storage medium, and the storage medium in the memory portion 131 may include a DRAM, an SCM, and the like, which is not limited in this application.
The external memory portion 132 refers to a storage medium that cannot be directly accessed by the CPU, data in the external memory portion 132 needs to be transferred to the internal memory portion 131 to be accessed by the CPU, the storage media included in the external memory portion 132 are all non-volatile storage media, and the storage media in the external memory portion 132 may include a Solid State Disk (SSD) and a mechanical Hard Disk (Hard Disk Drive, HDD), which is not limited in this application. The Swap partition in the external memory portion 132 is a logical partition, for example, the Swap partition may include a part or all of the memory space on the SSD and a part or all of the memory space on the HDD, or only include a part or all of the memory space on the SSD, or only include a part or all of the memory space on the HDD.
Table 5: performance and cost example table for heterogeneous storage systems in the present application
Figure BDA0003281905780000131
Please refer to table 5, where table 5 is an example table of performance and cost of the heterogeneous storage system after the storage medium is allocated to the virtual memory page by using the management method of the heterogeneous storage system in the embodiment of the present application. Table 5 shows three examples, one for each row of data. In each example, the size of each of the M characteristic values is positively correlated with the number of times of accessing the virtual memory page, and the M characteristic values may be sorted in descending order, and then divided into 3 sets according to the set division rule 2.
In the first row example, the capacity of the memory portion of the heterogeneous storage system comprises 8G and 12G of DRAM and SCM, respectively, and the capacity of the Swap partition of the external memory portion is 4G. The ratio of the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 1, the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 2, and the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 3 is as follows: 36:54:10. the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 1 are allocated are DRAMs, the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 2 are allocated are SCMs, and the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 3 are allocated are storage media in Swap partitions. And then updating the mapping relationship of each storage medium based on the storage medium allocated to each virtual memory page, and the specific updating process is not described herein again. The Transaction number Per Minute (Transaction Per Minute Total, tpmTotal) of the heterogeneous storage system after the mapping relation is updated is 74053.9; compared with the prior art which does not use the management method in the application, the performance of the heterogeneous storage system is improved by 10.98%; compared with a storage system of 20 GDRAMs, the performance of the heterogeneous storage system is reduced by 1.9%, but the cost can be saved by 45%.
In the second row example, the memory portion of the heterogeneous storage system includes DRAMs and SCMs with capacities of 6G and 14G, respectively, and the Swap partition of the external memory portion has a capacity of 4G. The ratio of the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 1, the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 2, and the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 3 is as follows: 27:63:10. the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 1 are allocated are DRAMs, the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 2 are allocated are SCMs, and the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 3 are allocated are storage media in Swap partitions. And then updating the mapping relationship of each storage medium based on the storage medium allocated to each virtual memory page, and the specific updating process is not described herein again. The per-minute transaction processing quantity of the heterogeneous storage system after the mapping relation is updated is 73356.3; compared with the prior art which does not use the management method in the application, the performance of the heterogeneous storage system is improved by 23.40%; compared with a storage system of 20 GDRAMs, the performance of the heterogeneous storage system is reduced by 2.8%, but the cost can be saved by 52%.
In the third row example, the capacity of the DRAM and SCM included in the memory portion of the heterogeneous memory system is 4G and 16G, respectively, and the capacity of the Swap partition in the external memory portion is 4G. The ratio of the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 1, the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 2, and the sum of the page capacities of the virtual memory corresponding to all the characteristic values in the set 3 is as follows: 18:72:10. the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 1 are allocated are DRAMs, the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 2 are allocated are SCMs, and the storage media to which the virtual memory pages corresponding to all the characteristic values in the set 3 are allocated are storage media in Swap partitions. And then updating the mapping relationship of each storage medium based on the storage medium allocated to each virtual memory page, and the specific updating process is not described herein again. The per-minute transaction processing quantity of the heterogeneous storage system after the mapping relation is updated is 73463.5; compared with the prior art which does not use the management method in the application, the performance of the heterogeneous storage system is improved by 44.28 percent; compared with a storage system of 20 GDRAMs, the performance of the heterogeneous storage system is reduced by 2.7%, but the cost can be saved by 60%.
In summary, the method in the embodiment of the present application is used to manage a heterogeneous storage system, and compared with a storage system using a single high-performance storage medium, the method can greatly save cost while sacrificing a small amount of performance. In addition, compared with a heterogeneous storage system which does not use the management method in the application, the application can effectively improve the performance of the heterogeneous storage system.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a management apparatus for a heterogeneous storage system according to an embodiment of the present disclosure. The heterogeneous storage system includes N different storage media, N being an integer greater than or equal to 2. As shown in fig. 5, the apparatus 500 includes an obtaining unit 501, an allocating unit 502, and an updating unit 503.
An obtaining unit 501, configured to obtain M feature values corresponding to M virtual memory pages, where an ith feature value in the M feature values is determined by state information of an ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \8230, M.
An allocating unit 502, configured to allocate, based on the M characteristic values, a storage medium to each virtual memory page in the M virtual memory pages from the N different storage media.
An updating unit 503, configured to update the mapping relationship of the ith virtual memory page by using the storage medium to which the ith virtual memory page is allocated, when the storage medium to which the ith virtual memory page is allocated is different from the storage medium to which the ith virtual memory page is mapped.
In a possible implementation manner, the state information of the ith virtual memory page includes frequency of accessing the ith virtual memory page and power consumption generated by operating a storage medium to which the ith virtual memory page is mapped.
In a feasible implementation manner, the allocated storage media of P virtual memory pages in the M virtual memory pages are the same, where the P virtual memory pages correspond to P distinct values that have consecutive values among the M distinct values, and P is a positive integer smaller than or equal to M.
In a feasible implementation manner, the M virtual memory pages include Q virtual memory pages, where the Q virtual memory pages correspond to Q consecutive characteristic values among the M characteristic values, and a minimum value among the P characteristic values is greater than a maximum value among the Q characteristic values; the storage medium to which the P virtual memory pages are allocated is a first storage medium of the N different storage media, and the storage medium to which the Q virtual memory pages are allocated is a second storage medium of the N different storage media; wherein the sum of P and Q is less than or equal to M, and N is less than or equal to M.
In a possible implementation manner, when the size of the characteristic value is positively correlated to how many times the virtual memory page is accessed, the read-write speed of the first storage medium is greater than or equal to the read-write speed of the second storage medium.
In a possible implementation manner, when the size of the characteristic value is positively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, the power consumption of the storage medium to which the ith virtual memory page is allocated is less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
In a feasible implementation manner, the M virtual memory pages correspond to a first process, and the obtaining unit is further configured to: acquiring K characteristic values corresponding to K virtual memory pages respectively, wherein the K virtual memory pages correspond to a second process, each characteristic value in the K characteristic values is determined by state information of the corresponding virtual memory page, and K is a positive integer; the allocation unit is further configured to: based on the K feature values, allocating a storage medium to each of the K virtual memory pages from the N different storage media, where the storage medium allocated to C virtual memory pages in the K virtual memory pages is the same as the storage medium allocated to D virtual memory pages in the M virtual memory pages, the C virtual memory pages correspond to C feature values with continuous values among the K feature values, the D virtual memory pages correspond to D feature values with continuous values among the M feature values, a ratio of C to K is equal to a ratio of D to M, and C and D are positive integers; the update unit is further configured to: when the storage medium allocated to the jth virtual memory page in the K virtual memory pages is different from the storage medium to which the jth virtual memory page is mapped, updating the mapping relationship of the jth virtual memory page by using the storage medium to which the jth virtual memory page is allocated, wherein j =1, \ 8230;, K.
Specifically, for a specific implementation process of the apparatus 500, reference may be made to corresponding descriptions in the foregoing method embodiments of fig. 2, fig. 3, and fig. 4, which are not described herein again.
Referring to fig. 6, fig. 6 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present application. As shown in fig. 6, the computer device 600 includes a processor 601, a memory 602, an interface circuit 603, and a bus 604. Computer device 600 may be computer device 100 in fig. 1. The processor 601, the memory 602, and the interface circuit 603 perform data transmission via the bus 604.
The memory 602 is used for storing a computer program product, which may be an application program or other forms of program products, etc., but is not limited thereto.
A processor 601, configured to obtain M feature values corresponding to M virtual memory pages, where an ith feature value of the M feature values is determined by state information of an ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \8230;, M; allocating a storage medium to each of the M virtual memory pages from the N different storage media based on the M characteristic values; when the storage medium allocated to the ith virtual memory page is different from the storage medium to which the ith virtual memory page is mapped, updating the mapping relationship of the ith virtual memory page by using the storage medium allocated to the ith virtual memory page.
Specifically, in this embodiment of the present application, for a specific operation process of a processor and a memory on the computer device 600, reference may be made to corresponding processes in the foregoing method embodiments of fig. 2, fig. 3, and fig. 4, which are not described herein again.
The embodiment of the application provides a chip system, which comprises at least one processor, a memory and an interface circuit, wherein the memory, the interface circuit and the at least one processor are interconnected through lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, the method of any of the method embodiments of fig. 2 is implemented.
Embodiments of the present application provide a computer-readable storage medium, in which program instructions are stored, and when the program instructions are executed on one or more processors, any one or more of the embodiments described in the method shown in fig. 2 are implemented.
The present application provides a computer program product, and when the computer program product runs on a computer device, any one or more of the embodiments described in the method shown in fig. 2 is implemented.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to the related descriptions of other embodiments. It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units is only one type of division of logical functions, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or units, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present application.

Claims (17)

1. A method for managing a heterogeneous storage system, wherein the heterogeneous storage system includes N different storage media, N being an integer greater than or equal to 2, the method comprising:
obtaining M characteristic values corresponding to M virtual memory pages respectively, wherein an ith characteristic value in the M characteristic values is determined by state information of the ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \ 8230;
allocating a storage medium to each virtual memory page in the M virtual memory pages from the N different storage media based on the M characteristic values;
when the storage medium allocated to the ith virtual memory page is different from the storage medium to which the ith virtual memory page is mapped, updating the mapping relationship of the ith virtual memory page by using the storage medium allocated to the ith virtual memory page.
2. The method of claim 1,
the state information of the ith virtual memory page includes an accessed frequency of the ith virtual memory page and power consumption generated by operating a storage medium to which the ith virtual memory page is mapped.
3. The method according to claim 1 or 2,
the allocated storage media of P virtual memory pages in the M virtual memory pages are the same, the P virtual memory pages correspond to P characteristic values with continuous values among the M characteristic values, and P is a positive integer smaller than or equal to M.
4. The method of claim 3,
the M virtual memory pages include Q virtual memory pages, where the Q virtual memory pages correspond to Q characteristic values with continuous values among the M characteristic values, and a minimum value of the P characteristic values is greater than a maximum value of the Q characteristic values;
the storage medium to which the P virtual memory pages are allocated is a first storage medium of the N different storage media, and the storage medium to which the Q virtual memory pages are allocated is a second storage medium of the N different storage media;
wherein the sum of P and Q is less than or equal to M, and N is less than or equal to M.
5. The method of claim 4,
when the size of the characteristic value is positively correlated with the accessed frequency of the virtual memory page, the read-write speed of the first storage medium is greater than or equal to the read-write speed of the second storage medium.
6. The method according to claim 4 or 5,
when the size of the characteristic value is positively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, the power consumption of the storage medium to which the ith virtual memory page is allocated is less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
7. The method according to any of claims 1 to 6, wherein the M virtual memory pages correspond to a first process, the method further comprising:
acquiring K characteristic values corresponding to K virtual memory pages respectively, wherein the K virtual memory pages correspond to a second process, each characteristic value in the K characteristic values is determined by state information of the corresponding virtual memory page, and K is a positive integer;
based on the K feature values, allocating a storage medium to each of the K virtual memory pages from the N different storage media, where the storage medium allocated to C virtual memory pages in the K virtual memory pages is the same as the storage medium allocated to D virtual memory pages in the M virtual memory pages, the C virtual memory pages correspond to C feature values with continuous values among the K feature values, the D virtual memory pages correspond to D feature values with continuous values among the M feature values, a ratio of C to K is equal to a ratio of D to M, and C and D are positive integers;
when the storage medium allocated to the jth virtual memory page in the K virtual memory pages is different from the storage medium to which the jth virtual memory page is mapped, updating a mapping relationship of the jth virtual memory page, where j =1, \\ 8230;, K, by using the storage medium allocated to the jth virtual memory page.
8. An apparatus for managing a heterogeneous storage system, wherein the heterogeneous storage system includes N different storage media, N being an integer greater than or equal to 2, the apparatus comprising:
an obtaining unit, configured to obtain M feature values corresponding to M virtual memory pages, where an ith feature value of the M feature values is determined by state information of an ith virtual memory page, M is an integer greater than or equal to 2, and i =1, \ 8230;
an allocating unit, configured to allocate, based on the M characteristic values, a storage medium for each virtual memory page in the M virtual memory pages from the N different storage media;
and an updating unit, configured to update the mapping relationship of the ith virtual memory page by using the storage medium to which the ith virtual memory page is allocated when the storage medium to which the ith virtual memory page is allocated is different from the storage medium to which the ith virtual memory page is mapped.
9. The apparatus of claim 8,
the state information of the ith virtual memory page includes an accessed frequency of the ith virtual memory page and power consumption generated by operating a storage medium to which the ith virtual memory page is mapped.
10. The apparatus of claim 8 or 9,
the allocated storage media of P virtual memory pages in the M virtual memory pages are the same, the P virtual memory pages correspond to P characteristic values with continuous values among the M characteristic values, and P is a positive integer smaller than or equal to M.
11. The apparatus of claim 10,
the M virtual memory pages include Q virtual memory pages, where the Q virtual memory pages correspond to Q characteristic values with consecutive values among the M characteristic values, and a minimum value among the P characteristic values is greater than a maximum value among the Q characteristic values;
the storage medium to which the P virtual memory pages are allocated is a first storage medium of the N different storage media, and the storage medium to which the Q virtual memory pages are allocated is a second storage medium of the N different storage media;
wherein the sum of P and Q is less than or equal to M, and N is less than or equal to M.
12. The apparatus of claim 11,
when the size of the characteristic value is positively correlated with the accessed frequency of the virtual memory page, the read-write speed of the first storage medium is greater than or equal to the read-write speed of the second storage medium.
13. The apparatus of claim 11 or 12,
when the size of the characteristic value is positively correlated with the power consumption of the storage medium to which the virtual memory page is mapped, the power consumption of the storage medium to which the ith virtual memory page is allocated is less than or equal to the power consumption of the storage medium to which the ith virtual memory page is mapped.
14. The apparatus according to any of claims 8-13, wherein the M virtual memory pages correspond to a first process,
the acquisition unit is further configured to:
acquiring K characteristic values corresponding to K virtual memory pages respectively, wherein the K virtual memory pages correspond to a second process, each characteristic value in the K characteristic values is determined by state information of the corresponding virtual memory page, and K is a positive integer;
the allocation unit is further configured to:
based on the K feature values, allocating a storage medium to each of the K virtual memory pages from the N different storage media, where the storage medium allocated to C virtual memory pages in the K virtual memory pages is the same as the storage medium allocated to D virtual memory pages in the M virtual memory pages, the C virtual memory pages correspond to C feature values with continuous values among the K feature values, the D virtual memory pages correspond to D feature values with continuous values among the M feature values, a ratio of C to K is equal to a ratio of D to M, and C and D are positive integers;
the update unit is further configured to:
when the storage medium allocated to the jth virtual memory page in the K virtual memory pages is different from the storage medium to which the jth virtual memory page is mapped, updating a mapping relationship of the jth virtual memory page, where j =1, \\ 8230;, K, by using the storage medium allocated to the jth virtual memory page.
15. A system on chip, the system on chip comprising at least one processor, memory and interface circuitry, the memory, the interface circuitry and the at least one processor interconnected by wires, the at least one memory having instructions stored therein; when the instructions are executed by the processor, the method of any one of claims 1-7 is implemented.
16. A computer-readable storage medium, having stored therein program instructions, which when run on one or more processors, implement the method of any one of claims 1-7.
17. A computer program product, characterized in that the method of any one of claims 1-7 is implemented when the computer program product is run on a computer device.
CN202111134158.4A 2021-09-27 2021-09-27 Management method and device of heterogeneous storage system Pending CN115878300A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111134158.4A CN115878300A (en) 2021-09-27 2021-09-27 Management method and device of heterogeneous storage system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111134158.4A CN115878300A (en) 2021-09-27 2021-09-27 Management method and device of heterogeneous storage system

Publications (1)

Publication Number Publication Date
CN115878300A true CN115878300A (en) 2023-03-31

Family

ID=85762839

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111134158.4A Pending CN115878300A (en) 2021-09-27 2021-09-27 Management method and device of heterogeneous storage system

Country Status (1)

Country Link
CN (1) CN115878300A (en)

Similar Documents

Publication Publication Date Title
CN111506262B (en) Storage system, file storage and reading method and terminal equipment
US9792227B2 (en) Heterogeneous unified memory
EP2645259B1 (en) Method, device and system for caching data in multi-node system
CN104583979A (en) Techniques for dynamic physical memory partitioning
US20180032429A1 (en) Techniques to allocate regions of a multi-level, multi-technology system memory to appropriate memory access initiators
US20140040541A1 (en) Method of managing dynamic memory reallocation and device performing the method
US20170062025A1 (en) Memory system including plural memory devices forming plural ranks and memory controller accessing plural memory ranks and method of operating the memory system
CN106598724B (en) Method for managing memory in a computing system
WO2024078429A1 (en) Memory management method and apparatus, computer device, and storage medium
JP6674460B2 (en) System and method for improved latency in a non-uniform memory architecture
JP7205033B2 (en) Cache allocation method and device, storage medium, electronic device
US6457107B1 (en) Method and apparatus for reducing false sharing in a distributed computing environment
US20170364442A1 (en) Method for accessing data visitor directory in multi-core system and device
US11157191B2 (en) Intra-device notational data movement system
CN113138851B (en) Data management method, related device and system
CN112817836A (en) Method, device, equipment and storage medium for determining server load
CN112596913A (en) Method and device for improving performance of transparent large page of memory, user equipment and storage medium
CN115729849B (en) Memory management method and computing device
CN105453066A (en) Processor and memory control method
CN113138941A (en) Memory exchange method and device
US20220382672A1 (en) Paging in thin-provisioned disaggregated memory
CN115878300A (en) Management method and device of heterogeneous storage system
JP2018502379A (en) System and method for enabling improved latency in heterogeneous memory architectures
CN1333346C (en) Method for accessing files
WO2021218502A1 (en) Page swapping method, storage system, and electronic device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication