CN116263703A - Transparent large page distribution method and device, electronic equipment and readable medium - Google Patents
Transparent large page distribution method and device, electronic equipment and readable medium Download PDFInfo
- Publication number
- CN116263703A CN116263703A CN202111525032.XA CN202111525032A CN116263703A CN 116263703 A CN116263703 A CN 116263703A CN 202111525032 A CN202111525032 A CN 202111525032A CN 116263703 A CN116263703 A CN 116263703A
- Authority
- CN
- China
- Prior art keywords
- memory
- allocator
- transparent
- memory area
- page
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
- Memory System (AREA)
Abstract
The invention provides a transparent large page allocation method, a transparent large page allocation device, electronic equipment and a readable medium, wherein in the method, a system memory is divided into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator; under the condition that a transparent large page allocation request sent by a target program is received, allocating the transparent large page for the target program on the basis of a general memory allocator in a common memory area; in the case where the general-purpose memory allocator cannot allocate a transparent large page, the continuous memory is allocated for the target program based on the specific memory allocator to allocate the transparent large page. Therefore, under the condition that the transparent large page cannot be allocated based on the common memory, the transparent large page is allocated for the target program based on the specific memory allocator, the success rate of allocation of the transparent large page is improved, and the system performance can be improved to a certain extent.
Description
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a transparent large page distribution method and device, electronic equipment and a readable medium.
Background
With the rapid development of computer hardware, the memory capacity is also increasing, but the memory capacity is limited, and it is impossible to put all programs and data required by all user processes and systems into the memory, so the operating system must reasonably divide and effectively dynamically allocate the memory space, that is, dynamically manage the memory.
During memory management and use, a processor stores a mapping relationship between virtual addresses and physical memory based on page tables (page tables). The page table tends to default to a smaller predetermined number of bytes of memory pages, such as 4 Kilobytes (KB) or 16 kilobytes, etc. With the increase of the physical memory accessed by the user process, the number of the page tables is gradually increased, and the storage of a large number of page tables occupies a large amount of storage resources and reduces the processing performance of the processor. On this basis, operating systems have introduced a mechanism for large pages (HugePage), which may refer to relatively large pages, typically 2 Megabytes (MB) or 32 megabytes, etc.
In the prior art, in the process of allocating transparent large pages by a processor based on an operating system, after a kernel system runs for a long time, immovable memory pages which are randomly distributed in a physical memory can appear, so that memory space fragmentation is caused, transparent large pages are difficult to separate, the allocation success rate of the transparent large pages is low, and the system performance is influenced.
Disclosure of Invention
The embodiment of the invention provides a transparent large page distribution method, a transparent large page distribution device, electronic equipment and a readable medium, which are used for improving the success rate of transparent large page distribution and improving the system performance.
In a first aspect, a transparent large page allocation method is provided, including:
dividing a system memory into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator;
under the condition that a transparent large page allocation request sent by a target program is received, allocating the transparent large page for the target program based on the general memory allocator in the general memory area;
and if the general-purpose memory allocator cannot allocate the transparent big page, allocating continuous memory for the target program based on the specific memory allocator to allocate the transparent big page.
Optionally, the allocating the transparent big page for the target program based on the general memory allocator in the general memory area includes:
determining the size of a target memory required by the transparent large page;
determining a continuous memory matched with the target memory in the common memory area;
and distributing the transparent big page for the target program based on the continuous memory.
Optionally, the allocating continuous memory for the target program based on the specific memory allocator to allocate the transparent big page includes:
determining an alternative memory area in the specific memory area through the specific memory allocator under the condition that no continuous memory matched with the target memory size exists in the common memory area; the alternative memory area comprises allocated memory pages;
clearing the allocated memory pages in the alternative memory area to obtain the continuous memory;
and distributing the transparent big page for the target program based on the continuous memory.
Optionally, the clearing the allocated memory page in the candidate memory area to obtain the continuous memory includes:
copying the allocated memory pages in the alternative memory area to other memory areas;
and mapping the virtual address corresponding to the allocated memory page to the other storage area so as to empty the alternative memory area.
Optionally, the specific memory allocator is a continuous memory allocator.
In a second aspect, there is provided a transparent large page dispensing device comprising:
the dividing module is used for dividing the system memory into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator;
the first allocation module is used for allocating the transparent big page for the target program based on the general memory allocator in the common memory area under the condition that a transparent big page allocation request sent by the target program is received;
and the second allocation module is used for allocating continuous memory for the target program based on the specific memory allocator to allocate the transparent big page under the condition that the general memory allocator cannot allocate the transparent big page.
Optionally, the first allocation module is specifically configured to:
determining the size of a target memory required by the transparent large page;
determining a continuous memory matched with the target memory in the common memory area; allocating the transparent big page for the target program based on the continuous memory
Optionally, the second allocation module is specifically configured to:
determining an alternative memory area in the specific memory area through the specific memory allocator under the condition that no continuous memory matched with the target memory size exists in the common memory area; the alternative memory area comprises allocated memory pages;
clearing the allocated memory pages in the alternative memory area to obtain the continuous memory;
and distributing the transparent big page for the target program based on the continuous memory.
Optionally, the second allocation module is further specifically configured to:
copying the allocated memory pages in the alternative memory area to other memory areas;
and mapping the virtual address corresponding to the allocated memory page to the other storage area so as to empty the alternative memory area.
Optionally, the specific memory allocator is a continuous memory allocator.
In a third aspect, an electronic device is provided, comprising:
one or more processors; and one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the electronic device to perform the transparent large page allocation method as described in the first aspect.
In a fourth aspect, there is provided one or more machine readable media having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the transparent large page allocation method as described in the first aspect.
In the embodiment of the invention, the system memory is divided into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator; under the condition that a transparent large page allocation request sent by a target program is received, allocating the transparent large page for the target program on the basis of a general memory allocator in a common memory area; in the case where the general-purpose memory allocator cannot allocate a transparent large page, the continuous memory is allocated for the target program based on the specific memory allocator to allocate the transparent large page. In the embodiment of the invention, the transparent large page is distributed in the common memory area through the common memory distributor, and the transparent large page is distributed for the target program based on the specific memory distributor under the condition that the common memory cannot be distributed, so that the success rate of the transparent large page distribution is improved, and the system performance can be improved to a certain extent.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of steps of a transparent large page allocation method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of system memory space fragmentation;
FIG. 3 is a schematic diagram of transparent large page allocation according to an embodiment of the present invention;
fig. 4 is a block diagram of a transparent large page distributing device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
First, a transparent large page is described, the transparent large page is a large page allocation management method that a kernel actively tries to allocate large pages for an application program, the kernel will always try to use the large page to satisfy memory allocation, if there is no available large page (such as that physical continuous memory is not available), the kernel will fall back to a conventional 4KB page. Transparent large pages differ from hugetlbfs in that they are exchangeable: by breaking up large pages into smaller 4KB pages, which can then be swapped out normally. In the user state, the transparent large page can be used without modifying the application program, so the transparent large page is enabled by default in many systems.
Fig. 1 is a flowchart of steps of a transparent large page allocation method according to an embodiment of the present invention, where an execution body of the method may refer to a processor that may be installed with an operating system, and the operating system may refer to a Linux system or the like. As shown in fig. 1, the method may include:
In the embodiment of the present invention, the system memory may refer to an internal storage area included in the processor. The specific memory region may refer to a pre-divided internal memory region managed by a specific memory allocator. The specific memory allocator is used for managing the specific memory area, and the specific memory allocator can independently realize memory allocation for the specific memory area managed by the specific memory allocator. The movable memory page refers to a memory page which can be moved at will and does not influence the running of the application program.
The normal memory area may refer to other internal memory areas than the specific memory area among the internal memory areas. The common memory area is managed by a general memory allocator, which may refer to a memory allocator preset in a kernel operating system of the processor to allocate memory for a user process or an application program.
In this step, during the boot phase of the os kernel, the processor may divide a continuous memory area as a specific memory area, and the specific memory area is managed by a specific memory allocator. The specific division manner of the specific memory area may refer to determining a continuous memory area as the specific memory area based on configuration options of the kernel system in the kernel starting stage; or in the starting stage, the user sets address parameters in the internal memory area to determine a continuous memory area as a specific memory area. The embodiment of the invention is not limited to a specific division manner of the specific memory area. The special memory distributor and the general memory distributor work cooperatively, and the processor kernel system performs unified scheduling to jointly realize memory management and transparent large page distribution.
Note that the operating system kernel classifies allocated memory pages into non-removable pages, recoverable pages, and removable page 3 types. Wherein, the immovable pages have fixed positions in the memory and cannot be moved to other places; the recyclable pages cannot be moved directly, but can be deleted; while the movable pages can be moved at will, mapped by page tables. The specific memory allocator is just used for allocating movable pages, so that continuous memory required by the transparent large page can be allocated through the movement of the memory pages, and the success rate of allocation of the transparent large page is improved. The particular memory allocator may be referred to as a continuous memory allocator (Contiguous Memory Allocator, CMA), a memory allocator based on a modification of the continuous memory allocator, or the like, and embodiments of the present invention are not limited in this regard.
In the embodiment of the present invention, the target program may refer to a user program needing to allocate a transparent large page, and the target program may also include other user processes needing to allocate a transparent large page, which is not limited in the embodiment of the present invention. A transparent large page allocation request may refer to a request instruction sent by an object program to a processor to request allocation of a transparent large page.
In this step, when the number of page tables is large due to the fact that the user process accesses more physical memory, the number of memory pages is reduced by using large pages in the operating system, so that fewer page tables are required, the number of memory occupied by the page tables is reduced, the required address translation is reduced, the number of times of cache invalidation of the translation lookaside buffer (Translation Lookaside Buffer, TLB) is reduced, and therefore the performance of memory access is improved. In addition, because the information required by address conversion is generally stored in the cache of the processor, the use of large pages reduces the address conversion information, thereby reducing the use of the cache of the processor, relieving the pressure of the cache of the processor, enabling the cache of the processor to be used for data cache of application programs more and improving the performance of the system as a whole.
On the basis, in the case that the target program sends a transparent large page allocation request to the processor, the kernel system of the processor can allocate the transparent large page for the target program through the general memory allocator. In the embodiment of the invention, when the transparent large page is allocated for the user process or the application program, the allocation is preferentially performed on the basis of the common memory area through the common memory allocator, namely, the memory in the common memory area is preferentially used for allocating the transparent large page.
Step 103, in the case that the general-purpose memory allocator cannot allocate the transparent large page, allocating continuous memory for the target program based on the specific memory allocator to allocate the transparent large page.
In the embodiment of the present invention, the continuous memory may refer to a continuous memory area required for a transparent large page. After receiving the transparent large page allocation request sent by the target program, the processor may first attempt to allocate the transparent large page from the common memory area through the general memory allocator, and when the general memory allocator cannot allocate the transparent large page, allocate the transparent large page for the target program based on the specific memory allocator.
In this step, after the processor core system runs for a long time, a random distribution of non-movable memory occurs in the physical memory, resulting in fragmentation of memory space, and it is difficult to separate transparent large pages. At this time, the common memory area does not have continuous memory required for allocating the transparent large page, and allocation of the transparent large page cannot be realized based on the general memory allocator. In the embodiment of the invention, under the condition that the general memory distributor cannot distribute the transparent large page, the transparent large page can be distributed based on the specific memory area based on the specific memory distributor, so that the distribution rate of the transparent large page is improved.
Illustratively, FIG. 2 shows a schematic diagram of a system memory space fragmentation. As shown in fig. 2, in the initial stage of the processor, the memory occupancy rate is low, the continuous memory in the normal memory area is large, and transparent large pages are easily allocated based on the general memory allocator. With the long-time running of the kernel system, the system memory accessed by the user program is increased, and at this time, a plurality of non-movable memory pages distributed randomly appear in the system memory, so that the memory space is fragmented, and the general memory allocator cannot allocate continuous memory for the transparent large page. When the general-purpose memory allocator cannot allocate the transparent large page, in some implementations, a response of failure in allocation of the transparent large page is often returned to the target program, and a common page table, i.e. a small page, of 4KB or 16KB is allocated to the target program. This affects the performance of the system and also creates more memory pressure for the processor.
In the embodiment of the invention, when the general memory distributor cannot distribute the transparent large page, the transparent large page is distributed based on the specific memory distributor, so that the transparent large page can be further ensured to be redistributed, the success rate of the transparent large page distribution is improved, and the running performance of the system is improved. The size of the transparent large page may be set based on actual requirements, which is not particularly limited in the embodiment of the present invention. For example, when the common page table is 16KB, the transparent large page may be 32MB, because the transparent large page tends to directly reduce the page table level, and each level of page table usually uses a standard small page, for example, 16KB holds a 64-bit address of 8 bytes and can hold 2048, so that the size of the large page is 2048×16kb, i.e., 32MB. Of course, based on actual requirements, the size of the transparent large page can also be set by adopting the size of 2 small pages or 3 small pages, and the like, the transparent large page can be flexibly set based on the size of the small page, and then the memory area can be allocated based on the actual size of the transparent large page.
In summary, the transparent large page allocation method provided by the embodiment of the invention divides the system memory into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator; under the condition that a transparent large page allocation request sent by a target program is received, allocating the transparent large page for the target program on the basis of a general memory allocator in a common memory area; in the case where the general-purpose memory allocator cannot allocate a transparent large page, the continuous memory is allocated for the target program based on the specific memory allocator to allocate the transparent large page. In the embodiment of the invention, the transparent large page is distributed in the common memory area through the common memory distributor, and the transparent large page is distributed for the target program based on the specific memory distributor under the condition that the common memory cannot be distributed, so that the success rate of the transparent large page distribution is improved, and the system performance can be improved to a certain extent.
Optionally, in the embodiment of the present invention, step 102 may specifically include the following steps 1021 to 1023:
step 1021, determining the target memory size required by the transparent large page.
In the embodiment of the invention, the target memory size may refer to the size of the memory area required by the transparent large page. The actually required target memory size of the transparent large page may be determined based on the actual type and operation requirement of the target program, and specifically may refer to a memory size of 2 small pages, 3 small pages or 5 small pages, which is not limited in the embodiment of the present invention.
Step 1022, determining a continuous memory matching the target memory size in the normal memory area.
In the embodiment of the invention, the continuous memory can refer to a memory area which is allocated by a user and is identical to the target memory in size. After determining the target memory size required for the transparent large page, the contiguous memory may be determined in the normal memory region based on the general memory allocator.
Step 1023, distributing the transparent big page for the target program based on the continuous memory.
In the embodiment of the invention, after the continuous memory is determined based on the size of the target memory, the transparent large page can be allocated for the target program based on the general memory allocator.
In the embodiment of the invention, the size of the target memory required by the transparent large page is determined; determining a continuous memory matched with the size of the target memory in the common memory area; and allocating transparent large pages for the target program based on the continuous memory. In this way, in the embodiment of the invention, after receiving the transparent large page allocation request of the target program, the processor can preferentially try to allocate the transparent large page from the common memory area, so as to maximize the memory space utilization, and improve the efficiency of allocation of the transparent large page to a certain extent.
Optionally, in the embodiment of the present invention, step 103 may specifically include the following steps 1031 to 1033:
step 1031, determining an alternative memory area in the specific memory area by the specific memory allocator when no continuous memory matched with the target memory size exists in the common memory area; the candidate memory area comprises allocated memory pages.
In the embodiment of the present invention, the candidate memory area may refer to a memory area in a specific memory area, which is matched with the target memory size required by the transparent large page. Allocated memory pages may refer to removable memory pages that a particular memory allocator has allocated in an alternative memory region.
In this step, in the case that the general memory allocator cannot find a continuous memory area matching the target memory size in the general memory area, the transparent large page cannot be allocated based on the general memory area at this time. The processor may allocate transparent large pages for the target program based on the particular memory allocator. At this time, the specific memory allocator may determine, based on the target memory size, an alternative memory area that matches the target memory size, and then may perform a flushing operation on the alternative memory area to implement allocation of the transparent large page.
Step 1032, clearing the allocated memory pages in the alternative memory area, so as to obtain the continuous memory.
In the embodiment of the invention, the specific memory allocator can independently manage the specific memory area and is used for allocating the movable memory pages. After determining the candidate memory area, the processor may empty the allocated memory pages in the candidate memory area to obtain a continuous memory in which the transparent large page can be directly used.
Step 1033, allocating the transparent big page for the target program based on the continuous memory.
In the embodiment of the invention, after the continuous memory is obtained, the processor can allocate the transparent large page for the target program through the specific memory allocator, so that the target program can normally operate based on the transparent large page.
Illustratively, FIG. 3 shows a schematic diagram of a transparent large page allocation in accordance with an embodiment of the present invention. As shown in fig. 3, after receiving the transparent large page allocation request sent by the target program, the processor allocates the transparent large page first in the normal memory area 301 based on the general memory allocator, but there are many non-movable memory pages in the normal memory area 301, so that it is impossible to determine the continuous memory to allocate the transparent large page. At this time, the processor may allocate the transparent large page in the specific memory area 302 based on the specific memory allocator, and since the specific memory area 302 allocates the movable memory page, the processor may empty the alternative memory area to determine the continuous memory and further allocate the transparent large page, thereby improving the success rate of allocating the transparent large page, and also ensuring the continuity of allocating the transparent large page by the empty allocated memory page.
In the embodiment of the invention, under the condition that continuous memory matched with the target memory size does not exist in the common memory area, an alternative memory area is determined in the specific memory area through the specific memory allocator; the alternative memory area comprises allocated memory pages; clearing the allocated memory pages in the alternative memory area to obtain the continuous memory; and distributing the transparent big page for the target program based on the continuous memory. Therefore, under the condition that the transparent large page cannot be allocated through the common memory area, the transparent large page can be allocated in the specific memory area based on the specific memory allocator, and the success rate of the transparent large page is improved.
Optionally, in the embodiment of the present invention, step 1032 may specifically include the following steps S21 to S22:
and S21, copying the allocated memory pages in the alternative memory area to other memory areas.
In the embodiment of the present invention, the other storage area may refer to a common memory area or a hard disk, etc. When the allocated memory pages in the candidate memory area are emptied, the allocated memory pages may first be copied to other memory areas, and then the secure swap of the allocated memory pages may be achieved through the mapped conversion.
And S22, mapping the virtual address corresponding to the allocated memory page to the other storage area so as to empty the alternative memory area.
In the embodiment of the invention, after the allocated memory page is copied to other memory areas, the mapping relation between the virtual address and the actual physical memory, namely other memory areas, can be re-established, and then the alternative memory area does not have the mapping relation with the virtual address, namely the clearing operation of the alternative memory area is realized, the continuous memory which can be used for allocating transparent large pages is obtained, and the alignment of the addresses is realized. For example, if the size of the transparent large page is 32MB, through the above-mentioned flush operation, the addresses below 32MB of the head address of the candidate memory area are all 0, so that the requirements of the translation look-aside buffer TLB can be met, and the normal allocation of the transparent large page can be realized.
In the embodiment of the invention, the allocated memory pages in the alternative memory area are copied to other memory areas; and mapping the virtual address corresponding to the allocated memory page to other storage areas so as to empty the alternative memory area. Therefore, through the copying and mapping reconstruction operation of the allocated memory pages, the allocated memory pages can be safely exchanged to a hard disk or other memory areas, the continuous memory with aligned addresses is provided for the allocation of the transparent large pages, the success rate of the allocation of the transparent large pages is improved, and meanwhile, the continuity of the allocation of the transparent large pages can be improved through the safe exchange of the allocated memory.
Optionally, in an embodiment of the present invention, the specific memory allocator is a continuous memory allocator.
In the embodiment of the present invention, the continuous memory allocator CMA may refer to a specific memory allocator for managing a specific memory area. The continuous memory allocator can be used for allocating shared memory for the image processor (graphics processing unit, GPU), the shared memory can be used for a central processing unit (central processing unit, CPU) when the GPU is not used, but if the GPU needs to use larger memory, the memory pages of the CPU can be migrated if the CMA is not separated, so that the CMA memory allocator takes the property of limiting the memory pages of the CPU as a necessary condition when designing, and the problem of memory fragmentation caused by allocation of immovable pages is avoided. The continuous memory distributor commonly adopted in the prior art has no function of distributing transparent large pages of the CPU memory, but the embodiment of the invention adds a new function of distributing transparent large pages for the continuous memory distributor based on the characteristics of the continuous memory distributor, thereby improving the success rate of distributing transparent large pages.
Furthermore, applications to large pages in the operating system of a processor (e.g., linux system) are largely divided into two modes: a large page file system (Hugetlbfs) and transparent large pages (Transparent Huge Pages, THP). The large page file system is to manually pre-allocate large pages and map the large pages into a file, and a user program can acquire memory pre-allocated according to the large pages through the file. Transparent large pages refer to large pages dynamically allocated by the system. In one implementation, when transparent large pages are allocated based on a large page file system, manual operation allocation is required by a user, and the operation is complex and the allocation efficiency is low.
In the embodiment of the invention, under the condition of receiving the transparent large page allocation request, firstly, the transparent large page is tried to be allocated from the common memory allocation area, and if the transparent large page cannot be allocated, the transparent large page is allocated in the specific memory area through the specific memory allocation device, so that the success rate of transparent large page allocation is improved, and the continuity of transparent large page allocation is also improved.
Fig. 4 is a block diagram of a transparent large page distribution device according to an embodiment of the present invention, where the device 40 includes:
the dividing module 401 is configured to divide the system memory into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator;
a first allocation module 402, configured to, when receiving a transparent large page allocation request sent by a target program, allocate the transparent large page for the target program based on the general memory allocator in the general memory area;
and a second allocation module 403, configured to allocate continuous memory for the target program based on the specific memory allocator to allocate the transparent big page if the general memory allocator cannot allocate the transparent big page.
In summary, the transparent large page allocation device provided by the embodiment of the invention divides the system memory into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator; under the condition that a transparent large page allocation request sent by a target program is received, allocating the transparent large page for the target program on the basis of a general memory allocator in a common memory area; in the case where the general-purpose memory allocator cannot allocate a transparent large page, the continuous memory is allocated for the target program based on the specific memory allocator to allocate the transparent large page. In the embodiment of the invention, the transparent large page is distributed in the common memory area through the common memory distributor, and the transparent large page is distributed for the target program based on the specific memory distributor under the condition that the common memory cannot be distributed, so that the success rate of the transparent large page distribution is improved, and the system performance can be improved to a certain extent.
Optionally, the first allocation module 402 is specifically configured to:
determining the size of a target memory required by the transparent large page;
determining a continuous memory matched with the target memory in the common memory area; allocating the transparent big page for the target program based on the continuous memory
Optionally, the second allocation module 403 is specifically configured to:
determining an alternative memory area in the specific memory area through the specific memory allocator under the condition that no continuous memory matched with the target memory size exists in the common memory area; the alternative memory area comprises allocated memory pages;
clearing the allocated memory pages in the alternative memory area to obtain the continuous memory;
and distributing the transparent big page for the target program based on the continuous memory.
Optionally, the second allocation module 403 is further specifically configured to:
copying the allocated memory pages in the alternative memory area to other memory areas;
and mapping the virtual address corresponding to the allocated memory page to the other storage area so as to empty the alternative memory area.
Optionally, the specific memory allocator is a continuous memory allocator.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
Preferably, the embodiment of the invention also provides an electronic device, which comprises one or more processors; and one or more machine readable media having instructions stored thereon, which when executed by the one or more processors, cause the electronic device to perform the transparent large page allocation method provided by the above embodiments.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements each process of the transparent large page allocation method provided in the above embodiment, and can achieve the same technical effect, so that repetition is avoided, and no further description is provided here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
As will be readily appreciated by those skilled in the art: any combination of the above embodiments is possible, and thus is an embodiment of the present invention, but the present specification is not limited by the text.
The methods provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a system constructed with aspects of the present invention will be apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in the transparent large page allocation method according to embodiments of the present invention may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present invention can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
Claims (10)
1. A transparent large page allocation method, characterized by comprising:
dividing a system memory into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator;
under the condition that a transparent large page allocation request sent by a target program is received, allocating the transparent large page for the target program based on the general memory allocator in the general memory area;
and if the general-purpose memory allocator cannot allocate the transparent big page, allocating continuous memory for the target program based on the specific memory allocator to allocate the transparent big page.
2. The method of claim 1, wherein the allocating the transparent large page for the target program based on the general-purpose memory allocator in the general memory area comprises:
determining the size of a target memory required by the transparent large page;
determining a continuous memory matched with the target memory in the common memory area;
and distributing the transparent big page for the target program based on the continuous memory.
3. The method of claim 2, wherein the allocating contiguous memory for the target program based on the particular memory allocator to allocate the transparent big page comprises:
determining an alternative memory area in the specific memory area through the specific memory allocator under the condition that no continuous memory matched with the target memory size exists in the common memory area; the alternative memory area comprises allocated memory pages;
clearing the allocated memory pages in the alternative memory area to obtain the continuous memory;
and distributing the transparent big page for the target program based on the continuous memory.
4. The method of claim 3, wherein the flushing the allocated memory pages in the candidate memory region to obtain the contiguous memory comprises:
copying the allocated memory pages in the alternative memory area to other memory areas;
and mapping the virtual address corresponding to the allocated memory page to the other storage area so as to empty the alternative memory area.
5. The method of any of claims 1-4, wherein the particular memory allocator is a continuous memory allocator.
6. A transparent large page dispensing device, comprising:
the dividing module is used for dividing the system memory into a specific memory area and a common memory area; the specific memory area is managed by a specific memory allocator; the specific memory allocator is used for allocating movable memory pages; the common memory area is managed by a common memory allocator;
the first allocation module is used for allocating the transparent big page for the target program based on the general memory allocator in the common memory area under the condition that a transparent big page allocation request sent by the target program is received;
and the second allocation module is used for allocating continuous memory for the target program based on the specific memory allocator to allocate the transparent big page under the condition that the general memory allocator cannot allocate the transparent big page.
7. The apparatus of claim 6, wherein the first allocation module is specifically configured to:
determining the size of a target memory required by the transparent large page;
determining a continuous memory matched with the target memory in the common memory area;
and distributing the transparent big page for the target program based on the continuous memory.
8. The apparatus of claim 7, wherein the second allocation module is specifically configured to:
determining an alternative memory area in the specific memory area through the specific memory allocator under the condition that no continuous memory matched with the target memory size exists in the common memory area; the alternative memory area comprises allocated memory pages;
clearing the allocated memory pages in the alternative memory area to obtain the continuous memory;
and distributing the transparent big page for the target program based on the continuous memory.
9. An electronic device, comprising:
one or more processors; and one or more machine readable media having instructions stored thereon, which when executed by the one or more processors, cause the electronic device to perform the transparent big page allocation method of any of claims 1-5.
10. One or more machine readable media having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the transparent big page allocation method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111525032.XA CN116263703A (en) | 2021-12-14 | 2021-12-14 | Transparent large page distribution method and device, electronic equipment and readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111525032.XA CN116263703A (en) | 2021-12-14 | 2021-12-14 | Transparent large page distribution method and device, electronic equipment and readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116263703A true CN116263703A (en) | 2023-06-16 |
Family
ID=86722006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111525032.XA Pending CN116263703A (en) | 2021-12-14 | 2021-12-14 | Transparent large page distribution method and device, electronic equipment and readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116263703A (en) |
-
2021
- 2021-12-14 CN CN202111525032.XA patent/CN116263703A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2266040B1 (en) | Methods and systems for dynamic cache partitioning for distributed applications operating on multiprocessor architectures | |
| US7206915B2 (en) | Virtual space manager for computer having a physical address extension feature | |
| US8200932B2 (en) | Managing memory systems containing components with asymmetric characteristics | |
| US8176282B2 (en) | Multi-domain management of a cache in a processor system | |
| US9547535B1 (en) | Method and system for providing shared memory access to graphics processing unit processes | |
| US9183157B2 (en) | Method for creating virtual machine, a virtual machine monitor, and a virtual machine system | |
| EP2581828B1 (en) | Method for creating virtual machine, virtual machine monitor and virtual machine system | |
| US6701421B1 (en) | Application-level memory affinity control | |
| US20140244916A1 (en) | Virtual memory management apparatus | |
| US20060236059A1 (en) | System and method of allocating contiguous memory in a data processing system | |
| US6366994B1 (en) | Cache aware memory allocation | |
| US8395631B1 (en) | Method and system for sharing memory between multiple graphics processing units in a computer system | |
| EP1805629A1 (en) | System and method for virtualization of processor resources | |
| JP2005293574A (en) | Data processing system and computer program which support system memory address with hole | |
| CN113760560A (en) | Inter-process communication method and inter-process communication device | |
| KR101893966B1 (en) | Memory management method and device, and memory controller | |
| US9552295B2 (en) | Performance and energy efficiency while using large pages | |
| US6457107B1 (en) | Method and apparatus for reducing false sharing in a distributed computing environment | |
| CN115543532A (en) | Processing method and device for missing page exception, electronic equipment and storage medium | |
| CN112596913B (en) | Method and device for improving performance of transparent large page of memory, user equipment and storage medium | |
| CN110674051A (en) | Data storage method and device | |
| CN113010452A (en) | Efficient virtual memory architecture supporting QoS | |
| CN116263703A (en) | Transparent large page distribution method and device, electronic equipment and readable medium | |
| JP3808058B2 (en) | Apparatus for allowing a plurality of hosts to share a set of memory sectors storing compressed data | |
| CN116225693A (en) | Metadata management method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |