WO2024001644A1

WO2024001644A1 - Method and apparatus for controlling transparent huge page, and storage medium

Info

Publication number: WO2024001644A1
Application number: PCT/CN2023/097197
Authority: WO
Inventors: 汪勇; 杨洋; 蒋学鑫; 张文涯; 徐鑫
Original assignee: 中兴通讯股份有限公司
Priority date: 2022-06-27
Filing date: 2023-05-30
Publication date: 2024-01-04
Also published as: CN117331643A

Abstract

Provided in the present application are a method and apparatus for controlling a transparent huge page, and a storage medium. The method comprises: according to a transparent huge page mode which is configured on the basis of a container, configuring a transparent huge page (S100); during a process of configuring the transparent huge page, acquiring a performance change amount when the container uses the transparent huge page (S200); and according to the performance change amount, adjusting the transparent huge page mode (S300).

Description

Method and device and storage medium for controlling transparent huge page

Cross-references to related applications

This application is filed based on a Chinese patent application with application number 202210736636.7 and a filing date of June 27, 2022, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference into this application.

Technical field

Embodiments of the present application relate to but are not limited to the field of memory management, and in particular, to a method of controlling transparent huge pages, a device thereof, and a storage medium.

Background technique

The system generally uses huge pages in two ways. One is that user-mode programs use the hugetlbfs library to reserve memory space for huge pages when the kernel starts, and at the same time explicitly complete mapping for huge pages in the code; the other is transparent giant pages. Transparent Huge Page (THP), which is transparent to the user, and the kernel automatically allocates huge pages to the application.

Currently, the host transparent huge page supports three modes: always mode means that transparent huge pages are used throughout the system, which may cause a lot of memory waste; madvise mode means that it can only be used in areas that support transparent huge pages by calling a dedicated (madvise interface) setting. Transparent huge pages means that users need to modify the application code to use transparent huge pages, which has a high threshold and is not flexible enough; never mode means that transparent huge pages are not used, but system performance will be reduced.

Using huge pages can reduce the number of page fault exceptions (Page Fault) and improve access performance, but it will cause more physical pages to be allocated than the actual memory required, resulting in memory waste. Transparent huge pages trade space for performance. More and more applications are being containerized. On container-based management equipment, such as some 5G base stations, if you want to optimize the use of transparent huge pages in a certain container, you need to manually modify the application source code in the container. Low-end devices with tight memory usually do not use transparent huge pages technology, but if the containers in low-end devices do not consume much memory and require high performance, such containers can use transparent huge pages. Currently, several modes of transparent huge pages are configured globally for the host, and each container has different performance requirements. Using the global transparent huge page mode cannot meet the needs of all containers. Currently, there is a lack of transparent huge page mechanism for refined management of containers.

Contents of the invention

Embodiments of the present application provide a method for controlling a transparent giant page, a device thereof, and a storage medium.

In a first aspect, embodiments of the present application provide a method of controlling transparent huge pages. The method of controlling transparent huge pages includes: setting transparent huge pages according to a transparent huge page mode based on container settings; and setting the transparent huge pages. In the process, the performance change amount when the container uses the transparent huge page is obtained; and the transparent huge page mode is adjusted according to the performance change amount.

In a second aspect, embodiments of the present application also provide a device for controlling transparent huge pages, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program. The program implements the above-mentioned method of controlling transparent huge pages.

In a third aspect, embodiments of the present application also provide a computer-readable storage medium that stores computer-executable instructions, and the computer-executable instructions are used to execute the above-mentioned method of controlling a transparent giant page.

Description of drawings

Figure 1 is a flow chart of a method for controlling transparent huge pages provided by an embodiment of the present application;

Figure 2 is a flow chart of a method for controlling transparent huge pages when triggering page faults to allocate physical memory provided by an embodiment of the present application;

Figure 3 is a flow chart of a method for controlling transparent huge pages in a virtual memory area that does not belong to a container provided by an embodiment of the present application;

Figure 4 is a flow chart of a method for controlling transparent huge pages after allocating physical memory to transparent huge pages according to an embodiment of the present application;

Figure 5 is a flow chart of a method for merging ordinary pages into transparent giant pages according to an embodiment of the present application;

Figure 6 is a flow chart of a calculation method for increasing physical memory when a container uses transparent huge pages according to an embodiment of the present application;

Figure 7 is a flowchart of a calculation method for reducing the number of page faults when a container uses transparent huge pages according to an embodiment of the present application;

Figure 8 is a flow chart of an example of controlling a transparent huge page provided by an embodiment of the present application;

Figure 9 is a flowchart of an example of calculating a performance change amount provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

It should be noted that although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that in the flowchart. The terms "first", "second", etc. in the description, claims, and above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific sequence or sequence.

Embodiments of the present application provide a method, device, and storage medium for controlling transparent huge pages. The transparent huge pages are set according to the transparent huge page mode set based on the container. In the process of setting the transparent huge pages, the time when the container uses the transparent huge page is obtained. The amount of performance change; adjust the transparent huge page mode according to the amount of performance change. Setting transparent huge pages according to the transparent huge page mode can reduce the number of page faults in the container and improve the performance of the container. The transparent huge page mode also provides the container with a way to control the setting of transparent huge pages; in the process of setting transparent huge pages, Obtain the performance change when the container uses transparent huge pages. Since the use of transparent huge pages requires more memory, the performance and memory required for different containers using transparent huge pages or the same container using transparent huge pages at different times. is different. By obtaining the performance change of the container after using transparent huge pages, and based on the performance change, it is determined whether the container will still run in the current transparent huge page mode, so that the container can choose the transparent huge page mode that meets its own needs. Achieve the purpose of refined management.

As shown in Figure 1, Figure 1 is a flow chart of a method for controlling transparent huge pages provided by an embodiment of the present application.

As shown in Figure 1, the method for controlling transparent giant pages includes step S100, step S200, and step S300.

Step S100: Set the transparent huge page according to the transparent huge page mode set based on the container;

Step S200: In the process of setting up transparent huge pages, obtain the performance change amount when the container uses transparent huge pages;

Step S300: Adjust the transparent huge page mode according to the performance change amount.

In this embodiment, setting transparent huge pages according to the transparent huge page mode can reduce the number of page failures in the container and improve the performance of the container. The transparent huge page mode also provides the container with a way to control the setting of transparent huge pages; because transparent huge pages needs for use It takes up more memory, so the performance and memory required for different containers that use transparent huge pages or the same container that uses transparent huge pages at different times are different. Therefore, in the process of setting up transparent huge pages, use transparent The performance change amount of the container after the huge page is used to determine whether the container will still run in the current transparent huge page mode in the future, so that the container can choose the transparent huge page mode that meets its own needs to achieve the purpose of refined management.

It is understandable that transparent huge pages can improve performance by reducing page faults (one page fault allocates a large block of memory), can also reduce the cost of converting virtual addresses to physical addresses (reduce the number of conversions), and can even avoid address translation. If the processor must translate a virtual address, it must go through up to four levels of page tables, and the processor maintains a translation lookaside buffer (TLB) that caches the translation results. The capacity of the TLB is very small, usually 128 instruction translations and 32 data translations, which is very easy to fill up, so a large number of address translations must be performed. Using transparent huge pages through the kernel address can reduce the pressure on tlb. Therefore, using transparent huge pages can increase performance, but it will allocate large blocks of memory and occupy more memory space. There is a corresponding transparent huge page mode for transparent huge pages. The transparent huge page mode can control the use of transparent huge pages. However, currently the transparent huge page mode can only be adjusted by modifying the program code, and it is configured globally, so the amount of change depends on the performance. To adjust the transparent huge page mode solves the problem of the transparent huge page mode, allowing the container to manage the transparent huge page mode in a refined manner.

In some embodiments, the transparent huge page mode includes one of the following: on mode, set to use transparent huge pages and allocate memory for transparent huge pages; off mode, set to use normal pages and allocate memory for normal pages; or select mode , set to set transparent huge pages for the preset target process and allocate memory for transparent huge pages.

The transparent huge page mode provides three modes for managing transparent huge pages. It can control whether to use transparent huge pages or specify the target process to use transparent huge pages. It also provides a way for the container to adjust the transparent huge page mode to achieve a balance between performance and memory. the basic mechanism.

In some embodiments, as shown in FIG. 2 , after step S300 is executed, the method of controlling a transparent giant page may also include, but is not limited to, step S400 and step S500.

Step S400: When a page fault is triggered and physical memory is allocated, determine whether the virtual memory area belongs to the container.

Step S500: If the virtual memory area belongs to the container, allocate physical memory for the transparent huge page according to the transparent huge page mode.

Determining whether the virtual memory area belongs to the container and then executing the current transparent huge page mode can avoid the current virtual memory area not belonging to the container and running the wrong transparent huge page mode, which will affect the performance.

In some embodiments, as shown in FIG. 3 , after performing step S500, the method of controlling a transparent giant page may also include but is not limited to step S600.

Step S600: If the virtual memory area does not belong to the container, allocate physical memory for the transparent huge page according to the host transparent huge page mode. When the virtual memory area does not belong to the container, physical memory is allocated for transparent huge pages according to the host's transparent huge page mode. When the virtual memory area does not belong to the container and there are still transparent huge pages that need to be allocated physical memory, physical memory is allocated according to the host's transparent huge page mode. Transparent huge pages allocate memory and solve the problem of how to use transparent huge pages when the current virtual memory area does not belong to the container.

In some embodiments, as shown in FIG. 4 , after step S600 is performed, the method of controlling a transparent giant page may also include, but is not limited to, step S700 and step S800.

Step S700: Obtain the remaining memory amount of the container.

Step S800: If the remaining amount of memory is greater than the remaining amount threshold, merge ordinary pages to obtain transparent giant pages.

When allocating physical memory for transparent huge pages according to the transparent huge page mode, the memory space of the container is constantly shrinking. If all allocation tasks are completed, it is detected that the container memory is still remaining, and the ordinary pages in the container are merged to obtain a transparent giant page. Pages can improve the utilization of the container's memory space, and convert ordinary pages into transparent huge pages if the memory space allows, which can improve the performance of the container and reduce the number of page faults in the container.

In some embodiments, as shown in Figure 5, step S800 is described. Step S800 includes but is not limited to step S810 and step S820.

Step S810: Determine the transparent huge page mode of the container.

Step S820: If the transparent huge page mode is the on mode or the selection mode, and the process using the transparent huge page in the selection mode is the default target process, merge the ordinary huge pages to obtain the transparent huge page.

To merge ordinary pages to obtain transparent huge pages, you need to verify the prerequisites and verify that the transparent huge page mode of the container is not a closed mode. The container may not need to improve performance to avoid affecting the memory space of the container in its normal state.

In some embodiments, the performance change includes at least one of the following: increased physical memory when the container uses transparent huge pages; or reduced number of page faults when the container uses transparent huge pages.

In some embodiments, the performance change amount is a reference amount that measures the performance change of the container, and the container makes corresponding adjustments to the transparent huge page mode based on the reference amount.

In some embodiments, as shown in Figure 6, when the performance change amount includes calculation of the increased physical memory when the container uses transparent huge pages, the performance change amount can be obtained according to step S311, step S312 and step S313.

Step S311: Initialize the increased physical memory when the container uses transparent huge pages to 0.

Step S312: Determine whether the container uses transparent huge pages.

Step S313: If the container uses transparent huge pages, obtain the physical memory increase when using transparent huge pages, and accumulate the physical memory increase to the increased physical memory when the container uses transparent huge pages.

To calculate the increased physical memory when the container uses transparent huge pages, first initialize the increased physical memory when the container uses transparent huge pages to 0, and then determine whether the container uses transparent huge pages. This can reduce the amount of calculation and eliminates the need to obtain the data of each container. Physical memory to reduce the computational burden. Use transparent huge pages and then obtain the increase in physical memory using transparent huge pages. Accumulate the increase in physical memory to the increased physical memory when the container uses transparent huge pages. Calculate the cumulative increase in physical memory. It can visually reflect the total increased physical memory of the container in the current transparent huge page mode, and facilitate real-time monitoring of the memory usage of the container in the current transparent huge page mode.

In some embodiments, after performing step S313, the method of controlling a transparent huge page may also include but is not limited to step S314.

Step S314: If the transparent huge page mode is changed, clear the physical memory increased when the container uses transparent huge pages.

After the transparent huge page mode is changed, the increased physical memory when the container uses transparent huge pages will be recalculated, and the increased physical memory when using transparent huge pages will be remeasured.

In some embodiments, as shown in Figure 7, when the performance change amount includes the reduced number of page misses when the container uses transparent huge pages, the performance change amount can be obtained according to steps S321 and S322.

Step S321: Initialize the number of page faults reduced when the container uses transparent huge pages to 0.

Step S322: If no page fault occurs in the container and the memory accessed for the first time belongs to a transparent huge page, increase the number of page faults reduced when the container uses transparent huge pages by 1 to obtain the number of page faults reduced when the container uses transparent huge pages.

To calculate the number of page faults reduced when the container uses transparent huge pages, first initialize the number of page faults reduced when the container uses transparent huge pages to 0. If no page faults occur in the container and the memory accessed for the first time is a transparent huge page, set the container to use transparent huge pages. The number of page faults reduced when using transparent huge pages is increased by 1, which facilitates statistics on the total number of page faults reduced by the container using the current transparent huge page mode. It more intuitively reflects the effect of using transparent huge pages, and also facilitates real-time monitoring of the container in the current transparent huge page mode. The number of page faults in this case.

In some embodiments, after performing step S322, the method of controlling a transparent huge page may also include but is not limited to step S323.

Step S323: If the transparent huge page mode is changed, clear the reduced number of page faults when the container uses transparent huge pages.

After the transparent huge page mode is changed, the number of page faults reduced when the container uses transparent huge pages will be recalculated, and the increased physical memory when using transparent huge pages will be re-measured.

As shown in FIG. 8 , in order to more clearly explain the processing flow of the method for controlling transparent huge pages provided by the embodiment of the present application, an example will be used to illustrate below.

The method of controlling transparent huge pages includes the following steps:

Start the container and set the container's transparent huge page mode. While the system is running, the user can reselect the transparent huge page mode of the container based on the performance change of the container using transparent huge pages. Transparent huge page modes include one of the following: On mode, set to use transparent huge pages and allocate memory for transparent huge pages; Off mode, set to use normal pages and allocate memory for normal pages; or Select mode, set to Default The target process sets up transparent huge pages and allocates memory for transparent huge pages.

When a page fault is triggered to allocate physical memory, it is judged whether the VMA (virtual memory area) belongs to the container; the VMA does not belong to the container, and physical memory is allocated for transparent huge pages according to the host transparent huge page mode.

VMA belongs to the container and allocates physical memory for transparent huge pages according to the transparent huge page mode of the container.

Get the remaining memory of the container. If the remaining memory is greater than the remaining threshold, ordinary pages will be merged to obtain transparent giant pages.

As shown in Figure 9, in order to explain the calculation of the performance change more clearly, the following is an example. The calculation of the performance change includes the following steps:

Start the container and set the container's transparent huge page mode to open mode or select mode;

Initialize the container's AMT (increased physical memory when the container uses transparent huge pages) and RFT (reduced page faults when the container uses transparent huge pages) to 0. Modifying the container's transparent huge page mode during operation will clear AMT and RFT records;

When a container triggers a page fault to allocate physical memory, it determines whether to allocate transparent huge pages based on the transparent huge page mode set by the container;

If transparent huge pages are allocated, obtain the physical memory increase when using transparent huge pages, and accumulate the physical memory increase to the increased physical memory when the container uses transparent huge pages;

If no page faults occur in the container and the memory accessed for the first time is a transparent huge page, increase the number of page faults reduced when the container uses transparent huge pages by 1 to obtain the number of page faults reduced when the container uses transparent huge pages.

In addition, an embodiment of the present application also provides a device for controlling transparent huge pages. The device for controlling transparent huge pages includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The above-mentioned method of controlling transparent huge pages is implemented when the computer program is executed by the computer.

In addition, an embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions. The computer-executable instructions are used to execute the above-mentioned method of controlling a transparent giant page, for example, Being executed by a processor in the embodiment of the apparatus for controlling a transparent huge page can cause the above processor to execute the method of controlling a transparent huge page in the above embodiment, for example, executing the method in Figure 1 and Figure 2 described above. method in and the method in Figure 3.

The non-transitory software programs and instructions required to implement the method of controlling transparent huge pages in the above embodiment are stored in the memory. When executed by the processor, the method of controlling transparent huge pages in the above embodiment is executed. For example, execute the above The method in Figure 1, the method in Figure 2 and the method in Figure 3 are described.

Embodiments of the present application include: setting transparent huge pages according to the transparent huge page mode based on container settings; in the process of setting transparent huge pages, obtaining the performance change amount when the container uses transparent huge pages; adjusting the transparent huge page mode according to the performance change amount . Setting transparent huge pages according to the transparent huge page mode can reduce the number of page faults in the container and improve the performance of the container. The transparent huge page mode also provides the container with a way to control the setting of transparent huge pages; in the process of setting transparent huge pages, Get container to use transparent The amount of performance change when using huge pages. Since the use of transparent huge pages requires more memory, the performance and memory required for different containers using transparent huge pages or the same container using transparent huge pages at different times are different. By obtaining the performance change amount of the container after using transparent huge pages, and based on the performance change amount, it is determined whether the container will still run in the current transparent huge page mode in the future, so that the container can choose the transparent huge page mode that meets its own needs to achieve refinement. management purposes.

Those of ordinary skill in the art can understand that all or some steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit . Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. removable, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage cell technology, CD-ROM, Digital Versatile Disk (DVD) or other optical disk storage, magnetic cassettes, tapes, disk storage or other magnetic storage devices, or Any other medium that can be used to store the desired information and that can be accessed by a computer. Additionally, it is known to those of ordinary skill in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Claims

A method to control transparent huge pages, including:

Set transparent huge pages according to the transparent huge page mode set based on the container;

In the process of setting the transparent huge page, obtain the performance change amount when the container uses the transparent huge page;

The transparent huge page mode is adjusted according to the performance change amount.
The method of controlling transparent huge pages according to claim 1, wherein the transparent huge page mode includes one of the following:

Enable mode, set to use the transparent huge page and allocate memory for the transparent huge page;

Off mode, set to use normal pages and allocate memory for said normal pages; or

Select a mode to set the transparent huge page for a preset target process and allocate memory for the transparent huge page.
The method for controlling transparent giant pages according to claim 1, further comprising:

When a page fault is triggered to allocate physical memory, determine whether the virtual memory area belongs to the container;

If the virtual memory area belongs to the container, allocate the physical memory to the transparent huge page according to the transparent huge page mode.
The method for controlling transparent giant pages according to claim 3, further comprising:

If the virtual memory area does not belong to the container, the physical memory is allocated to the transparent huge page according to the host transparent huge page mode.
The method of controlling transparent huge pages according to claim 3, wherein after allocating the physical memory to the transparent huge page according to the transparent huge page mode, the method of controlling transparent huge pages further includes:

Get the remaining memory of the container;

If the remaining amount of memory is greater than the remaining amount threshold, ordinary pages are merged to obtain transparent huge pages.
The method of controlling transparent huge pages according to claim 5, wherein said merging ordinary pages to obtain transparent huge pages includes:

Determine the transparent huge page mode of the container;

If the transparent huge page mode is the open mode or the selection mode, and the process using the transparent huge page in the selection mode is a preset target process, the ordinary huge pages are merged to obtain the Transparent giant page.
The method for controlling transparent huge pages according to claim 2, wherein the performance change amount includes at least one of the following:

The physical memory increased when the container uses the transparent huge page; or

The number of page faults reduced when the container uses the transparent huge page.
The method of controlling transparent huge pages according to claim 7, wherein when the performance change amount includes calculation of the increased physical memory when the container uses the transparent huge page, the performance change amount can be obtained according to the following steps:

Initialize the increased physical memory when the container uses the transparent huge page to 0;

Determine whether the container uses the transparent huge page;

If the container uses the transparent huge page, obtain the physical memory increase when the transparent huge page is used, and accumulate the physical memory increase to the physical memory increase when the container uses the transparent huge page.
The method for controlling transparent giant pages according to claim 8, further comprising:

If the transparent huge page mode is changed, clear the physical memory that is increased when the container uses the transparent huge page.
The method for controlling transparent huge pages according to claim 7, wherein when the performance change amount includes the content The number of page faults reduced when the server uses the transparent huge page, and the performance change amount can be obtained according to the following steps:

Initialize the number of page faults reduced when the container uses the transparent huge page to 0;

If no page fault occurs in the container and the memory accessed for the first time belongs to a transparent huge page, increase the number of page faults reduced when the container uses the transparent huge page by 1 to obtain the reduced number of page faults when the container uses the transparent huge page. Number of missing pages.
The method of controlling a transparent huge page according to claim 10, wherein if the transparent huge page mode is changed, the number of page faults reduced when the container uses the transparent huge page is cleared.
A device for controlling transparent giant pages, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any one of claims 1 to 11 The method of controlling transparent huge pages as described in item 1.
A computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the method of controlling a transparent giant page according to any one of claims 1 to 11.