WO2022188778A1

WO2022188778A1 - Method and apparatus for processing page thrashing in memory recovery, and electronic device

Info

Publication number: WO2022188778A1
Application number: PCT/CN2022/079747
Authority: WO
Inventors: 黄朝阳
Original assignee: 北京紫光展锐通信技术有限公司
Priority date: 2021-03-08
Filing date: 2022-03-08
Publication date: 2022-09-15
Also published as: CN113051186B; CN113051186A

Abstract

Provided in the present application are a method and apparatus for processing page thrashing in memory recovery, and an electronic device. The method comprises: performing data collection on a plurality of application scenarios, and acquiring a thrashing page data set corresponding to each application scenario; according to the thrashing page data set corresponding to each application scenario, generating a set of pages that thrash in each application scenario; and storing, in a shared memory file system, a file corresponding to the set of pages that thrash in each application scenario, wherein the shared memory file system is located in a random access memory. By means of the present application, the effect of page thrashing on the system performance can be reduced.

Description

Method, device and electronic device for processing page thrashing in memory reclamation

This application claims the priority of the Chinese patent application filed on March 8, 2021 with the application number CN202110252780.9 and entitled "Method, Apparatus and Electronic Equipment for Handling Page Bumps in Memory Recovery", the entire contents of which are Incorporated herein by reference.

technical field

The present application relates to the field of computer technology, and in particular, to a method, device, and electronic device for processing page thrashing in memory reclamation.

Background technique

At present, the Linux kernel adopts the principle of "useable and usable, recycling when busy" for memory. The "excessive" reclamation of memory will cause some frequently used file pages to be scanned twice in a row, so that the activity is quickly reduced to INACTIVE (inactive), and finally reclaimed to the Buddy system (partner system). Subsequent applications visit the corresponding location of the file, causing the action of allocating physical pages or reading the contents of the file to occur again. This process is called page thrashing. One or two page thrashing is no problem, but if a large number of file pages are reclaimed due to memory requirements and then read in, this situation will cause system performance to drop, which is manifested as system freezes and inconsistencies. smooth.

Application content

The method, device and electronic device for processing page thrashing in memory reclamation provided by the present application can reduce the impact of page thrashing on system performance.

In a first aspect, the present application provides a method for processing page thrashing in memory reclamation, the method comprising:

Collect data for multiple application scenarios, and obtain the bump page data set corresponding to each application scenario;

generating, according to the thrashing page data set corresponding to each application scenario, a page set in which thrashing occurs in each application scenario;

The files corresponding to the page sets in which thrashing occurs in each application scenario are stored in a shared memory file system, wherein the shared memory file system is located in a random access memory.

Optionally, the method further includes:

After the system is started, confirm whether a file is placed in the shared memory file system through the cat /proc/meminfo command and by searching the file system.

Optionally, the performing data collection on multiple application scenarios, and acquiring the bump page data set corresponding to each application scenario includes:

For each application scenario, extract the pagecache add/delete records in the ftrace information, and find the add/delete records that conform to the equality principle from the add/delete records as an element in the thrashing page data set;

The equality principle is: add records and delete records of the same type and the same page in the add/delete records.

Optionally, the same type includes: pid, time, dev, ino or ofs.

In a second aspect, the present application provides a device for processing page thrashing in memory reclamation, the device comprising:

The acquisition unit is used to collect data for multiple application scenarios, and obtain the bump page data set corresponding to each application scenario;

a generating unit, configured to generate a page set in which thrashing occurs in each application scenario according to the thrashing page data set corresponding to each application scenario;

The processing unit is configured to store the file corresponding to the page set in which thrashing occurs in each application scenario into a shared memory file system, wherein the shared memory file system is located in a random access memory.

Optionally, the device further includes:

The confirmation unit is used for confirming whether a file is placed in the shared memory file system through the cat/proc/meminfo command and the method of searching the file system after the system is started.

Optionally, the acquisition unit is configured to extract the pagecache add/delete records in the ftrace information for each application scenario, and find the add/delete records that conform to the equality principle from the add/delete records as the bump page data. an element in the collection;

In a third aspect, the present application provides an electronic device, the electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned method for processing page thrashing in memory reclamation.

In a fourth aspect, the present application provides a chip, the chip comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the above-mentioned method for processing page thrashing in memory reclamation is implemented.

The method, device, and electronic device for processing page thrashing in memory reclamation provided by the embodiments of the present application quantitatively analyze the page thrashing caused by the memory reclamation process of the Linux system, and can accurately obtain the file in which the page thrashing occurs, and then The file with page thrashing is placed in the shared memory file system, and the shared memory can quickly read from the block device into the memory to reduce the impact of page thrashing on performance.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

FIG. 1 is a flowchart of a method for processing page thrashing in memory reclamation according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a device for processing page thrashing in memory reclamation according to an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

An embodiment of the present application provides a method for processing page thrashing in memory reclamation. The method is applied to an electronic device. As shown in FIG. 1 , the method includes:

S11. Data collection is performed on multiple application scenarios, and a bump page data set corresponding to each application scenario is acquired.

S12. Generate a page set in which thrashing occurs in each application scenario according to the thrashing page data set corresponding to each application scenario.

S13. Store the file corresponding to the page set in which thrashing occurs in each application scenario into a shared memory file system, where the shared memory file system is located in a random access memory.

The method for dealing with page thrashing in memory reclamation provided by the embodiments of the present application quantitatively analyzes the page thrashing caused by the memory reclamation process of the Linux system, and can accurately obtain the file with page thrashing, and then retrieve the file with page thrashing. Put it into the shared memory file system, and use the characteristics of shared memory to quickly read into memory from block devices, reducing the impact of page thrashing on performance.

The processing method for page thrashing in memory reclamation of the present application will be described in detail below with reference to specific embodiments.

The processing method for page thrashing in memory reclamation provided by this embodiment includes the following steps:

S21 , data collection is performed on multiple application scenarios, and a thrashing page data set page_thrashing[] corresponding to each application scenario is obtained.

Specifically, for each application scenario, extract the pagecache (page cache) add/delete records in the Ftrace information, and find the add/delete records that conform to the equality principle from the add/delete records as one of the thrashing page data sets element.

Among them, Ftrace is a powerful debug tool provided by the Linux kernel. Ftrace is an embedded debugging method that uses some functions in the GNU tool chain. Its functional framework is mature, supports arbitrary extensions, and has little interference to the load of the system itself. Ftrace has preset trace_event (point of outputting trace information) at the location where the file is read into memory (inode's page tree) and reclaimed by memory. These trace_events include the block device number, inode (inode) and file. The offset of the page, which constitutes the data object for analyzing memory reclamation and page thrashing.

Wherein, the equality principle is: addition records and deletion records of the same type and the same page in the addition/deletion records.

The implementation method can specifically include:

Running multiple application scenarios, each time an application scenario is run, the system extracts the pagecache addition records and pagecache deletion records in the ftrace information, so that multiple pagecache addition records and multiple pagecache deletion records are obtained. The pagecache delete record searches for the same page of the same type, and the same page is included in multiple pagecache add records and multiple pagecache delete records.

The same types as above can include: pid, time, dev, ino, or ofs, but are not limited thereto.

S22, find the intersection of each element in the thrashing page data set page_thrashing[], and obtain the page set list_thrashing_common in which thrashing occurs in the multiple application scenarios; find a collection of each element in the thrashing page data set page_thrashing[], Get all thrashing page sets list_thrashing_all.

Specifically, the intersection of each element in the thrashing page data set is obtained, and the pages that have thrashing in all application scenarios are collected into list_thrashing_common. For elements in list_thrashing_common, that is, pages with thrashing in all application scenarios;

Collect each element in the thrashing page data set, and collect the pages that have thrashed one or more times in all application scenarios into list_thrashing_all. For elements in list_thrashing_all, that is, at least one page where thrashing occurs in an application scenario.

S23. Put the files corresponding to the page set list_thrashing_common in which thrashing occurs in the multiple application scenarios into the shmem (shared memory) file system.

Relying on the characteristics of the file system to compress the swap (swap) area based on DDR SDRAM (double rate synchronous dynamic random access memory), the files in it will be removed from the swap area at a faster speed even if thrashing occurs. Back in memory to achieve the effect of reducing the impact of thrashing.

The shmem file is a memory-based file format provided by the Linux operating system. Compared with other file formats stored in non-volatile memory, shmem files can provide faster file reading speeds and smaller page thrashing penalties. . Compared with the previous MLock file page method, putting the files corresponding to the page set list_thrashing_common that has thrashed in the multiple application scenarios into the shmem file system can release more available memory for the system to use, and thrash the pages. Penalties are reduced to a minimum.

Further, after the system is started, you can use the cat/proc/meminfo command and the method of searching the file system to confirm whether there are files in the shmem file system to ensure whether there are files under the operating system. Bumpy handling.

Through verification on multiple platforms, the cold start of the application, including facebook, messenger, templerun, camera, etc., will be optimized by about 10% of the time by using the method provided in this embodiment.

The method for dealing with page thrashing in memory reclamation provided by the embodiments of the present application quantitatively analyzes the page thrashing caused by the memory reclamation process of the Linux system, and can accurately obtain the file with page thrashing, and then retrieve the file with page thrashing. Put it into the shared memory file system, and take advantage of the feature that shared memory can quickly read memory from block devices, reduce the impact of page thrashing on performance, and avoid the use of non-thrashing scenarios brought by the lock-thrashing page method to a certain extent. Memory reduction.

An embodiment of the present application further provides a device for processing page thrashing in memory reclamation. The device is located in an electronic device. As shown in FIG. 2 , the device includes:

The acquiring unit 11 is configured to collect data for multiple application scenarios, and acquire a bump page data set corresponding to each application scenario;

A generating unit 12, configured to generate a page set that thrashes in each application scenario according to the thrashing page data set corresponding to each application scenario;

The processing unit 13 is configured to store the files corresponding to the page sets in which thrashing occurs in each application scenario into a shared memory file system, wherein the shared memory file system is located in a random access memory.

The device for processing page thrashing during memory reclamation provided by the embodiment of the present application quantitatively analyzes the page thrashing caused by the memory reclamation process of the Linux system, and can accurately obtain the file with page thrashing, and then treat the file with page thrashing. Put it into the shared memory file system, and use the characteristics of shared memory to quickly read into memory from block devices, reducing the impact of page thrashing on performance.

Optionally, the device further includes:

Optionally, the obtaining unit 11 is configured to extract the pagecache add/delete records in the ftrace information for each application scenario, and find the add/delete records that conform to the equality principle from the add/delete records as the thrashing page. an element in the dataset;

The apparatus of this embodiment can be used to implement the technical solutions of the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

The embodiment of the present application also provides an electronic device, the electronic device includes:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The embodiment of the present application also provides a chip, the chip includes:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

Embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the above-mentioned method for processing page thrashing in memory reclamation is implemented.

Those of ordinary skill in the art can understand that the realization of all or part of the processes in the above method embodiments can be accomplished by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. During execution, the processes of the embodiments of the above-mentioned methods may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, All should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

A method for processing page thrashing in memory reclamation, wherein the method includes:

Collect data for multiple application scenarios, and obtain the bump page data set corresponding to each application scenario;

generating, according to the thrashing page data set corresponding to each application scenario, a page set in which thrashing occurs in each application scenario;

The files corresponding to the page sets in which thrashing occurs in each application scenario are stored in a shared memory file system, wherein the shared memory file system is located in a random access memory.
The method of claim 1, wherein the method further comprises:

After the system is started, confirm whether a file is placed in the shared memory file system through the cat /proc/meminfo command and by searching the file system.
The method according to claim 1 or 2, wherein the performing data collection on multiple application scenarios, and acquiring the bump page data set corresponding to each application scenario comprises:

For each application scenario, extract the pagecache add/delete records in the ftrace information, and find the add/delete records that conform to the equality principle from the add/delete records as an element in the thrashing page data set;

The equality principle is: add records and delete records of the same type and the same page in the add/delete records.
The method of claim 3, wherein the same type comprises: pid, time, dev, ino, or ofs.
A device for processing page thrashing in memory reclamation, wherein the device includes:

The acquisition unit is used to collect data for multiple application scenarios, and obtain the bump page data set corresponding to each application scenario;

a generating unit, configured to generate a page set in which thrashing occurs in each application scenario according to the thrashing page data set corresponding to each application scenario;

The processing unit is configured to store the file corresponding to the page set in which thrashing occurs in each application scenario into a shared memory file system, wherein the shared memory file system is located in a random access memory.
The apparatus of claim 5, wherein the apparatus further comprises:

The confirmation unit is used for confirming whether a file is placed in the shared memory file system through the cat/proc/meminfo command and the method of searching the file system after the system is started.
The device according to claim 5 or 6, wherein the acquisition unit is configured to extract pagecache add/delete records in the ftrace information for each application scenario, and find out the add/delete records that meet the equality principle from the add/delete records. /Delete the record as an element in the bump page data set;

The equality principle is: add records and delete records of the same type and the same page in the add/delete records.
An electronic device, wherein the electronic device comprises:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the execution of any one of claims 1 to 4 Methods.
A chip, wherein the chip includes:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the execution of any one of claims 1 to 4 Methods.
A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method of any one of claims 1 to 4.