CN116225976B - Water line self-adjusting method and system under Linux operating system - Google Patents
Water line self-adjusting method and system under Linux operating system Download PDFInfo
- Publication number
- CN116225976B CN116225976B CN202310495310.4A CN202310495310A CN116225976B CN 116225976 B CN116225976 B CN 116225976B CN 202310495310 A CN202310495310 A CN 202310495310A CN 116225976 B CN116225976 B CN 116225976B
- Authority
- CN
- China
- Prior art keywords
- memory
- water line
- pages
- idle
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/10—Providing a specific technical effect
- G06F2212/1016—Performance improvement
- G06F2212/1024—Latency reduction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/10—Providing a specific technical effect
- G06F2212/1032—Reliability improvement, data loss prevention, degraded operation etc
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Feedback Control In General (AREA)
Abstract
The invention discloses a water line self-adjusting method and a system under a Linux operating system, wherein the method comprises the steps of sampling the number of idle pages of a memory, calculating the recovery efficiency of the memory pages after starting the recovery of the memory, and constructing a prediction model to predict the memory exhaustion time according to the number of the idle pages obtained by sampling in a current time window and the sampling time if the recovery efficiency is smaller than a set value; before the memory exhaustion time comes, if the number of idle pages is between the high water line and the low water line, judging that the idle memory is about to be exhausted, and triggering an asynchronous recovery process in advance to recover pages by changing the height of the water line; if the memory is severely fragmented, memory regularity is enabled to eliminate memory fragmentation. The invention aims to avoid system blocking and operation delay, reduce CPU resource waste caused by page recovery and ensure the running stability of an operating system.
Description
Technical Field
The invention relates to a memory management technology in the field of operating systems, in particular to a water line self-adjusting method and system under a Linux operating system.
Background
The Linux kernel is in the form of pages with different sizes (page blocks with different sizes are represented by steps 0-10 in the kernel and are the power of 2, the higher the step is, the larger the page is represented, and the higher the page is, the higher the step is, the higher the page is n A group of physically contiguous pages, where n is the page order) manages physical memory and virtual memory in units called base page pages, each rank maintaining a linked list of pages allocated to users and kernel tasks for use. When processing large amounts of data from slower disk devices, the Linux kernel uses page caching techniques to cache the content, but the memory for page caching must be from the same memory pool used by the rest of the system. The kernel allocates all currently unused memory to the page cache. Since the kernel needs to allocate more memory for other tasks, it can reclaim pages from the page cache, as the contents of the page cache can be restored from disk when needed. When in the middleWhen the core applies for a insufficient number of free pages in the memory, the number of free pages is lower than that of the low water line, and memory reclamation can occur. When pages are reclaimed, any consecutive pages are combined together, forming higher order pages by normalization.
The Linux kernel recovers the condition of insufficient free memory by combining page reclamation and memory regularity with a water line (watermark): when the amount of free memory in the system is lower than a low water line (low water mark), the memory management mechanism releases and reclaims the recyclable pages in the cache by enabling an asynchronous reclaiming process (kswapd) for asynchronous memory reclaiming. If the recovered pages are low-order pages, the memory is fragmented and high-order page blocks cannot be allocated. At this time, the kernel completes memory fragmentation regulation by starting the kcompactd thread, and merges the low-order continuous pages into high-order page blocks to recover the number of the high-order idle pages. If the amount of free memory continues to consume below the minimum water line (min watermark), any allocation request must be reclaimed or normalized before the preamble is completed to meet the needs of the memory application. The latter synchronization method is called a "direct" path, and at this time, the system may get stuck due to waiting for the memory to be recovered, the running speed is quite slow, and the corresponding stall generated in the memory call may also cause the delay of the operation being performed to increase, which causes uncertainty of the system and data, and affects the system performance. If asynchronous recovery and memory regularity can be faster than the speed of memory consumption, the phenomenon of killing processes in direct memory recovery can be avoided, and no blocking and operation delay can be caused. In contrast, if the memory is consumed faster than it is likely to be recovered, when the memory is below the lowest water line, direct memory recovery must occur to kill processes occupying relatively large amounts of memory, or long waiting times due to insufficient application of available memory occur, reflecting the phenomenon of system stuck, and operation delays. At this time, the most ideal mode is to activate the asynchronous recovery process in advance to recover the memory by increasing the water line, thereby meeting the requirement of the memory application and avoiding the occurrence of the blocking and delay phenomena. However, if the system is not truly memory-exhausted, the application of higher-order memory blocks may not be completed due to memory fragmentation, which results in excessive reclamation, resulting in CPU resource consumption (the kswapd process is to utilize CPU resources to reclaim memory resources). In this case, the low-order pages are combined into the high-order page blocks in a memory-ordered manner, so that the application requirement of the system on the high-order page blocks is met, the waterline is not excessively increased, the reserved memory of the system is increased, and the memory is repeatedly recovered by the kernel, so that the CPU resource is occupied and wasted. This situation is further exacerbated by the fact that different workloads have different memory consumption patterns. Some workloads allocate and release memory at a steady rate, such as media streaming programs that read disk blocks, video rendering, release rendered data, and acquire more data for rendering. Other workloads such as scientific computation and modeling may allocate a large number of pages to read data from disk for a period of time, then perform data computation on the accumulated data, and allocate a smaller amount of memory at the same time, and finally may release all memory that previously saved the data, and start the next modeling operation. There are still some workloads, such as transactions, that may peak periodically for memory allocation requests that may be repeated over a long period of time. This makes it difficult to define a static water line suitable for a large number of workloads. A workload running for a long time may also have different memory consumption patterns during its lifecycle, which makes the setting of a static water line at best a compromise, as it cannot meet the needs of various load scenarios.
Disclosure of Invention
The invention aims to solve the technical problems: aiming at the problems in the prior art, the invention provides a water line self-adjusting method and a system under a Linux operating system, which aim to realize the self-adjustment of the water line according to the current actual memory consumption and the condition of triggering asynchronous recovery, can start memory recovery in advance to avoid system blocking and operation delay, can reduce pause even under the condition of high memory consumption rate, can avoid CPU resource waste caused by page recovery before absolute necessity, can meet the requirements of various load scenes, and can ensure the running stability of the operating system.
In order to solve the technical problems, the invention adopts the following technical scheme:
a water line self-adjusting method under a Linux operating system comprises the following steps:
s101, sampling the number of idle pages of a memory, calculating recovery efficiency of the memory pages after starting memory recovery, if the recovery efficiency is smaller than a set value, jumping to the step S102, otherwise jumping to the step S101, and continuing to sample;
s102, constructing a prediction model according to the number of idle pages obtained by sampling in a current time window and sampling time, and predicting the memory exhaustion time according to a model function obtained by linear fitting of the prediction model;
And S103, before the time of memory exhaustion comes, judging whether the number of idle pages is between a high water line and a low water line, if so, judging that the idle memory is about to be exhausted, and triggering an asynchronous recovery process in advance to recover pages by changing the height of the water line.
Optionally, in step S101, the recovery efficiency of the memory pages is calculated by dividing the number of memory pages recovered during memory recovery by the number of memory pages to be scanned and multiplying the number by 100%.
Optionally, constructing the prediction model in step S102 means that the least square method is adopted to perform linear fitting on two variables, namely the number of idle pages and time, and a function expression of a model function obtained by linear fitting is as follows:
，
in the above-mentioned method, the step of,for the number of free pages at time x +.>And->For fitting parameters +.>Time is; and when linear fitting is carried out, the function expression of fitting calculation of the fitting parameters is as follows:
，/>，
in the above-mentioned method, the step of,is->Number of free pages at time, +.>Is the number of free pages sampled.
Optionally, the predicting the memory exhaustion time according to the fitted curve obtained by the fitting in step S102 refers to: and obtaining the intercept of the fitting curve and the time axis, and taking the moment corresponding to the intercept as the predicted memory exhaustion time.
Optionally, step S103 further includes determining whether the number of free pages of the higher level of the memory is increased, if the number of free pages of the higher level of the memory is not increased, determining that the memory is severely fragmented, calculating the number of free pages of each level respectively, and removing memory fragments by enabling memory regularity for pages of each level of the memory.
Optionally, after enabling memory regularity to eliminate memory fragments in step S103, setting a memory regularity flag; the step S103 of changing the height of the water line to trigger the asynchronous reclaiming process in advance to reclaim the page includes:
s201, acquiring a current water line scale factor and a lowest water line;
s202, calculating a high water level line and a low water level line according to the water level line scale factors and the lowest water level line;
s203, acquiring the current number of idle pages of the memory;
s204, judging whether the current free page number of the memory is larger than a high water line, if the current free page number of the memory is smaller than or equal to the high water line, judging that the memory pressure is not relieved, continuously judging whether the current free page number of the memory is smaller than half of the difference value between the high water line and the low water line, if not, jumping to the step S203, if so, adjusting the water line scale factor according to the appointed proportion to reduce the water line, ending and exiting; if the current number of free pages of the memory is greater than the high water line, determining that the memory pressure is relieved, and jumping to step S205;
S205, judging whether the current number of idle pages of the memory reaches the upper part of the high water line, if not, adjusting the water line scale factor according to the appointed proportion to enlarge the water line, ending and exiting; otherwise, jumping to step S206;
s206, judging whether the current number of the high-order idle pages of the memory is smaller than a set value, if so, judging whether the kernel enables memory regularity according to the mark of whether the memory regularity exists, and if so, clearing the mark of the memory regularity; if the memory regularity is not started, the memory regularity is started to eliminate memory fragments so as to obtain more memory pages.
Optionally, after the adjusting the water line scale factor according to the designated proportion, the method further includes limiting the adjusted water line scale factor within a preset threshold range of the water line scale factor to prevent user progress from being killed due to overrun of the water line scale factor.
Optionally, the adjusting the water line scale factor according to the specified proportion to reduce the water line refers to reducing the water line scale factor according to the proportion of 10% to reduce the water line; the adjusting the water line scale factor according to the specified proportion to enlarge the water line refers to enlarging the water line scale factor according to the proportion of 10% to reduce the water line.
In addition, the invention also provides a water line self-adjusting system under the Linux operating system, which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the water line self-adjusting method under the Linux operating system.
In addition, the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is used for being programmed or configured by a microprocessor to execute the water line self-adjusting method under the Linux operating system.
Compared with the prior art, the invention has the following advantages: the method comprises the steps of regularly sampling the number of idle pages of a memory, calculating recovery efficiency of the memory pages after starting memory recovery, if the recovery efficiency is smaller than a set value, constructing a prediction model according to the number of idle pages obtained by sampling in a current time window and sampling time, and predicting the memory exhaustion time according to a model function obtained by linear fitting of the prediction model; and before the time of memory exhaustion comes, judge whether the idle page quantity is located between high water line, low water line, if so, change the height of water line and trigger asynchronous recovery process in advance and carry out page recovery, through the above-mentioned mode, can realize realizing the self-adjustment of water line according to the current actual memory consumption and the circumstances of triggering asynchronous recovery, can proactively start memory recovery and avoid appearing system card and operation delay, even also can reduce the pause under the very high circumstances of memory consumption rate, and avoid the CPU wasting of resources that the page recovery caused before absolutely necessary, can satisfy the demand of various load scenes, ensure operating system's stability.
Drawings
In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed for the description of the present invention will be briefly described below, and it is obvious that the drawings in the following description are only descriptions of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a basic flow of a method according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of the general principle framework of the method according to the embodiment of the invention.
Fig. 3 is a schematic flow chart of a monitoring process in an embodiment of the invention.
FIG. 4 is a diagram illustrating the operation of a kernel memory management waterline of a Linux system according to an embodiment of the present invention.
FIG. 5 is a graph showing curve fitting of the prediction mechanism in an embodiment of the present invention.
FIG. 6 is a flow chart of a prediction process according to an embodiment of the invention.
Fig. 7 is a flow chart of an adjustment process according to an embodiment of the invention.
Detailed Description
In order to better understand the scheme of the present application, the platform adopting the phy ium processor and running the Galaxy kylin operating system will be taken as an example to clearly and completely describe the method and the technical scheme in the system for self-adjusting the water line under the Linux operating system of the present invention, and obviously, the described embodiments are only a part of embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure. Moreover, it should be noted that the method and system for self-adjusting the water line under the Linux operating system of the present invention are applicable to all Linux kernels that use water lines (watermarks) to perform memory management, and do not depend on the specific processor or Linux operating system version.
As shown in fig. 1, the method for self-adjusting a water line under the Linux operating system of the present embodiment includes:
s101, sampling the number of idle pages of a memory, calculating recovery efficiency of the memory pages after starting memory recovery, if the recovery efficiency is smaller than a set value, jumping to the step S102, otherwise jumping to the step S101, and continuing to sample;
s102, constructing a prediction model according to the number of idle pages obtained by sampling in a current time window and sampling time, and predicting the memory exhaustion time according to a model function obtained by linear fitting of the prediction model;
s103, before the time of memory exhaustion comes, judging whether the number of idle pages is between a high water line and a low water line, if so, judging that the idle memory is about to be exhausted, and triggering an asynchronous recovery process (kswapd) in advance to carry out page recovery by changing the height of the water line, thereby achieving the purposes of solving the problems of memory allocation request waiting and system blocking caused by idle memory exhaustion and reducing the pause time caused by memory exhaustion through preemptive memory recovery.
As shown in fig. 2, step S103 of the present embodiment further includes determining whether the number of free pages of the higher level of the memory is increased, if the number of free pages of the higher level of the memory is not increased, determining that the memory is severely fragmented, calculating the number of free pages of each level respectively, and eliminating memory fragments by enabling memory regularity for the pages of each level of the memory. If serious fragmentation occurs in the memory, the application program can not apply for the large memory, and the mechanism can manually trigger the memory regulation mechanism of the kernel so as to avoid the system from memory shortage and system blocking caused by fragmentation.
As shown in fig. 2, the water line self-adjusting method under the Linux operating system of the present embodiment may be generally divided into three functions, that is, a monitoring mechanism, a prediction mechanism and an adjusting mechanism, where the monitoring mechanism mainly designates the number of idle pages of the sampling memory; the prediction mechanism is used for calculating consumption trend of the total idle memory and the fragmented idle memory, determining whether the idle memory is consumed immediately or not through a mathematical prediction method, constructing a prediction model according to the number of idle pages obtained by sampling in a current time window and sampling time, and predicting the memory consumption time according to a model function obtained by linear fitting of the prediction model; the adjustment mechanism is mainly used for triggering the asynchronous recovery process in advance to carry out page recovery by changing the height of the water line after judging that the idle memory is about to be exhausted; the processing related to the three functions, including the calculation of the recovery efficiency of the memory page, the judgment of the recovery efficiency, and the judgment of the about to be exhausted of the free memory, can be divided into related functions according to actual needs, for example, the calculation of the recovery efficiency of the memory page after the memory recovery is started in fig. 2 is divided into monitoring functions. Referring to fig. 2, in the platform adopting the phy ium processor and running the gale kylin operating system in this embodiment, the monitoring mechanism needs to obtain the number of idle pages of the memory of each level by obtaining the partner system information, the virtual memory statistics information and the memory node statistics information respectively. The free pages of these memories need to be ordered before use, and at any time, the free pages of these memories represent the amount of memory available for immediate use, and once exhausted, memory allocation will cease immediately. Specifically, the monitoring mechanism in this embodiment is implemented by a daemon kdaemon in the kernel of the Linux operating system, and monitors the number of idle pages, page allocation information, partner system information and waterline related information (access of file information needs to ensure that the mechanism operates in root identity) by accessing/proc/statistics information of three files including virtual memory statistics vmstat, partner system information buddyinfo and memory node statistics information zoneinfo in a memory management area under a file system, and monitors the rate and usage of system memory consumption and the serious condition of memory about to be consumed and memory fragmentation by using the information. The daemon kdaemon is used for periodically monitoring and optimizing the state of the idle memory on the system and providing idle pages as much as possible. As shown in fig. 3, it supports three levels of aggressiveness: -1 = low, 2 = normal (default), 3 = high. These three different aggressiveness levels determine how aggressive the mechanism is at sampling the system state and adjusting the water line scaling factor. Low: the memory statistics are sampled at longer intervals and the system is slowly and gradually adjusted. Normal: the sample memory statistics are at a normal level so as not to significantly increase the system load, maintaining the adjustment at the normal level. High: memory statistics are sampled more frequently and the system is often tuned to maintain a large number of free pages available. The daemon kdaemon monitors the number of the whole idle pages and the number of idle pages of different order types by acquiring the statistical information and the memory management information of the kernel partner system, and judges whether the water line scale factor needs to be adjusted to adjust the water line height according to the number of the current idle pages and the water line height. Memory allocation because of the paging mechanism of the memory management unit MMU, only memory blocks of a predetermined size can be allocated to users, so that the extra space created by rounding off the required size is called internal fragmentation. The degree of memory fragmentation is from 0 to 1000 relative to the continuous free memory blocks of a certain level type page, and the larger the value, the higher the fragmentation degree. The higher the memory fragmentation degree is, the fewer the number of the high-order idle pages is, and the application requirement of the process on the high-order page block cannot be met, so that the system is blocked and does not respond. Therefore, the monitoring process needs to judge the fragmentation degree of the current memory according to the number of the high-order idle pages of the current memory and the difference value between the number of the high-order idle pages and the number of the low-order idle pages, and when the memory reclamation is not started, the smaller the number of the high-order idle pages, the larger the number of the low-order idle pages and the more serious the memory fragmentation.
In this embodiment, the calculation of the recovery efficiency of the memory pages in step S101 is obtained by dividing the number of memory pages recovered during memory recovery by the number of memory pages to be scanned and multiplying by 100%, and may be expressed as:
recovery efficiency of memory pages= (number of recovered memory pages/number of scanned memory pages) ×100%.
The adjustment triggering time of the prediction mechanism is determined according to the predicted memory consumption trend line. In order to implement the prediction mechanism, in this embodiment, periodically sampling the idle pages of each order on each node to collect the number of idle pages and sampling time, and fitting a line formed by the sampling points in the sliding window period by using a least square method to obtain a best fitting line as a derived equation. The condition when idle memory exhaustion occurs is calculated according to the best fit equation.
In this embodiment, the step S102 of constructing the prediction model means that the least square method is used to linearly fit two variables, i.e. the number of idle pages and the time, and the function expression of the model function obtained by the linear fitting is:
，
in the above-mentioned method, the step of,for the number of free pages at time x +.>And->For fitting parameters +. >Time is; and when linear fitting is carried out, the function expression of fitting calculation of the fitting parameters is as follows:
，/>，
in the above-mentioned method, the step of,is->Number of free pages at time, +.>Is the number of free pages sampled. To realize the data needed for constructing the predictive model, it is necessary to sample each +.>The number of free pages at the moment>Then according to the number of idle pages sampled in the current time window +.>Sample time->And constructing a prediction model by utilizing a curve formed by linearly fitting sample points by using a least square method, and predicting the time of memory exhaustion according to a model function obtained by linearly fitting the prediction model. Fitting parameters ∈>And->The derivation of the fit calculated functional expression of (c) is as follows: most of the workload in Linux systems follows the memory usage pattern, but it is usually a short-term pattern, which varies over time. By analyzing the current memory usage pattern of the workload, a recent memory usage pattern may be predicted. Conversely, if the current memory consumption and idle mode remain unchanged, any upcoming idle page exhaustion on the system can be predicted. The prediction mechanism uses a least square fitting method to calculate a functional relationship curve conforming to the number of free pages and time. It periodically collects the number of free pages of each order by means of sliding time window sampling as data points of the predictive model. Fig. 5 (a) shows a data set of the number of free pages and time collected periodically, and the horizontal axis shows time, and the vertical axis shows the number of free pages. Using the least squares method, it is found by observation that it is more linear, two variables with linear relationship can be expressed as F (x) =c 1 f 1 (x) + C 2 f 2 (x) Where F (x) is the number of free pages at time x, C 1 And C 2 Is two unknown constants, and the coordinates of points on the image are noted as (x 1 ,y 1 ),…,( x n ,y n ) The function being in accordance with a linear relationship, i.e. f 1 (x)=1，f 2 (x) =x, equation 1 can be obtained:
，
the error term is noted as E, representing the sum of squares of the differences between these quantities at each data point, resulting in equation 2:
，
in the above-mentioned method, the step of,is the theoretical value of the free memory pages. Minimizing the error term E, minimizing the bias of each fitting parameter by the error term E, and making the result 0, yields equation 3:
，
。
converting equation 3 into a matrix form, the reduction can yield equation 4:
，
can be solved to fit parametersAnd->The expression of the calculation function of (c) is:
，/>，
in the above-mentioned method, the step of,is->Number of free pages at time, +.>Is the number of free pages sampled. Fitting parameters +.>Andand substituting the values of (2) into equation F (x) to obtain a regression equation, namely a prediction model equation. According to the obtained prediction model, the slope represents the consumption speed of the idle page, and the curve is only used as a trend line to compare the fitting effect of the sample points, does not participate in the actual calculation process, and only reflects the consumption condition of the idle page. Moreover, the curve cannot be consumed to the minimum due to the existence of reserved memory, and intercept cannot exist in practice. The intercept a is only used to represent the coefficient of the fitted curve equation, and is not of practical significance in order to obtain the predictive equation. By adopting the sliding window mode, the trend line in the sliding window is calculated, and the sliding window advances at each data collection interval, so that the current memory consumption trend can be effectively calculated even when the memory consumption trend of the workload changes, as shown by the trend line after fitting in (b) in fig. 5. Therefore, the prediction of the memory exhaustion time according to the fitted curve obtained in step S102 in this embodiment means: and obtaining the intercept of the fitting curve and the time axis, and taking the moment corresponding to the intercept as the predicted memory exhaustion time.
The prediction mechanism calculates the recovery rate of the current page according to the total amount of the idle pages of the memory and the number of the idle pages of each level, if the recovery rate exists, if the number of the idle pages of the recovery rate is far greater than the number of the idle pages of the memory, the idle memory of the system is not consumed, a prediction model is not required to be built, the condition for building the prediction model is waited to be achieved (namely, the number of the idle pages of the recovery rate is smaller than the number of the idle pages of the consumption, when the risk of the memory consumption exists), a least square method is used for fitting the function of the number of the idle pages of the memory and the time, the time when the memory is about to be consumed is predicted, and the adjustment is made as early as possible before. In Linux memory management, various operations of idle memory management are triggered by min, low, high of the 3 memory water lines. As shown in fig. 4, when the number of free pages reaches a low water line (low water mark), the memory allocation process will enter a slow path, at which time an asynchronous reclamation process (kswapd) wakes up and starts scanning the page cache for pages that can be reclaimed, where zone represents a memory management region. When the method finishes scanning the page cache, if the received small memory pages with low orders appear, the number of the high-order page blocks is small, and when serious fragmentation occurs, a memory regularization process (kcompactd) is awakened to regularize any continuous idle page, and the low-order pages are combined into high-order page blocks, so that the application requirement of the high-order page blocks is met. If the memory continues to consume rapidly, the number of free pages will reach the minimum water line (min watermark), at which point any non-GFP_ATOMIC request to allocate memory will cause the process of the allocated memory request to enter a direct reclamation/normalization path. Once the free memory is lower than the lowest water line, the residual memory of the system is very small at the moment, the situation of direct recovery/regularity can be entered, the risk of problems of the application and the system is greatly increased at the moment, the memory requirement can be met by killing the process occupying higher memory when serious, and the instability of the system is greatly increased. The memory allocation request of the normal process is placed in the waiting queue, the process is not awakened until enough free pages can meet the allocation request, the memory allocation instruction is executed, and only the allocation request of the process marked as GFP_ATOMIC can be met under the condition that enough free memory is not recovered. However, in this case, the memory allocation or memory regularity may be stopped, and the system may be delayed, which may have a significant performance impact on the workload. The magnitude of this effect is also uncertain, depending on the frequency of occurrence of the pauses and how long each instance lasts. As the asynchronous reclamation process (kswapd) continues to reclaim the available pages, the number of free pages eventually reaches a high water line (high watermark), at which time the memory reclamation mechanism stops, the number of free pages reaches a normal level, and the asynchronous reclamation process (kswapd) enters a sleep state. The kernel responds to the out-of-memory event after it occurs. On the slow path, if the idle memory allocation is unsuccessful, the method of direct page reclamation, direct memory regularity, minimum warning water line, OOM Killer mechanism or repeated reclamation is used to reclaim the memory, the number of idle pages is increased, and the request of memory allocation is satisfied. The normal priority allocation requests cannot access memory below the lowest guard water line. If the allocation is unsuccessful, only an error message can be output, returning NULL. OOM: the full term Out Of Memory, i.e., memory exhaustion. When the system memory is insufficient, OOM is generated, the OOM killer process is started, and an appropriate process is selected to kill the process. The rank of pages in memory management means that the buddy allocator can allocate up to the number of pages at a time. The high-priority allocation request can access the reserved memory below the lowest warning water level, and the access ranges are different according to different priorities. When the memory consumption speed is high, the low water line to the lowest water line may become a repeated event. This can result in frequent memory stalls. In order to reduce this situation, the minimum water line min and the low water line low can be increased, and the interval between the low water line low and the high water line high can be increased, the rise of the low water line low can wake up the asynchronous recovery process (kswapd) of the kernel as early as possible, the rise of the high water line high can delay the sleep of the asynchronous recovery process (kswapd), and more idle memories can be recovered to meet the requirement of system memory allocation.
When the impending idle memory exhaustion is predicted, the adjustment mechanism manually triggers the memory ordering mechanism at the last necessary moment to restore the fragmented memory before the subsequent memory allocation request. If the number of free pages is predicted to run out, the high and low waterline is regulated by the waterline proportion factor to start memory reclamation in advance/postpone, and whether page reclamation is caused by changing the waterline is determined to be significant by checking the number of inactive pages in the cache buffer. It should be noted that, there may be many alternative different ways to change the height of the water line by the adjustment mechanism, for example, as an alternative implementation, in the present embodiment, the height of the water line is changed by changing the water line scale factor watermark_scale_factor in the Galium operating system. The kernel provides an interface under the/proc/file system that can adjust the inter-waterline spacing, which indirectly adjusts the height between the water lines while keeping the lowest water line unchanged, which is called the waterline scale factor (watermark scale factor). By changing the magnitude of the factor values, the water line heights of the low water line low and the high water line high can be increased or decreased. The kernel provides a parameter min_free_kbytes as a minimum number of free memory pages in the user space to meet the allocation requirements of the memory application. This minimum is reflected in the calculation of the lowest water line. The kernel calculates the three water line values using the two parameters min_free_kbytes and the watermark scale factor watermark_scale_factor of the __ setup_per_zone_wmarks () function in mm/page_alloc.c, the value of min_free_kbytes being calculated by the init_per_zone_wmark_min () function in mm/page_alloc.c. Increasing the distance between the water lines allows the asynchronous reclamation process (kswapd) to begin reclamation earlier and reclaim more pages before re-entering the sleep state. This approach works well during periods of high memory consumption. When the memory consumption rate decreases, maintaining a higher water line height may result in a decrease in the amount of free memory originally available to the system, and fewer free pages may be used for page caching, which may affect the system performance by reducing the caching effect. To solve this problem, it is necessary to lower the water line in the period where the memory demand is low and raise the water line in the period where the memory demand is high. Therefore, the adjustment mechanism of the water level line is preferably self-adaptive, and can meet the requirement of specific scenes on the height of the water level line according to the judgment of the current state.
When the kernel generates memory reclamation, the reclaimed pages influence the quantity of the idle pages, at the moment, the page reclamation rate needs to be calculated to judge the influence degree of the memory reclamation on the quantity of the idle pages, the prediction model presumes that the function relation between the idle memory consumption and the time is linearly related, but in practice, the speed of the memory reclamation and the consumption needs to be judged to be faster, and then whether the height of a water line needs to be adjusted to determine the time for waking up the memory reclamation is judged. After the prediction equation is obtained, the number of idle pages when the idle pages are exhausted (i.e. the value of the lowest water line is reached) can be substituted into the prediction equation, the corresponding time when the memory is exhausted can be roughly estimated, and the time x when the number of idle pages reaches the high water line or the low water line can be solved by using the value of the low water line and the value of the high water line according to the prediction equation. With a trend line that can be used to predict the amount of free pages that will run out for a corresponding time, the prediction mechanism can calculate in advance how long it will take to adjust the water line to ensure that free pages will not run out.
According to the number of idle pages of each level provided by the kernel, if the number of idle pages of the level is small but the number of pages of the level smaller than the number of idle pages of the level is large, namely the situation that fragmentation exists, a difference value indicates the intensity of the fragmentation, the fragmentation degree of each level reflects the number of idle pages of the level and the adjacent level, if the fragmentation degree is high, the number of recovered low-level pages is large, but the number of high-level pages is small, if the memory application requirement of the high-level pages is not met, the kcompact thread of the kernel is required to be called for memory regulation, and the low-level pages are combined into the high-level pages so as to meet the requirement of the application of the high-level memory pages. The number of the first four idle pages with the largest order is generally taken for observation, compared with the number of the idle pages with other orders, and the difference result is used as a basis for judging the number of the idle pages with the higher order. The number of pages refers to the number of free pages that can be applied without allocation. The monitoring process needs to transmit the number of idle pages of each order, if the total number of idle pages is lower than the low water level line low, the memory recovery is triggered, the recovery rate is calculated according to the number of scanned pages and the number of recovered pages, whether the consumption trend of the idle pages is changed or not is judged according to the recovery rate, otherwise, the equation of the prediction curve is inaccurate, the condition that the idle pages are exhausted does not exist, and the height of the water level line does not need to be adjusted at the moment. The number of idle pages of each level of the memory is obtained in a prediction stage, the memory fragmentation is judged in an adjustment stage, the fragmentation degree of the current memory is judged according to the number of idle pages of each level, and then whether manual intervention adjustment is performed or not is determined, wherein the flow of the prediction process is shown in fig. 6.
In step S103 of this embodiment, after memory regularity is enabled to eliminate memory fragments, the method further includes setting a memory regularity flag; as shown in fig. 7, changing the height of the water line to trigger the asynchronous reclamation process in advance for page reclamation in step S103 includes:
s201, acquiring a current water line scale factor and a lowest water line (min);
s202, calculating a high water level line (high) and a low water level line (low) according to the water level line scale factor and the lowest water level line, wherein the three methods are self-contained methods in a water level line mechanism;
s203, acquiring the current number of idle pages of the memory;
s204, judging whether the current free page number of the memory is larger than a high water line, if the current free page number of the memory is smaller than or equal to the high water line, judging that the memory pressure is not relieved, continuously judging whether the current free page number of the memory is smaller than half of the difference value between the high water line and the low water line, if not, jumping to the step S203, if so, adjusting the water line scale factor according to the appointed proportion to reduce the water line, ending and exiting; if the current number of free pages of the memory is greater than the high water line, determining that the memory pressure is relieved, and jumping to step S205;
S205, judging whether the current number of idle pages of the memory reaches the upper part of the high water line, if not, adjusting the water line scale factor according to the appointed proportion to enlarge the water line, ending and exiting; otherwise, jumping to step S206;
s206, judging whether the current number of the high-order idle pages of the memory is smaller than a set value, if so, judging whether the kernel enables memory regularity according to the mark of whether the memory regularity exists, and if so, clearing the mark of the memory regularity; if the memory regularity is not started, the memory regularity is started to eliminate memory fragments so as to obtain more memory pages.
As a preferred implementation, the present embodiment further includes limiting the adjusted water line scaling factor to a preset threshold range of the water line scaling factor after adjusting the water line scaling factor according to the specified ratio, so as to prevent the user from being killed due to overrun of the water line scaling factor.
In this embodiment, adjusting the water line scale factor according to the specified ratio to reduce the water line refers to reducing the water line scale factor according to the 10% ratio to reduce the water line; the adjusting the water line scale factor according to the specified proportion to enlarge the water line refers to enlarging the water line scale factor according to the proportion of 10% to reduce the water line.
The adjustment mechanism is an important step of adjusting the waterline according to the prediction result of the prediction mechanism in this embodiment. As memory consumption increases, the adjustment mechanism will increase the water line height by increasing the water line scale factor (watermark_scale_factor); as memory consumption decreases, it decreases the water line height by decreasing the water line scale factor. In the memory management, the computation of the low water line and the high water line is obtained by the computation of __ setup_per_zone_wmarks () function in the kernel according to two parameters of the memory min_free_k bytes reserved by the system and the watermark scale factor watermark_scale_factor. The min_free_kbytes parameter is set at kernel initialization, depending on the amount of direct mapped region physical memory. The total value of min_free_bytes is approximately equal to the sum of the available memory of all the memory management areas zones multiplied by 16, the low water line low and the high water line high of one zone are calculated according to the value of the lowest water line min, the low water line low is about 1/4 larger than the value of the lowest water line min, the high water line high is about 1/2 larger than the value of the lowest water line min, and the ratio of the three is approximately 4:5:6. The adjustment mechanism can acquire the number of idle pages in the current time, judges whether the current memory pressure is relieved, namely the memory consumption is not continuously increased, and when the number of recovered pages exceeds the number of pages being applied (or the number of the recovered pages is equal to the number of the pages) and reaches above a low water line, the memory pressure is gradually relieved, when the number of the idle pages reaches a high water line, an asynchronous recovery process (kswapd) is stopped, the number of the idle pages is recovered to a normal level, and the memory pressure is relieved. If the memory pressure is being relieved, judging whether the number of the idle pages reaches above a high water line or not; if no relief exists, whether the number of the idle pages is less than 1/2 of the difference value between the low water line and the high water line is continuously judged, if the number of the idle pages is less than 1/2 of the difference value between the low water line and the high water line, the water line is reduced according to a proportion of 10% by adjusting the proportion factor, and the kernel waits for memory recovery. If the number of the idle pages reaches above the high water line, the memory recovery is completed, whether the number of the high-order idle pages is smaller than a set value is continuously judged, if the number is smaller than the set value, fragmentation of the current memory is indicated, whether the memory is received by the kernel or not needs to be judged, and a memory regulation mechanism is started. Two memory management mechanisms of the kernel send out the memory management signals when the memory is insufficient, so that asynchronous memory management is triggered, and in a slow path, the memory management signals from the kernel can be received, and the memory management is performed directly. The prediction mechanism is added with a signal capable of sending out the recommended memory regulation, and if the recommended memory regulation signal from the prediction process is received, the memory regulation is started by manually writing parameters. If the memory regulation is started, the error of the memory regulation is indicated, a request mark of the memory regulation is cleared, repeated requests are not needed, resource consumption caused by secondary regulation is avoided, and the recovery of the kswapd kernel thread is continued. If no memory ordering signal is received, then at the last possible time, and only if absolutely necessary, the ordering is manually initiated to restore fragmented memory before the subsequent allocation needs. The method starts memory normalization by writing an interface of 1 to/sys/devices/system/node/node% d/compare parameters, combines low-order idle memories, and obtains a larger number of high-order idle memories. If the number of free pages does not reach above the high water line, the memory pressure is recovered, more memory can be recovered, the waterline scaling factor needs to be adjusted to increase the waterline height by ten times each time of expansion, and then the asynchronous recovery process (kswapd) is waited for completing memory recovery. If the mechanism predicts that free pages will be used up, it will look at the number of inactive pages in the cache buffer, if the number of inactive pages is greater, indicating that more cache pages can be reclaimed, it will be further determined if changing the height of the water line will make page reclamation meaningful, thereby reclaiming a greater number of free pages. It is observed from the actual workload that if a fixed threshold is not set to limit the adjustment range of the watermark scale factor, it may either be possible to raise the watermark to a sufficiently high value, resulting in the number of currently free pages being below the low watermark value. This can lead to the OOM-killer mechanism immediately killing the user process, increasing the system's instability factors and risk, so it needs to be ensured that the adjustment mechanism will not change the watermark scale factor to an uncontrollable value, and a fixed changed threshold needs to be set, avoiding the situation that the OOM killer is inadvertently invoked to kill the user process during the adjustment process. Therefore, the prediction process needs to predict the occurrence time of memory exhaustion according to the number of idle pages of each stage at the sampling time point according to the statistics condition of idle pages obtained by the monitoring process, and transmit the time point to the adjustment process, and the height of the water line is adjusted in advance before the time point arrives, so as to change the time for memory reclamation. The number of each rank page is counted in the period, if the number of the higher rank page blocks is too small, the allocation request of the higher rank page cannot be reached, and the memory regulation mechanism is manually triggered by writing parameters in the adjustment process, so that more higher rank pages are obtained, the fragmentation outside the memory is relieved, and the requirement of a low-memory scene is met.
In summary, the method of the embodiment realizes the self-adaptive adjustment mechanism of the waterline in the Linux kernel memory management, and can automatically adjust the height of the waterline according to the use condition of the memory to meet the requirement of memory management stability. The method of the embodiment can be applied to memory management of a Linux system, can effectively solve the problems of stuck state, delay and operation uncertainty caused by waiting for memory recovery when the memory is about to be exhausted, and enhances the system stability. The method of the embodiment monitors the total idle memory amount according to the statistical information of the partner system, and once the idle memory amount is exhausted, the memory allocation request is stopped immediately. According to the method, idle memory consumption conditions and memory fragmentation trends are calculated, a function curve of idle memory quantity and time is fitted through a least square method, whether memory exhaustion is about to happen or not is predicted according to the idle memory quantity, a waterline threshold value is adaptively adjusted through changing a waterline scale factor watermark_scale_factor, and memory is recovered as soon as possible; and finally, the compact memory is started to restore the fragmented memory when necessary, thereby meeting the requirement of the system on the allocation of the high-order memory and reducing the stop times and stop time when the system restores the memory.
In addition, the embodiment also provides a water line self-adjusting system under the Linux operating system, which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the water line self-adjusting method under the Linux operating system. In addition, the embodiment also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is used for being programmed or configured by a microprocessor to execute the water line self-adjusting method under the Linux operating system.
To measure the impact of the watermarks self-tuning mechanism of the invention on memory allocation stalls, a workload is artificially created that can meet the requirement that a memory allocation request can be stalled in a reasonable time. The workload performs a large number of I/O operations, modifies the file system at high speed, allocates a large number of pages of different orders, and may result in a large number of page caches, resulting in pages that need to be reclaimed quickly. The load is realized by the following task combination: the dd command is used to add 3G data (from/dev/zero) to a file on one/mnt/side. Three such tasks are run simultaneously. The dd command is used to add 6G data (from/dev/zero) to a file on one/mnt/up. Three such tasks are run simultaneously. 12G of data (from/dev/zero) is added to a file on one/mnt/using dd command. Three such tasks are run simultaneously.
While the kernel compilation process is running in a loop on another file system. The watermarks self-adjustment mechanism realized based on the Galaxy kylin operation system is started, and the mechanism is divided into three modules, namely a monitoring mechanism, a prediction mechanism and an adjustment mechanism: in the monitoring mechanism, a daemon process kdamond is constructed at the time of use firstThe method is used for periodically monitoring and optimizing the state of the idle memory on the system and providing idle pages as much as possible. The aggressiveness of its use is given by default option, the normal sampling frequency is maintained and the aggressiveness is adjusted. By giving the root manager authority, the statistical information of the memory management area and the statistical information of the partner system under the proc/file system can be checked, and the process can monitor the current state of the whole idle page and the state of each level page and count the idle page consumption and the memory fragmentation of the current memory. In the prediction mechanism, according to the consumption condition of the free pages of the current memory and the fragmentation condition of the memory obtained in the previous stage, by calculating the recovery efficiency of the pages and the number of the free pages, a least square method is used to fit a trend line which accords with the function relation between the number of the free pages of the memory and time, and the two quantities with linear relation are expressed as Wherein->Is at->Number of free pages at time, +.>And->Representing two fitting coefficients, +.>Is time. And calculating a trend line in each sliding window by adopting a sliding window mode (the data points in the window are set as 8 here), and advancing the sliding window at each data collection interval to obtain a fitted trend line. It is possible to calculate for each order page when the high and low watermark values are reached using the low and high watermark values by this equation. The trend line is predicted forward to determine when pages of each rank will run out. Inner collected by vmstat and budysinfoAnd (3) storing information, predicting whether the memory is about to be exhausted according to the result of a prediction equation, calculating how long to adjust the watermark in advance, sending out a suggestion of adjusting a waterline scale factor to preemptively recycle the memory, and reducing the number of times and the time of system pause under the condition of high memory consumption rate so as to ensure the stable operation of the system in a low memory scene.
In the adjustment mechanism, the value of the current waterline proportion factor is firstly obtained, and the current high waterline value and low waterline value of the node are calculated. Then, according to the predicted trend line, observing whether the memory consumption of the system is gradually increased, when the memory consumption is lower than a low water line, the kernel wakes up an asynchronous recovery process (kswapd) to carry out asynchronous recovery, judging whether the memory pressure is relieved at the moment, if so, further confirming whether the number of idle pages is lower than half of the low water line and the high water line, if so, the mechanism can retract 10% of the reduced water line each time by changing the scale factor of the water line, and continuously waiting for the kswapd to carry out recovery; if the memory pressure is not relieved, the number of recovered pages is smaller than the number of consumed pages, whether the buffer area has recoverable pages or not is checked, if yes, a waterline proportion factor is adjusted, waterline is enlarged, and the number of the pages is increased by 10 times each time, so that kswapd can continuously recover enough pages. If there is not enough page available, it needs to judge whether the memory regulation signal is received, the memory regulation has been started, avoiding the resource consumption caused by the secondary regulation, if the memory regulation has been started, the memory regulation request mark of the sampling interval is cleared, and the memory regulation process is started manually by writing the interface of 1 to/system/devices/system/node/node% d/compare parameters. The mechanism can lead the memory water line management of the kernel to have initiative and self-adaption, and enhance the stability and reliability of the system in the face of low memory scenes. By using the water line self-adjusting method under the Linux operating system of the embodiment to adaptively adjust the height of the water line, the system pause times and pause time under the condition of high memory consumption rate are effectively reduced, which is significant for the memory management mechanism of the kernel. According to the method, the self-adjustment of the water line is realized according to the current actual memory consumption and the situation of triggering asynchronous recovery, the memory recovery can be started in advance to avoid system blocking and operation delay, the pause can be reduced even under the condition of high memory consumption rate, the CPU resource waste caused by page recovery is avoided before the absolute necessity, the requirements of various load scenes can be met, and the running stability of an operating system is ensured.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.
Claims (8)
1. A water line self-adjusting method under a Linux operating system is characterized by comprising the following steps:
s101, sampling the number of idle pages of a memory, calculating recovery efficiency of the memory pages after starting memory recovery, if the recovery efficiency is smaller than a set value, jumping to the step S102, otherwise jumping to the step S101, and continuing to sample;
s102, constructing a prediction model according to the number of idle pages obtained by sampling in a current time window and sampling time, and predicting the memory exhaustion time according to a model function obtained by linear fitting of the prediction model;
s103, before the time of memory exhaustion comes, judging whether the number of idle pages is between a high water line and a low water line, if so, judging that the idle memory is about to be exhausted, and triggering an asynchronous recovery process in advance by changing the height of the water line to recover pages;
Step S103 also comprises judging whether the number of free pages of the high-order memory is increased, if the number of free pages of the high-order memory is not increased, judging that the memory is severely fragmented, respectively calculating the number of free pages of each order, and eliminating memory fragments for the pages of each order of the memory by starting memory regularity;
step S103, after enabling memory regularity to eliminate memory fragments, setting a memory regularity mark; the step S103 of changing the height of the water line to trigger the asynchronous reclaiming process in advance to reclaim the page includes:
s201, acquiring a current water line scale factor and a lowest water line;
s202, calculating a high water level line and a low water level line according to the water level line scale factors and the lowest water level line;
s203, acquiring the current number of idle pages of the memory;
s204, judging whether the current free page number of the memory is larger than a high water line, if the current free page number of the memory is smaller than or equal to the high water line, judging that the memory pressure is not relieved, continuously judging whether the current free page number of the memory is smaller than half of the difference value between the high water line and the low water line, if not, jumping to the step S203, if so, adjusting the water line scale factor according to the appointed proportion to reduce the water line, ending and exiting; if the current number of free pages of the memory is greater than the high water line, determining that the memory pressure is relieved, and jumping to step S205;
S205, judging whether the current number of idle pages of the memory reaches the upper part of the high water line, if not, adjusting the water line scale factor according to the appointed proportion to enlarge the water line, ending and exiting; otherwise, jumping to step S206;
s206, judging whether the current number of the high-order idle pages of the memory is smaller than a set value, if so, judging whether the kernel enables memory regularity according to the mark of whether the memory regularity exists, and if so, clearing the mark of the memory regularity; if the memory regularity is not started, the memory regularity is started to eliminate memory fragments so as to obtain more memory pages.
2. The method for self-adjusting a water line under a Linux operating system according to claim 1, wherein in step S101, the recovery efficiency of the memory pages is calculated by dividing the number of memory pages recovered during memory recovery by the number of memory pages to be scanned and multiplying the number of memory pages by 100%.
3. The method for self-adjusting a water line under a Linux operating system according to claim 1, wherein in step S102, the constructing a prediction model means to linearly fit two variables, i.e. the number of idle pages and time, by using a least square method, and a function expression of a model function obtained by the linear fitting is:
，
In the above-mentioned method, the step of,for the number of free pages at time x +.>And->For fitting parameters +.>Time is; and when linear fitting is carried out, the function expression of fitting calculation of the fitting parameters is as follows:
，/>，
in the above-mentioned method, the step of,is->Number of free pages at time, +.>Is the number of free pages sampled.
4. The method for self-adjusting a water line under a Linux operating system according to claim 1, wherein the predicting the memory exhaustion time according to the fitted curve obtained by fitting in step S102 means: and obtaining the intercept of the fitting curve and the time axis, and taking the moment corresponding to the intercept as the predicted memory exhaustion time.
5. The method for self-tuning a water line in a Linux operating system according to claim 1, wherein after tuning the water line scale factor according to the specified ratio, the method further comprises limiting the tuned water line scale factor to a preset water line scale factor threshold range to prevent user progress from being killed due to overrun of the water line scale factor.
6. The method for self-adjusting a water line under a Linux operating system according to claim 5, wherein adjusting the water line scale factor according to a specified ratio to reduce the water line means that the water line scale factor is reduced according to a 10% ratio to reduce the water line; the adjusting the water line scale factor according to the specified proportion to enlarge the water line refers to enlarging the water line scale factor according to the proportion of 10% to reduce the water line.
7. A water line self-adjusting system under a Linux operating system, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to perform the water line self-adjusting method under the Linux operating system according to any one of claims 1 to 6.
8. A computer readable storage medium having a computer program stored therein, wherein the computer program is configured or programmed by a microprocessor to perform the method of self-adjusting a water line under a Linux operating system according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310495310.4A CN116225976B (en) | 2023-05-05 | 2023-05-05 | Water line self-adjusting method and system under Linux operating system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310495310.4A CN116225976B (en) | 2023-05-05 | 2023-05-05 | Water line self-adjusting method and system under Linux operating system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116225976A CN116225976A (en) | 2023-06-06 |
| CN116225976B true CN116225976B (en) | 2023-08-08 |
Family
ID=86582901
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310495310.4A Active CN116225976B (en) | 2023-05-05 | 2023-05-05 | Water line self-adjusting method and system under Linux operating system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116225976B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109697119A (en) * | 2017-10-20 | 2019-04-30 | 中兴通讯股份有限公司 | Terminal Memory recycle processing method and processing device |
| CN110888746A (en) * | 2019-12-10 | 2020-03-17 | Oppo（重庆）智能科技有限公司 | Memory management method and device, storage medium and electronic equipment |
| CN111831440A (en) * | 2020-07-01 | 2020-10-27 | Oppo广东移动通信有限公司 | Memory recovery method and device, storage medium and electronic equipment |
| CN113138940A (en) * | 2021-04-02 | 2021-07-20 | Oppo广东移动通信有限公司 | Memory recovery method and device, electronic equipment and storage medium |
| CN113986559A (en) * | 2021-12-24 | 2022-01-28 | 荣耀终端有限公司 | Memory management method and related device |
| CN114168490A (en) * | 2020-09-10 | 2022-03-11 | 华为技术有限公司 | Method for determining memory recovery threshold and related equipment |
| CN114218126A (en) * | 2021-11-04 | 2022-03-22 | 北京紫光展锐通信技术有限公司 | Memory recovery method and device |
| CN114443268A (en) * | 2020-10-31 | 2022-05-06 | 华为终端有限公司 | Memory management method and device, electronic equipment and computer readable storage medium |
| CN115168052A (en) * | 2022-07-29 | 2022-10-11 | 苏州浪潮智能科技有限公司 | Self-adaptive memory recovery control method, device, terminal and storage medium |
| CN115421907A (en) * | 2022-08-18 | 2022-12-02 | Oppo广东移动通信有限公司 | Memory recovery method and device, electronic equipment and storage medium |
| CN115421909A (en) * | 2022-08-19 | 2022-12-02 | Oppo广东移动通信有限公司 | Memory management method and device, electronic equipment and storage medium |
| CN115587049A (en) * | 2022-09-29 | 2023-01-10 | Oppo广东移动通信有限公司 | Memory recovery method and device, electronic equipment and storage medium |
-
2023
- 2023-05-05 CN CN202310495310.4A patent/CN116225976B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109697119A (en) * | 2017-10-20 | 2019-04-30 | 中兴通讯股份有限公司 | Terminal Memory recycle processing method and processing device |
| CN110888746A (en) * | 2019-12-10 | 2020-03-17 | Oppo（重庆）智能科技有限公司 | Memory management method and device, storage medium and electronic equipment |
| CN111831440A (en) * | 2020-07-01 | 2020-10-27 | Oppo广东移动通信有限公司 | Memory recovery method and device, storage medium and electronic equipment |
| CN114168490A (en) * | 2020-09-10 | 2022-03-11 | 华为技术有限公司 | Method for determining memory recovery threshold and related equipment |
| CN114443268A (en) * | 2020-10-31 | 2022-05-06 | 华为终端有限公司 | Memory management method and device, electronic equipment and computer readable storage medium |
| CN113138940A (en) * | 2021-04-02 | 2021-07-20 | Oppo广东移动通信有限公司 | Memory recovery method and device, electronic equipment and storage medium |
| CN114218126A (en) * | 2021-11-04 | 2022-03-22 | 北京紫光展锐通信技术有限公司 | Memory recovery method and device |
| CN113986559A (en) * | 2021-12-24 | 2022-01-28 | 荣耀终端有限公司 | Memory management method and related device |
| CN115168052A (en) * | 2022-07-29 | 2022-10-11 | 苏州浪潮智能科技有限公司 | Self-adaptive memory recovery control method, device, terminal and storage medium |
| CN115421907A (en) * | 2022-08-18 | 2022-12-02 | Oppo广东移动通信有限公司 | Memory recovery method and device, electronic equipment and storage medium |
| CN115421909A (en) * | 2022-08-19 | 2022-12-02 | Oppo广东移动通信有限公司 | Memory management method and device, electronic equipment and storage medium |
| CN115587049A (en) * | 2022-09-29 | 2023-01-10 | Oppo广东移动通信有限公司 | Memory recovery method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116225976A (en) | 2023-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10037274B2 (en) | Optimizing memory usage across multiple applications in the presence of garbage collection | |
| US9256469B2 (en) | System and method for improving memory usage in virtual machines | |
| US6681305B1 (en) | Method for operating system support for memory compression | |
| US6877081B2 (en) | System and method for managing memory compression transparent to an operating system | |
| US7672983B2 (en) | Method and apparatus for tracking activity of a garbage collector with a plurality of threads that operate concurrently with an application program | |
| CN1133942C (en) | Cache architecture to enable accurate cache sensitivity | |
| US7165255B2 (en) | Method and apparatus for managing surplus memory in multitasking system | |
| US7024512B1 (en) | Compression store free-space management | |
| WO2021233261A1 (en) | Multi-task dynamic resource scheduling method | |
| US7325108B2 (en) | Method and system for page-out and page-in of stale objects in memory | |
| EP0472868A2 (en) | Method and apparatus for determining and using program paging characteristics to optimize system productive CPU time | |
| US20050149585A1 (en) | Method and apparatus for scheduling and performing garbage collection in a real-time system with guaranteed space bounds | |
| JP5989908B2 (en) | Method, system and apparatus for caching code in non-volatile memory | |
| WO2017069646A1 (en) | Method and device for determination of garbage collector thread number and activity management in log-structured file systems | |
| KR20140035416A (en) | Memory manager with enhanced application metadata | |
| JPH11328031A (en) | Trace ranking method in dynamic conversion system | |
| CN108563506A (en) | The method for recovering internal storage and device of mobile terminal | |
| CN116225976B (en) | Water line self-adjusting method and system under Linux operating system | |
| CN110795323A (en) | Load statistical method, device, storage medium and electronic equipment | |
| US20060179225A1 (en) | Reducing cache trashing of certain pieces | |
| US6842838B2 (en) | Preemptive memory-block splitting | |
| Park et al. | Suspend-aware segment cleaning in log-structured file system | |
| US20120239901A1 (en) | Dynamic measurement and adaptation of a copying garbage collector | |
| CN109086223B (en) | Method and device for controlling garbage recovery | |
| CN111090627A (en) | Log storage method and device based on pooling, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |