CN116382579A

CN116382579A - Memory normalization method, memory normalization device, computer equipment and storage medium

Info

Publication number: CN116382579A
Application number: CN202310299813.4A
Authority: CN
Inventors: 郑豪
Original assignee: Alibaba Cloud Computing Ltd
Current assignee: Alibaba Cloud Computing Ltd
Priority date: 2023-03-20
Filing date: 2023-03-20
Publication date: 2023-07-04

Abstract

The present disclosure provides a memory ordering method, apparatus, device, and storage medium, where the method includes: determining a regular target object and a regular source object according to the use state and the position of the memory block; the regular target object comprises scattered swapped-out memory block sets, and the corresponding regular source object comprises a cold block set adjacent to the continuous swapped-out space; the regular target object comprises a scattered free memory block set, and the corresponding regular source object comprises a cold block set adjacent to the continuous free space; the number of the continuous swapped out memory blocks in the scattered swapped out memory block set is smaller than a first threshold value; in the continuous swapped-out space, the number of the continuous swapped-out memory blocks is larger than or equal to a first threshold value; the number of continuous idle memory blocks in the scattered idle memory block set is smaller than a second threshold value; in the continuous idle space, the number of the continuous idle memory blocks is greater than or equal to a second threshold; and replacing the use state of the regular target object with the use state of the corresponding regular source object.

Description

Memory normalization method, memory normalization device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a memory management method, a memory management device, a computer device, and a storage medium.

Background

For each running process, the operating system allocates a certain number of memory blocks to it. For flexible use of memory, for each memory block allocated to each process, the memory management module of the operating system records the cold and hot state information of the memory blocks, and for the memory which is not frequently used by the process, the problem of memory space shortage can be solved through memory exchange. However, as the memory swap proceeds, the memory fragmentation may become worse, and a large amount of scattered free memory appears, so that a continuous large-segment memory cannot be allocated.

Disclosure of Invention

In order to overcome the problems in the related art, the present disclosure provides a memory management method, apparatus, computer device, and storage medium.

According to a first aspect of an embodiment of the present disclosure, there is provided a memory management method, including:

determining a regular target object and a corresponding regular source object according to the use state of each memory block in a storage space to be regular and the position of each memory block in the storage space to be regular; the regular target object comprises scattered swapped-out memory block sets, and the corresponding regular source object comprises a cold block set adjacent to the continuous swapped-out space; and/or the regular target object comprises a scattered free memory block set, and the corresponding regular source object comprises a cold block set adjacent to the continuous free space; the number of the continuous swapped out memory blocks in the scattered swapped out memory block set is smaller than a first threshold value; in the continuous swapped-out space, the number of the continuous swapped-out memory blocks is larger than or equal to a first threshold value; the number of continuous idle memory blocks in the scattered idle memory block set is smaller than a second threshold value; in the continuous idle space, the number of the continuous idle memory blocks is larger than or equal to a second threshold value;

And replacing the use state of the regular target object with the use state of the corresponding regular source object.

According to a second aspect of the embodiments of the present disclosure, there is provided a memory regulating apparatus, including:

the determining module is used for determining a regular target object and a corresponding regular source object according to the use state of each memory block in the storage space to be regular and the position of each memory block in the storage space to be regular; the regular target object comprises scattered swapped-out memory block sets, and the corresponding regular source object comprises a cold block set adjacent to the continuous swapped-out space; and/or the regular target object comprises a scattered free memory block set, and the corresponding regular source object comprises a cold block set adjacent to the continuous free space; the number of the continuous swapped out memory blocks in the scattered swapped out memory block set is smaller than a first threshold value; in the continuous swapped-out space, the number of the continuous swapped-out memory blocks is larger than or equal to a first threshold value; the number of continuous idle memory blocks in the scattered idle memory block set is smaller than a second threshold value; in the continuous idle space, the number of the continuous idle memory blocks is larger than or equal to a second threshold value;

And the replacement module is used for replacing the use state of the regular target object with the use state of the corresponding regular source object.

According to a third aspect of the embodiments of the present disclosure, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the memory arrangement method embodiments of the aforementioned first aspect are implemented when the processor executes the computer program.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the memory arrangement method embodiments of the first aspect described above.

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:

in the embodiment of the disclosure, by acquiring the use state of each memory block in the storage space to be regulated and the position of each memory block in the storage space to be regulated, scattered memory blocks which are replaced and corresponding cold blocks adjacent to the continuous replaced space can be determined; by replacing the two, a continuous large segment of swapped memory blocks is formed, and the large segment of swapped memory blocks can be recovered by the memory management module to allocate the continuous large segment of memory. Or, the embodiment can also determine scattered idle memory blocks and corresponding cold blocks adjacent to the continuous idle space; by replacing the two, a continuous free space can be formed. The embodiment can regulate the memory, change the memory in the form of continuous addresses as far as possible, and actively regulate the memory according to the situation of the fragmented memory at the same time, thereby ensuring that the idle memory is continuous as far as possible.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a schematic diagram of a reserved memory scenario according to an exemplary embodiment of the present disclosure.

Fig. 2A is a flow chart illustrating a memory management method according to an exemplary embodiment of the present disclosure.

FIG. 2B is a diagram of a NUMA architecture shown in accordance with an exemplary embodiment of the specification.

FIG. 2C is a schematic diagram of a portion of memory space of a memory according to an exemplary embodiment of the present disclosure.

Fig. 3 is a block diagram of an apparatus in which a memory management device is located according to an exemplary embodiment of the present disclosure.

Fig. 4 is a block diagram of a memory regulating device according to an exemplary embodiment of the present specification.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present description as detailed in the accompanying claims.

The terminology used in the description presented herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

The memory management module of the operating system divides the memory according to a set management granularity, and each management granularity may be called a page (page) or a block. In some examples, the computer device may employ a conventional memory management architecture, i.e., the entire memory is managed by the operating system. In other scenarios, for example, in a virtual machine scenario, a computer device may employ a memory allocation architecture of reserved memory, as shown in fig. 1, which is a schematic diagram of a reserved memory scenario according to an exemplary embodiment of the present disclosure, where a memory of a host may include multiple storage spaces, where two storage spaces of the memory are shown in fig. 1 with different filling manners, including unreserved storage space a (with diagonal filling in the drawing) for a kernel, and reserved storage space b (with grid lines filling in the drawing) for a virtual machine. That is, the unreserved storage space a is used for the kernel in the figure, and the granularity of division of the storage space by the kernel is usually small, for example, 4KB (Kilobyte) or the like; applications running on the operating system (e.g., application 1 through application 3 as illustrated in the figures) may use this unreserved storage space a. The reserved storage space b is available for Virtual Machines (VM), such as VM1 to VMn as shown in the figure. The two storage spaces may be managed with different granularity, i.e. the memory may be partitioned differently. For ease of illustration in fig. 1, two storage spaces are illustrated in a continuous fashion in the figure. It will be appreciated that in practice, the two storage spaces may be non-contiguous. In practical applications, the memory may be divided into more storage spaces to implement more different functions.

The reserved storage space occupies most of the memory and is not available to the host kernel, and a module can be inserted into the kernel of the operating system to be specially used for managing the reserved storage space. In order to facilitate management of the series of memories and avoid occupation of a large amount of metadata on the memories, and considering that when the memories are allocated to the virtual machines, the minimum time is hundreds of MB (megabytes), the reserved storage space is managed by dividing the reserved storage space into Memory blocks (MS) with the size of 2MB, for example; in some scenarios, large granularity is also commonly used, such as 1GB (GigaByte) is optional, and this embodiment is not limited thereto.

The memory management module in the operating system is used for managing the memory, when the memory management module is applied to the reserved memory scene, the operating system can adopt different memory management modules to respectively manage the reserved memory space and the unreserved memory space, for example, the reserved memory management module of the operating system manages the reserved memory space, and the kernel memory management module manages the unreserved memory space.

Since the physical memory of a computer is limited and the use of memory by processes is uncertain, there is always a possibility that the physical memory will be used up. The operating system adopts memory exchange to solve the problem of insufficient physical memory. The kernel selects the memory block in cold state for releasing, and performs memory block combination, data compression to other memory blocks or swap out to disk, etc. on the data stored in the memory block. And then the memory block is distributed to other more urgent processes for use, and when the original process is used, the data is replaced into a new memory block and is returned to the original process.

The memory management module scans the allocated memory blocks for cold and hot states. Specifically, the memory management module may continuously monitor information such as time or frequency of access of each memory block by the process, and determine a cold and hot state of the memory block. The hot state (active) indicates that the memory block is recently accessed by the process, the cold state (inactive) indicates that the memory block is not recently accessed by the process, and the memory management module records each memory block in the hot state through the hot page set, and records each memory block in the cold state through the cold page set. The cold page set records a set of memory blocks that have not been accessed recently. The portion of the memory block may be migrated to secondary storage in the event of a memory shortage to free up more memory for active virtual machines. Of course, the moved memory space can be used for inventory scheduling, so as to improve the utilization rate of host resources. Of course, this migration is temporary, and if the secondary storage is not a device (e.g., a disk) that can be accessed directly, a subsequent virtual machine continues to access, which triggers a page fault, and this time requires a fast reload operation. The memory blocks swapped out to the secondary storage can be actively swapped back to the main memory according to the requirements of scheduling and the like.

However, considering that the cold and hot degree of the memory of one virtual machine may be dynamically changed, after a period of operation, the exchanged memory may be scattered and distributed in different places of the memory space allocated to the virtual machine. Meanwhile, for the whole host machine memory, scattered memory blocks of a plurality of virtual machines are recovered, so that idle memory in the host machine is broken into a plurality of small segments, and continuous large-segment memory is difficult to allocate.

Based on this, the memory management scheme provided in the embodiments of the present disclosure can manage the memory, and can change out the memory in an address continuous manner as far as possible according to the cold and hot changes of the memory, and at the same time, actively manage the memory according to the situation of the fragmented memory, so as to ensure that the idle memory is continuous as far as possible. The following detailed description is given.

As shown in fig. 2A, a schematic diagram of a memory management method according to an exemplary embodiment of the present disclosure includes the following steps:

in step 202, a regular target object and a corresponding regular source object are determined according to the usage status of each memory block in the storage space to be regular and the location in the storage space to be regular.

The regular target object comprises scattered swapped-out memory block sets, and the corresponding regular source object comprises a cold block set adjacent to the continuous swapped-out space; and/or the regular target object comprises a scattered free memory block set, and the corresponding regular source object comprises a cold block set adjacent to the continuous free space; the number of the continuous swapped out memory blocks in the scattered swapped out memory block set is smaller than a first threshold value; in the continuous swapped-out space, the number of the continuous swapped-out memory blocks is larger than or equal to a first threshold value; the number of continuous idle memory blocks in the scattered idle memory block set is smaller than a second threshold value; and in the continuous idle space, the number of the continuous idle memory blocks is larger than or equal to a second threshold value.

In step 204, the usage status of the regular target object is replaced with the usage status of the corresponding regular source object.

The solution of this embodiment may be applied to any computer device, where the computer device may be a single-core CPU (Central Processing Unit ) device, or may also include a device with multiple physical CPUs, and may employ a non-uniform memory access (Non Uniform Memory Access Architecture, NUMA) architecture according to needs, where the NUMA architecture includes at least two NUMA nodes (NUMA nodes), as shown in fig. 2B, and the host may include a NUMA node 1 and a NUMA node 2, taking two NUMA nodes as an example. Under a NUMA architecture, the host's multiple physical CPUs and multiple memories are subordinate to different NUMA nodes. Each NUMA node includes at least one physical CPU and at least one physical memory, and fig. 2B exemplifies that the NUMA node includes one physical CPU and one physical memory. Within NUMA nodes, the physical CPUs communicate with the physical memory using an integrated memory controller Bus (Integrated Memory Controller Bus, IMC Bus), while NUMA nodes communicate using a fast path interconnect (Quick Path Interconnect, QPI). Because QPI has a higher latency than IMC Bus, the host has a far and near (remote/local) access to memory by the physical CPU. The physical CPU has a faster physical memory access speed to the node, and has a slower physical memory access speed to other NUMA nodes.

In a NUMA architecture scenario, the memory of this embodiment may include any of the physical memories described above. Optionally, any physical memory in the NUMA architecture may also use a reserved memory architecture. Based on this, the storage space managed in this embodiment may also refer to a reserved storage space in any physical memory in the NUMA architecture.

It can be understood that in practical application, the computer device may also adopt other architectures, and according to actual needs, the memory referred to in this embodiment may have various implementation manners according to actual application scenarios, which are not listed here.

The method of the embodiment can be applied to a memory management module of an operating system of any computer equipment. The reserved memory scenario described above may be applied to a reserved memory management module of an operating system. The method can continuously run after the computer equipment is started, and the decision of memory exchange is continuously carried out; the operation can also be started when needed, for example, the operation can be performed after the start instruction is received, and the start instruction can be initiated by a technician, can be automatically initiated according to a preset rule, and the like.

The process in this embodiment may include any process running in a computer device, such as the aforementioned virtual machine, and the like.

The memory swap in this embodiment may be understood as swapping data from main memory to secondary storage, and may include a variety of swap methods, such as memory merging (or memory block merging), memory compression (or memory block data compression), and swapping data with external memory. As an example, the memory merging refers to that after the memory merging module of the operating system is started, the memory block to be released in a cold state is taken out and is initially taken as a candidate memory block; and then, comparing the retrieved memory blocks to be released with data stored in each candidate memory block to determine whether there are combinable memory blocks until the combinable candidate memory blocks are found or the traversal of all the candidate memory blocks is finished. If the combinable candidate memory block is found, the virtual address of the memory block to be released is mapped to the physical address of the found candidate memory block, so that the physical address of the memory block to be released is released. The memory compression refers to compressing data of an original memory block by a compression algorithm and storing the compressed data into a smaller memory block, thereby releasing the original memory block. Exchanging data with the external memory means that the data of the memory block is stored in the external memory such as a disk, so that the memory block is released, and then taken out from the external memory when needed, and stored in the memory block for the process.

The memory exchange in this embodiment includes memory swap-out and memory swap-in, where memory swap-out refers to releasing a memory block allocated to a process and recovering the memory block by a memory management module of an operating system; memory swap-in refers to storing the data of the memory block to the memory block and giving the data to the process when swapping out.

The storage space to be regulated in the embodiment can be flexibly configured according to actual requirements; for example, in implementation, a process may be a one-time normalization object, for example, a plurality of virtual machines are run in a computer device, and once normalization is performed for each virtual machine in normalization, and a storage space to be normalized may be a storage space allocated to the process. In other examples, the memory is globally ordered, and the storage space to be ordered may be all or part of the storage space in the memory.

The use state of the memory block in this embodiment may include swapped out and not swapped out from the viewpoint of whether to swap out or not; from the point of view of whether it is assigned to a process, it may include idle and assigned. Fig. 2C is a schematic diagram illustrating a usage state of a plurality of memory blocks in a storage space of a memory according to an embodiment of the present disclosure. In the figure, each square represents a memory block, and the number of the squares is 1 to 16 from left to right, and the memory space belongs to a certain process, so that the use state adopts swapped-out and non-swapped-out examples, the square filled with grid lines represents the non-swapped-out memory block, and the non-filled square represents the swapped-out memory block. As can be seen from fig. 2C, the swapped out memory blocks of the process are distributed in a scattered manner, such as the 12 th and 16 th positions in the figure, and there are 3 positions before the 4 th position, and there are two positions between the 4 th and 8 th positions, if there are positions which can be suitably swapped out (cold pages are suitable for being swapped out), the 12 th and 16 th positions can be swapped in again; for example, the 6 th bit and the 7 th bit are swapped out, the 12 th bit and the 16 th bit are swapped in to the 6 th bit and the 7 th bit, and then the 4 th bit to the 10 th bit can form continuous swapped out positions, namely, a continuous large-section swapped out memory block is formed, and the large-section swapped out memory block can be recycled by the memory management module to allocate continuous large-section memory.

The same applies to the case where the usage status is idle and allocated, the grid lines are used to fill the squares in fig. 2C to represent allocated memory blocks, the unfilled squares represent idle memory blocks, and the 12 th and 16 th, i.e. scattered idle memory blocks in the figure, can find suitable memory blocks from other allocated memories, for example, the 12 th bit and 16 th bit are replaced with the 6 th bit and 7 th bit, so that the 4 th bit to 10 th bit can form continuous idle space.

Based on this, the embodiment relates to two normalization processing modes, and the actual implementation may be a normalization function under the memory management module, where the normalization function may perform two types of normalization processing at the same time; or, the memory management module may be two normalization functions, and when in implementation, both the two normalization functions may be started, or any one of the two normalization functions may be started, that is, any one of the normalization processes may be executed as needed. The present embodiment may be implemented by using a process as a primary regularization object, for example, running multiple virtual machines in a computer device, where the primary regularization is performed for each virtual machine. In other examples, from a memory global perspective, it is also optional to normalize all or part of the storage space in the memory.

For step 202, the usage status of each memory block in the memory and the location in the memory are obtained, which can be obtained by the metadata of the memory. In practical applications, the memory management module creates one or more metadata during the process of managing the memory, so as to facilitate management of the memory, for example, metadata for recording allocation information of each allocation, metadata for recording cold and hot states of the memory block, and so on. As an example, each time the reserved memory management module allocates a memory space to a process, a dedicated metadata mmap is created correspondingly, where the dedicated metadata is a dedicated structure for managing memory information allocated to the process, information about the physical address range allocated this time and the process is recorded, for example, identification information about the process, a virtual address range of the process, and so on, and all the dedicated metadata mmap is placed in a linked list for maintaining mmaps, where each mmap uses a physical address as a unique identifier and corresponds to a process identifier pid one by one. In this embodiment, the usage status of each memory block may be obtained by traversing the linked list of mmaps; and determining the position of the memory block in the memory according to the address of the memory block. Alternatively, other ways of traversing each memory block in the storage space to be regulated may be adopted.

The regular target object comprises scattered swapped-out memory block sets, and the corresponding regular source object comprises a cold block set adjacent to the continuous swapped-out space; based on this, the present embodiment can swap out scattered swapped out memory blocks into the memory, and then swap out the cold blocks adjacent to the consecutive swapped out space. The embodiment can find out scattered swapped-out memory block sets and continuous swapped-out space adjacent cold block sets which meet the conditions by scanning the use state of each memory block in the memory. The set of scattered swapped out memory blocks may include one or more scattered swapped out memory blocks, and the set of cold blocks adjacent to the continuous swapped out space may include one or more cold blocks adjacent to the continuous swapped out space. And the number of the continuous swapped out memory blocks in the scattered swapped out memory block set is smaller than a first threshold value. That is, for a sporadic swapped out memory block, the number of swapped out memory blocks consecutive to it is less than the first threshold minus 1. In the continuous swapped-out space, the number of the continuous swapped-out memory blocks is larger than or equal to a first threshold value; in practical applications, the first threshold may be flexibly configured according to needs, for example, may be a positive integer such as 2, 3 or 5, which is not limited in this embodiment.

Similarly, the regular target object comprises a scattered free memory block set, and the corresponding regular source object comprises a cold block set adjacent to the continuous free space; the scattered free memory block set may include one or more scattered free memory blocks, and the cold block set adjacent to the continuous free space may include one or more cold blocks. And in the scattered idle memory block set, the number of continuous idle memory blocks is smaller than a second threshold value. That is, for a scattered free memory block, the number of free memory blocks consecutive to it is less than the first threshold minus 1. In the continuous idle space, the number of the continuous idle memory blocks is larger than or equal to a second threshold value; in practical applications, the second threshold may be flexibly configured according to needs, for example, may be a positive integer such as 2, 3 or 5, which is not limited in this embodiment.

Because the memory blocks included in the memory are more, in order to reduce occupation of memory space and improve efficiency in determining the regular target object and the corresponding regular source object, in some examples, the regular target object and the corresponding regular source object are determined according to the use state of each memory block in the memory and the position in the memory;

Generating an unset bitmap according to the number of memory blocks in the memory;

acquiring metadata of the memory, determining corresponding bits of the memory blocks in the bitmap according to the positions of the memory blocks in the memory, which are recorded in the metadata, and setting the corresponding bits of the memory blocks in the bitmap according to the use states of the memory blocks, which are recorded in the metadata;

and determining a regular source object and a corresponding regular target object according to each bit of the bitmap.

A Bitmap (Bitmap) records and indexes a chart (map) of states of a certain object in terms of bits (bits), that is, each bit (bit) is used to store a certain state of each object (for example, if 0 and 1 are represented respectively, two states of the same object may be represented). The set bit and clear bit are two operations of the bitmap. A set indicates that the bit is set to 1 and unset (clear) indicates that the bit is set to 0.

In this embodiment, an unset bitmap is generated according to the number of memory blocks in the memory, that is, the number of bits in the bitmap is consistent with the number of memory blocks. The memory range to be regulated and the size of the memory blocks can be determined according to the ratio of the memory range to be regulated and the memory blocks, namely the number of the memory blocks, and if the memory range to be regulated is a memory space range allocated to a process if the memory range to be regulated is executed once according to the process.

And each bit in the unset bitmap is 0, the corresponding bit in the bitmap of the memory block is determined according to the position of the memory block recorded in the metadata in the memory, and the corresponding bit of the memory block in the bitmap is set according to the use state of the memory block recorded in the metadata. For example, the bitmap corresponding to the embodiment shown in fig. 2C may be:

0001 1001 1101 0001; where 1 indicates a swap out and 0 indicates no swap out.

In practical applications, a bitmap may be generated for the usage status of whether to swap out, and the scattered swapped out memory blocks and the cold blocks adjacent to the continuous swapped out space are determined by the bitmap. Generating a bitmap for whether the space is free, and determining scattered free memory blocks and cold blocks adjacent to the continuous free space through the bitmap. The usage state of each memory block and the position in the memory are obtained by using metadata, and are recorded in a bitmap mode, so that the occupation of the memory space can be reduced, and the regular target object and the regular source object can be determined with higher efficiency through the bitmap.

In some examples, the bitmap includes a swap bitmap indicating that a usage state of a memory block is a swap situation, where whether a bit in the swap bitmap is set indicates whether a corresponding memory block is a swapped memory block;

The regular target object is determined by:

generating scattered changed-out bitmaps according to the exchange bitmaps; the bit number of the scattered bit-out bitmap is the same as the bit number of the exchange bitmap, and the bit set in the scattered bit-out bitmap corresponds to the bit with the bit number continuously set in the exchange bitmap smaller than a first threshold value;

and determining the scattered swapped-out memory block set according to the set bit in the scattered swapped-out bitmap.

And/or the number of the groups of groups,

the bitmap comprises an idle bitmap which indicates that the use state of the memory block is idle, and bits set in the idle bitmap indicate that the corresponding memory block is the idle memory block;

the regular target object is determined by:

generating scattered idle bitmaps according to the idle bitmaps; the bits of the scattered free bitmap are the same as the bits of the free bitmap, and the bits set in the scattered free bitmap correspond to bits with bits continuously set in the free bitmap, wherein the bits are smaller than a first threshold;

and determining a scattered free memory block set according to the set bit in the scattered free bitmap.

As an example, the exchange Bitmap1:0001 1001 1101 0001, where a set (i.e., bit 1) indicates that a swap has been made, and a 0 indicates that no swap has been made; taking the first threshold as 2 as an example, bits 12 and 16 are consecutive only 1, which is less than the threshold. The bits of the scattered changed Bitmap2 are the same as the Bitmap1, the bits 1 of the corresponding positions (also the 12 th bit and the 16 th bit) of the Bitmap2 are set to be 0; that is, the positions of the set bits in Bitmap2 are the same as those of Bitmap1 in which the number of bits of sequentially set 1 is smaller than the first threshold. Based on the scattered Bitmap2 is: 0000 0000 0001 0001. By this step, it can be found that the set of 1 in Bitmap2 is the set of candidate memory regular objects, including the candidate memory regular objects, i.e. the scattered memory blocks are swapped out. If each bit in Bitmap2 is 0, no break has been swapped out of the memory block. As can be seen from the above embodiments, the scattered swapped out bitmap can be quickly generated by swapping the bitmap, so that the scattered swapped out memory blocks can be quickly analyzed.

Similarly, the free Bitmap1:0001 1001 1101 0001 where a set (i.e. bit 1) indicates idle and a 0 indicates swapped out; taking the first threshold as 2 as an example, bits 12 and 16 are consecutive only 1, which is less than the threshold. The bits of the scattered free Bitmap2 are the same as that of Bitmap1, the bits 1 of the corresponding positions (also the 12 th bit and the 16 th bit) of Bitmap2 are set to be 0, and the other bits are set to be 0; that is, the positions of the set bits in Bitmap2 are the same as those of Bitmap1 in which the number of bits of sequentially set 1 is smaller than the first threshold. Based on the scattered free Bitmap2 is: 0000 0000 0001 0001. By this step, the scattered free memory blocks can be found, for example, the set 1 in Bitmap2 is the set of candidate memory regular objects, including the candidate memory regular objects, i.e. the scattered free memory blocks. If each bit in Bitmap2 is 0, there is no scattered free memory block. As can be seen from the above embodiments, the scattered free bitmap can be quickly generated by the free bitmap, so that the scattered free memory blocks can be quickly analyzed.

In some examples, the regular source object may be determined by:

generating a continuous swapped-out bitmap by using the swap bitmap and the scattered swapped-out bitmap, wherein a set bit in the continuous swapped-out bitmap represents the continuous swapped-out space;

inquiring whether a corresponding memory block belongs to a cold block or not according to unset bits in the exchange bitmap, and generating a first cold block bitmap according to an inquiry result; the set bit in the first cold block bitmap indicates that the corresponding memory block is a cold block;

determining a set of cold blocks adjacent to the continuous swapped out space according to the continuous swapped out bitmap and the first cold block bitmap;

and/or the number of the groups of groups,

generating a continuous idle bitmap by using the idle bitmap and the scattered idle bitmap, wherein a set bit in the continuous idle bitmap represents the continuous idle space;

inquiring whether the corresponding memory block belongs to a cold block or not according to the unset bit in the idle bitmap, and generating a second cold block bitmap according to an inquiry result; the set bit in the second cold block bitmap indicates that the corresponding memory block is a cold block;

and determining a cold block set adjacent to the continuous idle space according to the continuous idle bitmap and the second cold block bitmap.

In this embodiment, a continuous swapped Bitmap3 may be generated according to the swapped Bitmap1 and the scattered swapped Bitmap 2; the location of 1 in Bitmap3 refers to consecutive swapped out locations in Bitmap1 for subsequent locating of cold blocks adjacent to those consecutive swapped out locations through Bitmap3 for swapping out.

There are various ways to generate Bitmap3, in some examples, it may be that a bit of 1 is set in Bitmap1, and a bit of 0 is set in Bitmap2 to generate Bitmap3, i.e. a bit of 1 is set in Bitmap3, a bit of 1 is set in Bitmap1 and a bit of 0 is set in Bitmap2, and other positions 0.

To achieve more efficient processing; in some examples, the generating the successive swapped out bitmaps using the swap bitmap and the scattered swapped out bitmap is implemented by a calculation of two bitmaps, including: and after inverting each bit in the scattered swapped-out bitmap, performing logical AND operation with the swapped bitmap to obtain a continuous swapped-out bitmap.

For example, the Bitmap2 may be inverted and logically and operated with the Bitmap1 to obtain the Bitmap3, which is taken as an example:

the Bitmap1 is: 0001 1001 1101 0001

The Bitmap2 is: 0000 0000 0001 0001, after inverting, is: 1111 1111 1110 1110;

The Bitmap3 is: 0001 1001 1100 0000.

It can be seen from the Bitmap3 that if the memory block is to be swapped out subsequently, whether the corresponding memory block is a cold block can be determined from the adjacent position of the 1 position in the Bitmap3, and if the memory block is a cold block, the memory block can be swapped out, so that the swapped-out position is adjacent to the continuous swapped-out position.

Further, for Bitmap1:0001 1001 1101 0001, for memory blocks in the 0 position that are not swapped out, it is possible to query whether the memory blocks are cold blocks from the cold page set. Assuming that the 1 st, 7 th and 15 th bits which are not swapped are all cold blocks, the first cold block Bitmap4 is: 1000 0010 0000 0010. Thus, the cold block can be found by the first cold block Bitmap 4.

Thus, by consecutively swapping out the Bitmap3 and the first cold block Bitmap4, it is possible to determine the consecutive swapped out spatially adjacent cold blocks. For example, the 7 th bit adjacent to the 8 th to 10 th bits in the Bitmap3 is set (cold block) in the Bitmap4, so that the 7 th bit is consecutively swapped out of the spatially adjacent cold block.

As can be seen from the above embodiments, by exchanging the bitmap and the scattered swapped-out bitmap, the continuous swapped-out bitmap can be quickly generated, thereby quickly analyzing the continuous swapped-out space. By exchanging unset bits in the bitmap, it is possible to query whether the corresponding memory block belongs to a cold block, and the first cold block bitmap can be quickly generated, so that cold blocks adjacent to the continuously swapped-out space can be quickly analyzed by continuously swapping out the bitmap and the first cold block bitmap.

Similarly, a continuous idle bitmap can be generated according to the idle bitmap and the scattered idle bitmap; the position of 1 in the continuous free bitmap refers to the continuous swapped-out position in the free bitmap, which is used for finding out the cold blocks adjacent to the continuous free spaces through the continuous free bitmap in the subsequent process so as to replace the scattered free memory blocks. Further, for the unset bits in the free bitmap, whether the corresponding memory block belongs to a cold block is queried, and a second cold block bitmap can be rapidly generated according to the query result, wherein the set bits in the second cold block bitmap indicate that the corresponding memory block is a cold block, so that the cold blocks adjacent to the continuous free space are rapidly analyzed.

In practical application, there may be a plurality of bits set in the scattered swapped-out bitmap, i.e. there are a plurality of scattered swapped-out memory blocks; similarly, there may be multiple corresponding regular source objects; in some examples, to obtain a plurality of regular source objects, a first candidate regular source object bitmap may be generated according to the positions of the set bits in the continuous swapped-out bitmap and the positions of the set bits in the first cold block bitmap; the bit set in the first candidate regular source object bitmap represents: one or more cold blocks in spatial continuity with the succession of swapped out blocks; and determining a set of cold blocks adjacent to the continuous swapped-out space according to the first candidate regular source object bitmap.

In this embodiment, the obtaining of the plurality of candidate regular source objects may be achieved by generating the first candidate regular source object bitmap from the continuously swapped bitmap and the first cold block bitmap, and by using one first candidate regular source object bitmap. For example:

bitmap1:0001 1001 1101 0001 it is indicated whether each memory block has been swapped out

Bitmap2:0000 0000 0001 0001 the scattered memory blocks are replaced

Bitmap3:0001 1001 1100 0000 the memory space has been swapped out continuously

Bitmap4:1110 0000 0000 0010 cold block

From the above Bitmap, bitmap3 includes two consecutive swapped-out spaces: 4 th to 5 th bits, and 8 th to 10 th bits; combining the 1 st to 3 rd cold blocks in Bitmap4, it can be known that the 3 rd memory block is a regular source object. If there are a plurality of scattered memory blocks, the 3 rd bit and one of the scattered memory blocks are replaced, and then the 3 rd bit to the 5 th bit form a continuous swapped-out space; bit 2 may also be a regular source object. In practical applications, the scheme of the embodiment may be executed multiple times, and only one memory block is found out at a time as a regular source object. In order to improve efficiency, the present embodiment may further find a plurality of memory blocks at a time as a regular source object, for example, by generating a first candidate regular source object bitmap, where bits set in the bitmap represent: one or more cold blocks spatially contiguous with the contiguous swapped out space. For example, the 3 rd bit is a cold block, and since the 3 rd bit is adjacent to the cold block at the 2 nd bit and the 2 nd bit is adjacent to one cold block at the 1 st bit, 3 consecutive cold blocks from the 1 st bit to the 3 rd bit can be spatially consecutive with the adjacent consecutive swapped-out space, so that 3 candidate regular source objects can be found.

In practical application, the first candidate regular source object bitmap can be obtained through analyzing the continuously swapped-out bitmap and the first cold block bitmap. As one example:

copying a Bitmap5 by taking the continuously shifted Bitmap3 as a reference, looking up the 1-setting jump position (namely, the Bitmap5 is set with 0 and the bit with 1 is changed) of the Bitmap, determining whether the bit is set in the first cold block Bitmap4, and if so, setting the bit in the Bitmap5 to be 1; and generating a Bitmap6 (each bit in the initial Bitmap6 is zero), wherein the position set to 1 is the newly set 1 position in the Bitmap5, and the other positions are set to 0. As an example:

Bitmap2:0000 0000 0001 0001 the scattered memory blocks are replaced

Bitmap3:0001 1001 1100 0000 the memory space has been swapped out continuously

Bitmap4:1110 0000 0000 0010 cold block

Bitmap5:0001 1001 1100 0000 it is obtained by duplication of Bitmap3

Bitmap6:0000 0000 0000 0000 the bits in the initial Bitmap6 are all zero

Firstly, searching a 1 st jump position in Bitmap5, for example, finding a 3 rd bit; since bit 3 is set in Bitmap4, bitmap5, bit 3, position 1, is updated as:

Bitmap5:0011 1001 1100 0000 because of the 3 rd position 1 of the Bitmap5, the 3 rd position of the Bitmap6 is correspondingly set to 1; bitmap6 is updated as:

bitmap6:0010 0000 0000 0000 a candidate regular source object is now found.

Since each bit is not traversed, query again, query after update of Bitmap5 whether bit 2 is set in Bitmap4, and similarly execute the same process:

bitmap5, position 2, 1, updated as: 0111 1001 1100 0000

Bitmap6 is also updated accordingly: 0110 0000 0000 0000

Again, query after update of Bitmap5, query if bit 2 is set at Bitmap4, and execute the same process:

bitmap5, 1 st position 1:1111 1001 1100 0000

Bitmap6:1110 0000 0000 0000, finishing the query to obtain a first candidate regular source object Bitmap6, wherein the set candidate regular source object sets, wherein each set 1 source memory block can be regular.

In some examples, the determining a set of cold blocks adjacent to the continuous free space according to the continuous free space bitmap and the second cold block bitmap may include:

generating a second candidate regular source object bitmap according to the positions of all set bits in the continuous idle bitmap and the positions of all set bits in the second cold block bitmap; the bit set in the second candidate regular source object bitmap represents: one or more cold blocks contiguous with the contiguous free space;

And determining a set of cold blocks adjacent to the continuous free space according to the second candidate regular source object bitmap.

The method and the device are similar to the method for obtaining the first candidate regular source object bitmap, and the embodiment can generate a second candidate regular source object bitmap through the continuous idle bitmap and the second cold block bitmap, and obtain a plurality of candidate regular source objects through one second candidate regular source object bitmap. Specific implementation may refer to the foregoing embodiments, and this embodiment is not described herein.

In some examples, the replacing the usage state of the regular target object with the usage state of the corresponding regular source object may include:

the data stored in the cold blocks in the cold block set adjacent to the continuous swapped-out space are swapped out, so that the cold blocks of the swapped-out data are changed into swapped-out memory blocks; the data corresponding to the scattered swapped-out memory blocks in the scattered swapped-out memory block set are swapped in, so that the scattered swapped-out memory blocks are changed into non-swapped-out memory blocks; and/or the number of the groups of groups,

and replacing the data stored in the cold blocks in the cold block set adjacent to the continuous idle space to scattered idle memory blocks in the scattered idle memory block set, so that the replaced cold blocks are changed to idle memory blocks, and the scattered idle memory blocks are changed to allocated memory blocks.

For the regularity of scattered swapped-out memory blocks, the embodiment may acquire a cold block adjacent to the continuous swapped-out space, swap out the stored data, change the use state of the swapped-out cold block to swapped-out, i.e. swapped-out memory blocks, where the swapped-out memory blocks are recovered by the memory management module and can be changed to idle memory blocks. After the memory block is swapped out, a scattered free memory block is taken out and swapped in; the memory exchange function of the memory management module generally sets the memory exchange water level (i.e. the memory exchange proportion), so that the memory exchange function is not affected by the memory regular processing, and scattered memory blocks are regular, so that a continuous exchange space can be formed. In practical application, the scattered swapped-out memory block set has a plurality of memory blocks, and the number of cold block sets adjacent to the continuous swapped-out space may also be plural, and the step of this embodiment may be to take out one memory block from the cold block set adjacent to the continuous swapped-out space each time when implementing, and after processing the swapped-out, take out one memory block from the scattered swapped-out memory block set for processing the swapped-in memory block.

For the regularity of the scattered free memory blocks, the embodiment does not need to involve data exchange, and the data stored by the cold blocks adjacent to the continuous free space can be replaced to the scattered free memory blocks, so that the cold blocks adjacent to the continuous free space are changed to the free memory blocks, and the scattered free memory blocks are changed to the allocated memory blocks, therefore, the replacement of the two can be completed rapidly, the scattered free memory blocks are regular, and the continuous free space can be formed.

The following examples are provided to illustrate the invention.

In the first embodiment, the host reserves the memory regularity of the exchanged memory of the memory;

taking a reserved memory scene as an example, the regular aim is to replace the memory blocks in the situation that the exchanged memory blocks of the process are scattered, find the cold blocks in the non-exchanged memory blocks of the process, and select the cold blocks which can be adjacent to the replaced memory blocks as far as possible from the cold blocks to replace the cold blocks. The method can comprise the following steps:

1. traversing metadata mmap, and obtaining the swap-out condition of the memory blocks by taking the ms granularity of each memory block as a bit of a Bitmap according to the memory allocation information recorded in the mmap; a swap Bitmap1 is generated for the memory swap-out case, where the swapped out is set 1 and there is no swapped out set 0.

Taking a reserved memory scene as an example, when a virtual machine starts to run, a reserved memory management module allocates a continuous memory space (comprising one or more memory blocks) for the virtual machine from a reserved memory space managed by the reserved memory management module, metadata mmap representing the memory space allocated at this time is established, the mmap uses a physical address as a unique identifier, information of the physical address range allocated at this time and information of a process, such as identification information of the process, a virtual address range of the process and the like, all the proprietary metadata mmap are placed into a linked list of maintenance mmap, and each mmap uses the physical address as a unique identifier and corresponds to the process identifier pid one by one.

Subsequently, memory swap processing may occur for these memory blocks allocated to the virtual machine, and in this embodiment, the swap-in and swap-out situations of each memory block may be determined by scanning metadata that records memory swap information.

By way of example, FIG. 2C illustrates a schematic diagram of a segment of memory allocated to a process, each square representing a block of memory; with the running of the process, the memory exchange of the section of memory space occurs; the square with 1 in it indicates that the memory block is swapped out. The adoption of the exchange Bitmap1 can be expressed as: 0001 1001 1101 0001; where 1 indicates a swap out and 0 indicates no swap out.

From Bitmap1:0001 1001 1101 0001 it can be seen that bits 12 and 16 belong to scattered swap-out positions, i.e. scattered swapped out memory blocks; the 3 memory blocks which are not swapped out are arranged before the 4 th bit, two memory blocks which are not swapped out are arranged between the 4 th bit and the 8 th bit, and if the positions which can be suitable for swapping out (the cold blocks are suitable for swapping out) are arranged between the 4 th bit and the 8 th bit, the 12 th bit and the 16 th bit of the memory blocks which are swapped out can be swapped in again; for example, the 6 th bit and the 7 th bit are swapped out, and the 12 th bit and the 16 th bit are swapped in to the 6 th bit and the 7 th bit, so that the 4 th bit to the 10 th bit can form a continuous swapped-out space, namely a continuous large section of idle memory block is formed.

2. According to a preset threshold (for example, the threshold is set to be 2, etc.), continuously setting positions with the number of 1 bits smaller than the threshold, namely, candidate memory regular objects, namely, candidate replaced objects; based on this, a scattered swapped out Bitmap2 is generated, which is based on bits in Bitmap1 that are less than the threshold being set to 1, otherwise being set to 0.

That is, the memory block corresponding to the bit 1 in Bitmap2 is the candidate memory management object (the candidate needs to be replaced with the memory block), which is taken as an example:

bitmap1:0001 1001 1101 0001, bits 12 and 16, only 1 in succession, are less than the threshold, thus setting 1 and the other bits 0; the Bitmap2 is: 0000 0000 0001 0001.

Therefore, through the step, the scattered swapped memory blocks can be found out, so that the candidate memory management object is obtained.

3. Generating a continuously shifted Bitmap3 according to the Bitmap1 and the Bitmap 2; the location of 1 in Bitmap3 refers to consecutive swapped out spaces in Bitmap1 for subsequent locating of cold blocks adjacent to those consecutive swapped out spaces through Bitmap3 for swapping out.

The Bitmap3 may be generated continuously by various ways, for example, the Bitmap2 is inverted and then logically and operated with the Bitmap1 to obtain the Bitmap3, which is taken as an example:

Bitmap1：0001 1001 1101 0001

Bitmap2:0000 0000 0001 0001, after inverting, is: 1111 1111 1110 1110

Bitmap3：0001 1001 1100 0000。

In other examples, it may be other ways, such as placing 1 in the Bitmap1, and placing 0 in the Bitmap2 generates the Bitmap3, that is, the position 1 in the Bitmap3 is the position 1 in the Bitmap1 and 0 in the Bitmap2, and other positions are 0.

4. And inquiring all unremoved memory blocks (namely 0 part in the Bitmap 1) in the hot and cold memory sets to generate a first cold block Bitmap4, wherein the 1 in the Bitmap4 represents the cold blocks, and the other memory blocks, whether the hot blocks or the memory blocks are replaced, are all set to 0. For example:

for Bitmap1:0001 1001 1101 0001, for the memory block in the 0 position which is not swapped, inquiring whether the memory block is a cold block from the cold page set. Assuming that the 1 st, 7 th and 15 th bits which are not swapped are all cold blocks, bitmap4 is: 1000 0010 0000 0010. Therefore, the cold block is found out through the Bitmap4, and the memory block which can be swapped out is obtained.

5. If the Bitmap3 is not empty, jumping to the step 7; if Bitmap3 is empty, the normalization can be ended; alternatively, a plurality of (if any) scattered swapped-out memory blocks in Bitmap2 may be ordered, so that they are combined into a continuous position.

For example, since Bitmap3 is empty, i.e., each bit is zero, it indicates that there is no contiguous swapped out space; thus, traversing Bitmap2, selecting the bit position 1 in which bit is centered 1, and adjacent bits (i.e., back and forth) have bit position 1 set in Bitmap4, while clearing that bit from Bitmap2 (i.e., clearing the bit of bit 1).

As example (1):

bitmap1:0001 0001 0001 0001 1 it indicates that the current 4 swapped out memory blocks are all scattered;

bitmap2:0001 0001 0001 0001 1 the fragmented memory blocks that have been swapped out are suitable for swap-in;

bitmap3:0000 0000 0000 0000 are all zero, indicating that there are no consecutive swapped out memory blocks

Bitmap4:0000 0000 0000 0010 1 cold block;

because bit 15 in Bitmap4 is adjacent to bit 16 of Bitmap2, bit 16 of Bitmap2 is cleared, and Bitmap2 is updated as: 0001 0001 0001 0000.

Because of the update of Bitmap2, the update of Bitmap3 is: 0000 0000 0000 0001.

As example (2):

Bitmap4:0000 0000 0100 0000 1 cold block;

because the 10 th position 1 is in the Bitmap4, the Bitmap2 does not have the 1 st position adjacent to the 10 th position, so that the Bitmap2 does not need to be updated, the Bitmap3 is still zero, and the step 6 is executed.

6. Bitmap3 remains zero, indicating that there is no regulatable Bitmap currently, returning directly. If the adjusted Bitmap3 is not zero, but the Bitmap2 is zero, the Bitmap3 and the Bitmap2 are randomly allocated to each half.

As an example:

bitmap2:0000 0000 0000 0000 1 the scattered areas which have been swapped out, are suitable for swapping in; at this time, no 1 is present, and all are zero.

Bitmap3:0000 0100 0000 1000 1 indicates that successive swapped out areas can be tried to find a cold block from its neighboring locations to swap out.

At this time, 1 is not placed in the Bitmap2 and is moved into the Bitmap3, which means that 1 in the Bitmap3 has adjacent cold blocks, and 1 is scattered. If only one 1 exists in the Bitmap3, the rule is not valid, and the return is direct. If 1 in the Bitmap3 is greater than 1, i 1 can be randomly taken out for Bitmap2, and the value of i is obtained by dividing the value of 1 in the Bitmap3 by 2, and then rounding up or rounding down.

For example, in the Bitmap3, the 6 th position and the 13 th position are 1, 1 position 0 is randomly selected, for example, the 6 th position 0, and the 6 th position 1 of the Bitmap2 is updated as follows:

Bitmap2：0000 0100 0000 0000

Bitmap3：0000 0000 0000 1000。

7. copying a Bitmap5 by taking Bitmap3 as a reference, looking up the 1-setting jump position (namely, the Bitmap5 is set with 0 and the bit with 1 changed) and determining whether the bit is set in the first cold block Bitmap4, and if so, setting the bit in the Bitmap5 by 1; and generating a Bitmap6 (each bit in the initial Bitmap6 is zero), wherein the position set to 1 is the newly set 1 position in the Bitmap5, and the other positions are set to 0. For example:

Bitmap2:0000 0000 0001 0001 the scattered memory blocks are replaced

Bitmap3:0001 1001 1100 0000 the memory space has been swapped out continuously

Bitmap4:1110 0000 0000 0010 cold block

Bitmap5:0001 1001 1100 0000 is replicated from Bitmap 3;

bitmap6:0000 0000 0000 0000 the bits in the initial Bitmap6 are all zero

Bitmap6:0010 0000 0000 0000 a candidate regular source object is now found.

bitmap5, position 2, 1, updated as: 0111 1001 1100 0000

Bitmap6 is also updated accordingly: 0110 0000 0000 0000

bitmap5, 1 st position 1:1111 1001 1100 0000

Bitmap6:1110 0000 0000 0000, the query is ended, and the first candidate regular source object Bitmap6 is obtained, wherein the set candidate regular source object set is obtained.

8. At this time, bitmap6 is a candidate regular object source set (i.e. memory blocks suitable for swap-out, which are all cold blocks and can be continuous with the swapped-out space in the replacement process), and Bitmap2 is a candidate regular object set (i.e. scattered swapped-out memory blocks, which need to be swapped in);

in this embodiment, the number of the objects in each of the two sets may be calculated and the minimum num may be taken, if num is 0, it indicates that there is no memory block suitable for swap out of Bitmap6 or there is no memory block suitable for swap in of Bitmap2, so that the user directly exits; otherwise, the next step is continued.

9. Selecting a bit position a which is set from the Bitmap6 and is adjacent to the Bitmap 3; if not found, finishing the memory regulation and returning; otherwise, continuing.

10. Querying again whether a is still in the cold page set, if so, continuing; otherwise, returning to the step 9, searching for the next memory regular position.

Since the cold page set may be changed from the step 4 to the step, the original cold block may be accessed by the process just in the process and changed from the cold block to the hot block, so that the re-outputting position a inquires whether the corresponding memory block is the cold block or not, and the hot block can be prevented from being changed out to affect the operation of the process.

11. Switching the memory block ms1 corresponding to location a to a secondary storage location (there are many ways, here a disk is an example);

a) Formally starting one memory block swap-out, swapping the granularity of one memory block ms (for example, 2 MB), and selecting one memory space ds (device section) from the secondary storage;

b) Selecting a memory block ms1 to be swapped out, wherein the memory block ms1 is physical address information paddr (physical memory address) when being acquired from a cold set and a hot set, and inquiring a corresponding virtual address vaddr (virtual memory address) according to metadata mmap of recorded memory allocation information;

c) Setting a page table entry corresponding to a virtual address vaddr to be in a read-only state, and avoiding inconsistent data caused by the fact that the content stored in a memory block is changed in the process of swapping out;

d) Calling a writing function of the secondary storage, and writing the content in the virtual address vaddr (namely the data stored in the pad) into the position of the secondary storage ds;

e) Waiting for writing, if the writing is wrong or the writing process is cancelled, the read-only state of the page table entry corresponding to the vaddr needs to be restored to the read-write state;

f) If the swap-out is successful, updating a page table entry corresponding to the vaddr, and recording the position ds information of the page table entry in the secondary storage; meanwhile, if the persistent memory can be directly read and written through the page table (for example, the mode of merging the persistent memory into other memory blocks is adopted), the page table is changed into writable; otherwise, the flag bit present of the page table needs to be set, for example, the present bit is cleared, so that the recovery is convenient when the subsequent process accesses;

g) Acquiring a protection lock of memory allocation information mmap, updating the memory allocation information mmap, and recording a secondary storage and a position ds thereof into the mmap;

h) Releasing the memory block ms1;

12. the bit position a is set in the Bitmap3 after successful replacement, and is cleared from the Bitmap 6.

Through the steps, the position a in the Bitmap6 can be swapped out; optionally, in this embodiment, in order to ensure stability of the swap-out water level (i.e. the ratio of the number of swapped out memory blocks to the number of memory blocks of the process), after swapping out one memory block, a suitable position is found to swap in one memory block.

13. And selecting a set position b from the Bitmap2, attempting to allocate a memory block ms2 connected with the physical address of the front or rear memory block, and optionally allocating the memory block ms2 if the continuous requirement cannot be met.

14. The content is swapped into a newly allocated memory block ms2 from the secondary storage position;

a) Establishing a temporary mapping relation (used by a kernel and not a relation in a process page table) from a physical address paddr of ms2 to a kernel-state virtual address vaddr 1;

b) Acquiring a secondary storage position ds of a swap-out memory block according to swap-out position information recorded by mmap;

c) And calling a reading function of the secondary storage, reading data stored in the ds position of the secondary storage, and writing the data into the memory by using the kernel-state virtual address vaddr 1.

d) Waiting for the read to complete, or making an error;

e) Canceling temporary mapping of the vaddr1 and the paddr, if the read error or the write process is canceled, the replacement fails, and the memory block ms2 is released;

f) If the replacement is successful, a page table corresponding to the process vaddr needs to be updated, and the association with the paddr is established;

g) Acquiring a memory allocation information protection lock, updating memory allocation information mmap, and recording the paddr to a position corresponding to vaddr in the mmap;

h) Releasing the secondary storage location ds;

15. and if the replacement is successful, resetting the bit position b in the Bitmap 2.

16. And (5) if the loop num times are finished, executing the step 9 if not, otherwise, finishing and returning.

2. Discontinuous memory management embodiment of reserved memory of host

The regular object in this embodiment aims at the idle memory blocks, i.e. the memory blocks which are managed by the reserved memory management module and are not allocated to the process, and if the memory blocks are scattered in the memory space managed by the reserved memory management module, the fragments in the memory blocks need to be regular, so as to group the fragments together. The regular mode is to exchange the data stored in the allocated memory blocks to the scattered free memory blocks, so that the scattered free memory blocks are replaced by the allocated memory blocks and are connected with other allocated memory blocks.

For example, in the following bitmap: 0001 1001 1101 0001, for example, a 1 indicates a free memory block and a 0 indicates an allocated memory block; it can be seen that the two idle memory blocks of the 12 th bit and the 16 th bit are scattered, and if the idle memory blocks can be replaced by the 6 th bit and the 7 th bit, the 3 rd bit to the 10 th bit can form continuous large-section idle space. Based on this, the method of the present embodiment may include the steps of:

1. Traversing all the memory blocks ms, wherein the granularity of each memory block ms is used as one bit of an idle Bitmap1, and acquiring the idle state of the memory block; and generating an idle Bitmap1 according to the memory idle condition, wherein a memory block 1 existing in the main memory (the memory block is positioned in the main memory and represents an unallocated idle memory block), and a non-existing memory block 0 (which represents non-idle and is already allocated to a process) are generated.

2. According to a preset threshold (for example, the threshold is set to 2), the length of a physical address continuous region of 1 in Bitmap1 is smaller than the threshold, which is the target object of memory regulation, and according to the memory region of Bitmap1 smaller than the threshold, 1 is set, otherwise, 0 is set, and a scattered free Bitmap2 is generated.

That is, the 1 position in Bitmap2 is the candidate regular target object (the candidate free memory block to be replaced), as an example:

Therefore, through the step, scattered free memory blocks can be found out, so that a candidate regular target object is obtained.

3. Generating a continuous idle Bitmap3 according to the Bitmap1 and the Bitmap 2; the location of 1 in Bitmap3 refers to the continuous idle memory blocks in Bitmap1, which are used for finding out the cold blocks of the continuous idle memory blocks through Bitmap3 later so as to replace the cold blocks with the idle memory blocks to be replaced.

There are various ways to generate the Bitmap3, for example, the Bitmap2 is inverted and then logically and operated with the Bitmap1 to obtain the Bitmap3, which is taken as an example:

Bitmap1：0001 1001 1101 0001

bitmap2:0000 0000 0001 0001, after inverting, is: 1111 1111 1110 1110

Bitmap3：0001 1001 1100 0000。

In other examples, it may be that a 1 in Bitmap1 is located, and a 0 in Bitmap2 is located to generate Bitmap3, i.e. a 1 in Bitmap3 is located in 1 in Bitmap1 and a 0 in Bitmap2, and other positions are 0.

It can be seen from the Bitmap3 that if the memory block is to be replaced subsequently, it can be determined whether the corresponding memory block is a cold block from the adjacent position of the 1 position in the Bitmap3, and if the memory block is a cold block, the memory block can be replaced so that the replaced position is adjacent to the continuous idle memory block.

4. For all allocated memory blocks (0 part in Bitmap 1), inquiring in a hot and cold memory set to generate a second cold block Bitmap4, wherein the cold blocks are set to 1, and the other memory blocks, whether hot page memory blocks or memory blocks are replaced, are set to 0;

5. if the Bitmap3 is not empty, jumping to the step 7; if Bitmap3 is empty, the normalization can be ended; alternatively, a plurality of (if any) scattered free memory blocks in Bitmap2 may be ordered to be combined into a continuous position.

6. If the Bitmap3 is still zero, indicating that no memory block which can be regulated exists currently, directly returning; if the adjusted Bitmap3 is not zero, but the Bitmap2 is zero, the Bitmap3 and the Bitmap2 each account for half randomly.

7. Copying a Bitmap5 by taking Bitmap3 as a reference, looking up the 1-setting jump position (namely, the bits of the Bitmap5 with 0 and 1 changed) of the Bitmap, determining whether the bit is set in a second cold block Bitmap4, and setting 1 in the Bitmap5 if the bit is set; and generating a Bitmap6 (each bit in the initial Bitmap6 is zero), wherein the position set to 1 is the newly set 1 position in the Bitmap5, and the other positions are set to 0.

8. At this time, bitmap6 is a candidate regular object source set (i.e. allocated memory blocks suitable for replacement, which are all cold blocks and can be continuous with the free space in the replacement process), bitmap2 is a candidate regular object set (i.e. scattered free memory blocks), calculates the respective number and takes the minimum num, and if num is 0, directly exits; otherwise, continuing.

10. And selecting a set position b from the Bitmap2, if the condition cannot be met, continuing to select the next candidate position in step 9, otherwise, continuing.

11. Querying again whether a is still in the cold page set, if so, continuing; otherwise, returning to the step 9, searching for the next memory regular position.

12. Replacing the data of the memory block ms1 corresponding to the position a to the memory block ms2 corresponding to the position b;

a) Formally starting the regularity of once memory blocks, and selecting the allocated continuous address memory blocks ms2 from the main memory;

b) Selecting memory blocks ms1 to be regulated, wherein the memory blocks ms1 are physical address information paddr when being acquired from a cold and hot set, and converting the physical address information paddr into a virtual address vaddr according to memory allocation information mmap;

c) The page table of the virtual machine vaddr is changed into read-only, so that the data inconsistency caused by the change of the content of the memory in the regular process is avoided;

d) Calling a writing function of the main storage, and writing the content in the virtual address vaddr into a target memory position ms2;

e) Waiting for writing, if the writing is error or the writing process is cancelled, canceling the read-only state of the page table corresponding to the vaddr, and changing the read-only state into the read-write state again;

f) If the regulation is successful, the page table corresponding to the vaddr needs to be updated, and if the page table is updated, the position ms2 information of the page table in the main memory is recorded into a page table item;

g) Acquiring a memory allocation information protection lock, updating memory allocation information mmap corresponding to the original ms1, and recording a main memory and a position ms2 thereof into the mmap;

h) The memory block ms1 is released.

13. The bit position a is set in the Bitmap3 after successful normalization, and is cleared from the Bitmap 6.

14. And (3) resetting the bit position b in the Bitmap2 when the data stored in the position b in the Bitmap2 has become the data stored in the original ms1.

15. And (5) whether the loop num times are finished, if not, returning to the step 9, otherwise, finishing.

Corresponding to the foregoing embodiments of the memory management method, the present disclosure further provides embodiments of the memory management device and a computer apparatus to which the memory management device is applied.

The embodiments of the memory regulating device in the present specification may be applied to a computer device, for example, a server or a terminal device. The apparatus embodiments may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking software implementation as an example, the device in a logic sense is formed by reading corresponding computer program instructions in a nonvolatile memory into a memory by a processor where the device is located. In terms of hardware, as shown in fig. 3, a hardware structure diagram of a computer device where the memory regulating device is located in the present specification is shown in fig. 3, and in addition to the processor 310, the memory 330, the network interface 320, and the nonvolatile memory 340 shown in fig. 3, the computer device where the memory regulating device 331 is located in the embodiment generally may include other hardware according to the actual function of the computer device, which is not described herein again.

As shown in fig. 4, fig. 4 is a block diagram of a memory regulating apparatus according to an exemplary embodiment of the present specification, the apparatus comprising:

a determining module 41 for: determining a regular target object and a corresponding regular source object according to the use state of each memory block in a storage space to be regular and the position of each memory block in the storage space to be regular; the regular target object comprises scattered swapped-out memory block sets, and the corresponding regular source object comprises a cold block set adjacent to the continuous swapped-out space; and/or the regular target object comprises a scattered free memory block set, and the corresponding regular source object comprises a cold block set adjacent to the continuous free space; the number of the continuous swapped out memory blocks in the scattered swapped out memory block set is smaller than a first threshold value; in the continuous swapped-out space, the number of the continuous swapped-out memory blocks is larger than or equal to a first threshold value; the number of continuous idle memory blocks in the scattered idle memory block set is smaller than a second threshold value; in the continuous idle space, the number of the continuous idle memory blocks is larger than or equal to a second threshold value;

a substitution module 42 for: and replacing the use state of the regular target object with the use state of the corresponding regular source object.

In some examples, the determining module 41 is further configured to:

generating an unset bitmap according to the number of memory blocks in the storage space to be regulated;

acquiring metadata of the storage space to be regulated, determining corresponding bits of the memory blocks in the bitmap according to the positions of the memory blocks in the memory recorded in the metadata, and setting the corresponding bits of the memory blocks in the bitmap according to the use states of the memory blocks recorded in the metadata;

In some examples, the bitmap includes a swap bitmap indicating that a usage status of a memory block is a swap situation, where a bit set in the swap bitmap indicates whether a corresponding memory block is a swapped memory block bit set indicates whether the corresponding memory block is a swapped memory block;

the regular target object is determined by:

generating scattered changed-out bitmaps according to the exchange bitmaps; wherein, bits set in the scattered changed bitmap correspond to bits with bits number smaller than a first threshold value, which are continuously set in the changed bitmap;

Determining the scattered swapped-out memory block set according to the set bit in the scattered swapped-out bitmap;

and/or the number of the groups of groups,

the regular target object is determined by:

generating scattered idle bitmaps according to the idle bitmaps; wherein, the bit set in the scattered free bitmap corresponds to the bit with the bit number of the bit continuously set in the free bitmap smaller than the second threshold value;

and determining the scattered free memory block set according to the set bit in the scattered free bitmap.

In some examples, the regular source object is determined by:

Determining a set of cold blocks adjacent to the continuous swapped-out space according to the continuous swapped-out bitmap and the first cold block bitmap;

and/or the number of the groups of groups,

and determining the adjacent cold block set of the continuous idle space according to the continuous idle bitmap and the second cold block bitmap.

In some examples, the determining module 41 is further configured to:

comprising the following steps:

after inverting each bit in the scattered swapped-out bitmap, performing logical AND operation with the swapped bitmap to obtain a continuous swapped-out bitmap;

and/or the number of the groups of groups,

generating a continuous idle bitmap by using the idle bitmap and the scattered idle bitmap, including:

and after inverting each bit in the scattered idle bitmap, performing logical AND operation with the idle bitmap to obtain a continuous idle bitmap.

In some examples, the determining module 41 is further configured to:

generating a first candidate regular source object bitmap according to the positions of all set bits in the continuous swapped-out bitmap and the positions of all set bits in the first cold block bitmap; the bit set in the first candidate regular source object bitmap represents: one or more cold blocks in spatial continuity with the succession of swapped out blocks;

determining a set of cold blocks adjacent to the continuous swapped-out space according to the first candidate regular source object bitmap;

and/or the number of the groups of groups,

In some examples, the permutation module is further to:

The implementation process of the functions and actions of each module in the memory regulating device is specifically detailed in the implementation process of corresponding steps in the memory regulating method, and is not repeated here.

Accordingly, the present embodiments also provide a computer program product comprising a computer program which, when executed by a processor, implements the steps of the foregoing memory management method embodiments.

Accordingly, the embodiments of the present disclosure also provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the memory-normalization method embodiment when the program is executed by the processor.

Accordingly, the present description also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the memory management method embodiments.

For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present description. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

The above-described embodiments may be applied to one or more computer devices, which are devices capable of automatically performing numerical calculations and/or information processing according to preset or stored instructions, the hardware of which include, but are not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.

The computer device may be any electronic product that can interact with a user in a human-computer manner, such as a personal computer, tablet computer, smart phone, personal digital assistant (Personal Digital Assistant, PDA), game console, interactive internet protocol television (Internet Protocol Television, IPTV), smart wearable device, etc.

The computer device may also include a network device and/or a user device. Wherein the network device includes, but is not limited to, a single network server, a server group composed of a plurality of network servers, or a Cloud based Cloud Computing (Cloud Computing) composed of a large number of hosts or network servers.

The network in which the computer device is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), and the like.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The above steps of the methods are divided, for clarity of description, and may be combined into one step or split into multiple steps when implemented, so long as they include the same logic relationship, and they are all within the protection scope of this patent; it is within the scope of this application to add insignificant modifications to the algorithm or flow or introduce insignificant designs, but not to alter the core design of its algorithm and flow.

Where a description of "a specific example", or "some examples", etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present description. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Other embodiments of the present description will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations of the specification following, in general, the principles of the specification and including such departures from the present disclosure as come within known or customary practice within the art to which the specification pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the specification being indicated by the following claims.

It is to be understood that the present description is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

The foregoing description of the preferred embodiments is provided for the purpose of illustration only, and is not intended to limit the scope of the disclosure, since any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the disclosure are intended to be included within the scope of the disclosure.

Claims

1. A memory management method, the method comprising:

2. The method according to claim 1, wherein determining the regular target object and the corresponding regular source object according to the usage status of each memory block in the storage space to be regular and the location in the memory includes:

3. The method of claim 2, wherein the bitmap includes a swap bitmap indicating that a usage status of a memory block is a swap situation, and whether a bit in the swap bitmap is set indicates whether a corresponding memory block is a swapped memory block;

the regular target object is determined by:

and/or the number of the groups of groups,

the regular target object is determined by:

4. A method according to claim 3, the regular source object being determined by:

generating a continuous swapped-out bitmap by using the swap bitmap and the scattered swapped-out bitmap, wherein a set bit in the continuous swapped-out bitmap represents a continuous swapped-out space;

and/or the number of the groups of groups,

generating a continuous idle bitmap by using the idle bitmap and the scattered idle bitmap, wherein a set bit in the continuous idle bitmap represents a continuous idle space;

5. The method of claim 4, the generating a continuous swapped out bitmap using the swap bitmap and the scattered swapped out bitmap, comprising:

And/or the number of the groups of groups,

6. The method of claim 4, the obtaining the set of consecutive swapped out spatially adjacent cold blocks from the consecutive swapped out bitmap and the first cold block bitmap, comprising:

acquiring a cold block set adjacent to the continuous swapped-out space according to the first candidate regular source object bitmap;

and/or the number of the groups of groups,

acquiring a cold block set adjacent to the continuous idle space according to the continuous idle bitmap and the second cold block bitmap, including:

And acquiring a cold block set adjacent to the continuous idle space according to the second candidate regular source object bitmap.

7. The method of any one of claims 1 to 6, the permuting the usage status of the regular target object with the usage status of the corresponding regular source object, comprising:

8. A memory management apparatus, the apparatus comprising:

9. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any of claims 1 to 7.

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed by the processor.