CN115756838A - Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium - Google Patents

Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium Download PDF

Info

Publication number
CN115756838A
CN115756838A CN202211406834.3A CN202211406834A CN115756838A CN 115756838 A CN115756838 A CN 115756838A CN 202211406834 A CN202211406834 A CN 202211406834A CN 115756838 A CN115756838 A CN 115756838A
Authority
CN
China
Prior art keywords
memory
memory block
block
released
zero
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211406834.3A
Other languages
Chinese (zh)
Inventor
郑豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Alibaba Cloud Computing Ltd
Original Assignee
Alibaba China Co Ltd
Alibaba Cloud Computing Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd, Alibaba Cloud Computing Ltd filed Critical Alibaba China Co Ltd
Priority to CN202211406834.3A priority Critical patent/CN115756838A/en
Publication of CN115756838A publication Critical patent/CN115756838A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present specification provides a method, an apparatus, a computer device and a storage medium for releasing and recovering a memory, including: determining a memory block to be released; determining whether the memory block to be released is a zero block; if the zero block is the zero block, accessing preset zero block data; if the preset zero block data is not written with information, determining not to release the memory block to be released, updating the memory block to be released as a read-only permission, taking the memory block to be released as a reference zero block, writing a physical address of the memory block to be released in the preset zero block data, creating metadata corresponding to the virtual address of the memory block to be released, and associating the metadata with the preset zero block data; if the physical address of a reference zero block is recorded in preset zero block data, mapping a virtual address corresponding to the memory block to be released to the physical address, creating metadata corresponding to the virtual address and associating the metadata with the preset zero block data, and then releasing the memory block to be released; wherein the metadata includes preset flag information.

Description

Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for releasing and restoring a memory, a computer device, and a storage medium.
Background
The operating system allocates a certain number of memory blocks to each running process, and creates corresponding page table data for each process, which is used to record the mapping relationship between each virtual address in the virtual storage space used by the process and the physical address of the memory block.
In order to flexibly use the memory, for each memory block allocated to each process, the memory management module of the operating system may also record the cold and hot state information of the memory blocks, so as to solve the problem of the shortage of memory space through a memory merging scheme.
Specifically, after a memory merging module of the operating system is started, a cold-state memory block to be released is taken out, and is initially taken as a candidate memory block; and then, comparing the memory blocks to be released, which are taken out again, with the data stored in each candidate memory block, and determining whether there are combinable memory blocks or not until the combinable candidate memory blocks are found or the traversal of all the candidate memory blocks is finished. If the mergeable candidate memory blocks are found, the virtual addresses of the memory blocks to be released are mapped to the physical addresses of the found candidate memory blocks, so that the physical addresses of the memory blocks to be released are released.
In the above scheme, each time a memory block to be released is processed, the data of the memory block to be released needs to be compared with the data of the candidate memory block, and the memory release speed is slow.
In the recovery scheme, when a memory block needs to be recovered to a certain virtual address, a candidate memory block mapped by the virtual address needs to be accessed, data of the candidate memory block is copied to a newly allocated memory block, and the virtual address is mapped to the newly allocated memory block; therefore, it is also desirable to increase the speed of memory recovery.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a memory release method, a memory recovery method, a memory release apparatus, a memory recovery apparatus, a computer device, and a storage medium.
According to a first aspect of embodiments herein, there is provided a memory release method, the method including:
responding to a memory release request, and determining a memory block to be released;
determining whether the memory block to be released is a zero block, wherein each bit of the memory block represented by the zero block is zero;
if the memory block to be released is a zero block, accessing preset zero block data;
if the preset zero block data is not written with information, determining not to release the memory block to be released, updating the memory block to be released as a read-only permission, taking the memory block to be released as a reference zero block, writing a physical address of the memory block to be released in the preset zero block data, creating metadata corresponding to the virtual address of the memory block to be released, and associating the metadata with the preset zero block data;
if the preset zero block data records the physical address of a reference zero block, mapping the virtual address corresponding to the memory block to be released to the physical address, creating metadata corresponding to the virtual address and associating the metadata with the preset zero block data, and then releasing the memory block to be released; wherein the metadata includes preset flag information.
According to a second aspect of embodiments herein, there is provided a memory recovery method, including:
responding to a memory recovery request, and acquiring metadata corresponding to a virtual address according to the virtual address carried by the memory recovery request;
allocating memory blocks to be written from the memory;
if the metadata comprises preset mark information, accessing preset zero block data;
if the residual reference zero block is determined through the metadata related to the preset zero block data, modifying the read-only permission of the reference zero block into the read-write permission, releasing the memory block to be written, emptying the preset zero block data and deleting the metadata;
and if at least two zero blocks are determined by the metadata associated with the preset zero block data, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
According to a third aspect of embodiments herein, there is provided a memory releasing apparatus, the apparatus including:
a request module to: responding to a memory release request, and determining a memory block to be released;
a zero block determination module to: determining whether the memory block to be released is a zero block, wherein each bit of the memory block characterized by the zero block is zero;
an access module to: if the memory block to be released is a zero block, accessing preset zero block data;
a release handling module to: if the preset zero block data is not written with information, determining not to release the memory block to be released, updating the memory block to be released as a read-only permission, taking the memory block to be released as a reference zero block, writing a physical address of the memory block to be released in the preset zero block data, creating metadata corresponding to the virtual address of the memory block to be released, and associating the metadata with the preset zero block data; if the preset zero block data records the physical address of a reference zero block, mapping the virtual address corresponding to the memory block to be released to the physical address, creating metadata corresponding to the virtual address and associating the metadata with the preset zero block data, and then releasing the memory block to be released; wherein the metadata includes preset flag information.
According to a fourth aspect of embodiments herein, there is provided a memory recovery apparatus, the apparatus including:
a request module to: responding to a memory recovery request, and acquiring metadata corresponding to a virtual address according to the virtual address carried by the memory recovery request;
an assignment module to: allocating memory blocks to be written from the memory;
an access module to: if the metadata comprises preset mark information, accessing preset zero block data;
a recovery processing module to: if the residual reference zero block is determined through the metadata associated with the preset zero block data, modifying the read-only permission of the reference zero block into the read-write permission, releasing the memory block to be written, emptying the preset zero block data and deleting the metadata; and if at least two zero blocks are determined by the metadata associated with the preset zero block data, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
According to a fifth aspect of embodiments herein, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method embodiments of the first or second aspect when executing the computer program.
According to a sixth aspect of embodiments herein, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method embodiments of the first or second aspect.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
in the embodiment of the disclosure, for the memory release scheme, the individual management and maintenance are performed for the zero block; determining whether the memory blocks to be released are zero blocks or not after the memory blocks to be released are determined according to the memory release request, recording the information of the zero blocks through preset zero block data, and determining whether the memory blocks to be released have combinable zero blocks or not through the preset zero block data; therefore, when the memory is released, the candidate memory blocks do not need to be traversed for the zero block, and the data stored in the other memory blocks do not need to be compared one by one, so that the memory release speed is increased. Similarly, when the memory is restored, the zero block is determined to be restored through the preset mark information, the data does not need to be copied in the restoring operation, and the speed of restoring the memory is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the specification.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present specification and together with the description, serve to explain the principles of the specification.
FIG. 1A is a schematic diagram of page table data shown in accordance with an exemplary embodiment of the present specification.
Fig. 1B is a schematic diagram illustrating a memory merge according to an exemplary embodiment of the present disclosure.
Fig. 2A is a flowchart illustrating a memory release method according to an exemplary embodiment of the disclosure.
Fig. 2B is a schematic diagram of a memory shown in this specification according to an example embodiment.
FIG. 2C is a schematic diagram of a NUMA architecture shown herein in accordance with an exemplary embodiment.
Fig. 2D-2G are each a schematic diagram of a tree shown in this specification according to an exemplary embodiment.
Fig. 2H is a flowchart illustrating a memory recovery method according to an exemplary embodiment of the present disclosure.
Fig. 3 is a block diagram of a computer device in which a memory releasing/restoring apparatus is located according to an exemplary embodiment of the present disclosure.
Fig. 4 is a block diagram illustrating a memory release device according to an exemplary embodiment of the present disclosure.
Fig. 5 is a block diagram of a memory recovery device according to an exemplary embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with this description. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the specification, as detailed in the claims that follow.
The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present specification. The word "if," as used herein, may be interpreted as "at \8230; \8230when" or "when 8230; \823030when" or "in response to a determination," depending on the context.
When a process runs, an operating system allocates a virtual address space and a physical address space for the process, and creates a page table corresponding to the process, wherein the page table is used for recording the mapping relation between the virtual address space and the physical address space. The operating system also maintains metadata for the managed memory for managing the memory. When the memory is exchanged, because the storage position of the data of the process is changed, the page table of the process needs to be updated; since a change in the storage location of data also changes the state of the memory block, the metadata also needs to be updated.
Page tables are a concept of virtual memory technology. Operating systems also use virtual memory technology in order to allow programs to gain more available memory and expand physical memory into larger logical memory. The physical memory is abstracted into an address space, an operating system allocates an independent set of virtual addresses to each process, and the virtual addresses of different processes are mapped with the physical addresses of different memories. If the program is to access a virtual address, it is translated to a different physical address by the operating system. This involves the concept of two addresses:
the Memory Address used by the program is called a Virtual Address (VA);
the spatial Address actually present in the hardware is called a Physical Memory Address (PA).
The virtual address is mapped to the physical address by a page table. The page table is stored in a Memory, and a Memory Management Unit (MMU) of the CPU converts the virtual Memory into a physical Memory. When the virtual address to be accessed by the process cannot be checked in the page table, the system generates a page missing exception, enters a system kernel space to allocate a physical memory, updates the page table of the process, and finally returns to the user space to recover the operation of the process.
Referring to fig. 1A, a schematic diagram of page table data according to an exemplary embodiment is shown in the present specification, in the related art, a memory management unit of an operating system divides a memory according to a set management granularity, where each management granularity may be referred to as a page (page) or may be referred to as a block. In this embodiment, taking the pages allocated by the process as 0 to N as an example, the page table includes N page table entries, and each page table entry is used to represent a corresponding relationship between a virtual address and a physical address of each page. Thus, the whole page table records the virtual address space of the process, the relationship between the page table and the physical address space, and a certain virtual address of the process can be mapped to a corresponding physical address through the page table.
In order to vacate memory space and improve resource utilization rate, memory pages (generally called cold pages) which are not frequently used in the memory can be processed by a memory combination method, so that the use amount of the memory is reduced. As shown in fig. 1B, a schematic diagram of memory merging and memory recovery is shown; a virtual address VA1 of a process 1 is mapped to a physical address PA1 (memory block 1), a virtual address VA2 of a process 2 is mapped to a physical address PA2 (memory block 2), and a virtual address VA3 of the process 1 is mapped to a physical address PA3 (memory block 3), because data stored in the three memory blocks are the same, the three memory blocks may be merged, a physical memory may only retain one memory block (such as the memory block 1 shown in the figure), and the three virtual addresses are all mapped to a physical address of the memory block 1; and, the read-write permission of the memory block 1 is read-only, that is, the access of the process to the three virtual addresses is read-only operation. If the process initiates a write operation to one of the virtual addresses, the operating system needs to restore the memory block to the virtual address for mapping according to the write operation. For example, when the process 1 initiates a write operation to the virtual address VA1, the operating system needs to allocate a new memory block (with a physical address PA 4), copy the data of the memory block 1 to the new memory block, and then map VA1 to the PA4, thereby implementing memory recovery, and the process 1 can implement the write operation to the virtual address VA1 and the physical address PA 4. From the above analysis, it can be known that, when the memory merge or the memory recovery occurs, the page table entry information of the memory block needs to be updated.
In addition, the operating system also maintains metadata of the memory block, the type of the metadata is various, and according to different scenarios, the metadata may include, but is not limited to, metadata indicating whether the memory block is allocated, total metadata of the memory or metadata indicating a cold state of the memory block, metadata indicating a process to which the memory block belongs, metadata indicating a memory allocation situation of the process to which the memory block belongs, and the like. Taking a virtual machine scenario as an example, in some schemes, a computer device is dedicated to a virtual machine, in a memory allocation scheme for the virtual machine, a fixed virtual address space is configured for each virtual machine, in order to facilitate querying a corresponding relationship between a virtual address and a physical address, on the basis of page table data, a metadata mmap (memory map, a mapping relationship between a virtual address and a physical address of a memory) for representing a memory allocation condition is additionally created, and by using the metadata, bidirectional querying of the virtual address and the physical address can be realized, so that querying efficiency can be improved. When the memory is merged, the various metadata of the memory block also needs to be updated.
In the related art, each time a memory block to be released is processed, data of the memory block to be released needs to be compared with data of the candidate memory block, and the memory release speed is slow. In the recovery scheme, when a memory block needs to be recovered to a virtual address, a candidate memory block mapped by the virtual address needs to be accessed, data of the candidate memory block is copied to a newly allocated memory block, and the virtual address is mapped to the newly allocated memory block; therefore, it is also desirable to increase the speed of memory recovery
Fig. 2A is a schematic diagram of a memory release method according to an exemplary embodiment in the present specification, and includes the following steps:
in step 200, in response to the memory release request, determining a memory block to be released;
in step 202, it is determined whether the memory block to be released is a zero block, where each bit of the zero block characterization memory block is zero;
in step 204, if the memory block to be released is a zero block, accessing preset zero block data;
in step 206, if the preset zero block data is not written with information, it is determined that the memory block to be released is not released, the memory block to be released is updated to be read-only permission, the memory block to be released is taken as a reference zero block, a physical address of the memory block to be released is written into the preset zero block data, and metadata corresponding to a virtual address of the memory block to be released is created and associated with the preset zero block data;
in step 208, if the preset zero block data has recorded the physical address of the reference zero block, mapping the virtual address corresponding to the memory block to be released to the physical address, creating metadata corresponding to the virtual address and associating the metadata with the preset zero block data, and then releasing the memory block to be released; wherein the metadata includes preset flag information.
The method of the embodiment can be applied to an operating system of any computer device, and can be used for releasing the memory blocks allocated in the memory.
In some examples, the computer device may employ a conventional memory management architecture, i.e., the entire memory is managed by the operating system. In other scenarios, for example, in a virtual machine scenario, a computer device may employ a memory allocation architecture for reserving a memory, as shown in fig. 2B, which is a schematic diagram of a reserved memory scenario exemplarily shown herein, in which a memory of a host includes two storage spaces, as shown in fig. 2B, two storage spaces of the memory are shown in different filling manners, including a non-reserved storage space a for a kernel (in the drawing, diagonal filling is used), and a reserved storage space B for a virtual machine (in the drawing, vertical line and grayscale filling are used). That is, the unreserved memory space a is used for the kernel in the figure, and applications (application 1 to application 3 as examples in the figure) running on the operating system can use the unreserved memory space a. The reserved storage space b is available for a Virtual Machine (VM), such as n Virtual machines of VM1 to VMn shown in the figure. The two storage spaces may adopt different management granularities, that is, the memory may be divided differently. For ease of illustration in fig. 2B, the two storage spaces are illustrated in a sequential manner in the figure. It will be appreciated that in practice, the two storage spaces may be non-contiguous.
The reserved storage space occupies most of the memory, and is unavailable to the host kernel, and a module can be inserted into the kernel of the operating system to be specially used for managing the reserved storage space. In order to facilitate management of the series of memories, at the same time, occupation of a large amount of metadata on the memories is avoided, and considering that a minimum memory is also hundreds of MB (MByte, megabyte) when allocating a memory for a virtual machine, the reserved storage space is divided by a larger granularity, for example, the reserved storage space is divided into memory blocks (ms) of 2MB and the like for management; in some scenarios, large granularity is also commonly used, and 1GB (GigaByte) or the like is optional, which is not limited in this embodiment.
When the method is applied to a reserved memory scenario, an operating system may use different modules to manage a reserved memory space and a non-reserved memory space, for example, the method of this embodiment may be applied to a module for managing a non-reserved memory space in an operating system, and is used to process compression of a memory block in a reserved memory space.
In other examples, the computer device may be a device that includes multiple physical CPUs, and a Non Uniform Memory Access Architecture (NUMA) that includes at least two NUMA nodes (NUMA nodes) may be employed as needed, as shown in fig. 2C, and the hosts may include NUMA node 1 and NUMA node 2, taking two NUMA nodes as an example. Under the NUMA architecture, multiple physical CPUs and multiple memories of a host belong to different NUMA nodes. Each NUMA node includes at least one physical CPU and at least one physical memory, and fig. 2C exemplifies that a NUMA node includes one physical CPU and one physical memory. Inside the NUMA nodes, the physical CPU and the physical Memory communicate with each other using an Integrated Memory Controller Bus (IMC Bus), and the NUMA nodes communicate with each other using a Quick Path Interconnect (QPI). Since the delay of QPI is higher than that of IMC Bus, there is a difference (remote/local) between physical CPU accesses to memory on the host. The physical CPU has a higher speed of accessing the physical memory of the node, and the physical CPU has a lower speed of accessing the physical memory of other NUMA nodes.
In a NUMA architecture scenario, the memory of this embodiment may include any one of the above physical memories. Optionally, any physical memory in the NUMA architecture may also adopt a reserved memory architecture. Based on this, the storage space managed by this embodiment may also refer to a reserved storage space in any physical memory in the NUMA architecture.
It can be understood that in practical application, the computer device may also adopt other architectures, and according to actual needs, the memory referred to in this embodiment may have a plurality of implementation manners according to practical application scenarios, which are not listed here.
The memory release request in step 202 may be obtained in various ways. In some examples, the memory management module of the operating system may have a memory aging management function, which may manage a cold-hot change condition of each memory block in the memory, and maintain metadata indicating a cold-hot state of the memory as needed. The cold and hot states of each memory block can be determined by scanning the usage of each memory block in the memory. For example, a cold page set may record a memory chunk in a cold state, and a hot page set may record a memory chunk in a hot state. As an example, the embodiment may specifically apply a memory merging module in an operating system, where the memory merging module is started according to a set condition, for example, the memory merging module may determine that a memory release request is received when a start instruction of a user is received; the memory release request may also be periodically started, or a start instruction of another module of the operating system is received, it is determined that a memory release request is received, and the like. Next, the memory chunks that need to be released may be determined from the cold page set, which is referred to as the to-be-released memory chunks in this embodiment. For example, a batch processing scenario may be adopted, for example, multiple memory blocks may be taken out from the cold page set at a time, and each memory block may be processed in a serial manner, that is, the method of this embodiment is executed for one memory block at a time to release the memory blocks. Of course, the parallel processing is also optional, and this embodiment does not limit this.
For the memory blocks to be released, the embodiment separately manages the zero blocks during the memory merge. The zero block in this embodiment refers to a memory block in which whether each bit is zero or not. For example, in an application scenario such as a virtual machine, one or more virtual machines may run on a host device, where a virtual machine is a process for the host, but the virtual machine itself is a virtual computer device, and the memory usage rate of a computer device does not reach 100% in a normal case, that is, in a normal case, a virtual machine may leave some storage spaces for the memory allocated to the virtual machine, and each bit of the unused storage spaces is zero.
Therefore, in the embodiment, the management of memory merging is performed on the zero block alone, and compared with a method in which data stored in the memory block to be released is directly compared with data stored in each candidate memory block in the prior art, performance can be significantly improved. Specifically, in this embodiment, the preset zero block data is configured, and in some examples, after the operating system starts the memory merge, a storage space in the memory application is used for storing the preset zero block data, the data may be maintained by using a tree structure or a linked list, and during initialization, taking the tree structure as an example, only a tree of a root node may be used to indicate that no information is written in.
In the memory merging scheme, in an execution process, if a to-be-released memory block that is a zero block is acquired for the first time, this embodiment determines not to release the to-be-released memory block, updates the to-be-released memory block to a read-only permission, and records a physical address of the to-be-released memory block in the preset zero block data by using the to-be-released memory block as a reference zero block, that is, a first to-be-released zero block is written into preset zero block data as a reference zero block, so that the first to-be-released memory block can be quickly merged when the to-be-released memory block is acquired in a subsequent process. In addition, in this embodiment, metadata corresponding to the virtual address of the memory block to be released is created and associated with preset zero block data, where the metadata includes preset flag information, and the preset flag information is used to indicate that the memory block is a zero block and is used in subsequent memory recovery.
The preset zero block data may be used to link to the metadata of each zero block, or the preset zero block data may be a tree structure including a root node, so that the zero block data is used to organize the metadata of each zero block, and the node may link the metadata of each zero block through a linked list. In practical application, other implementation manners may be adopted according to needs, and this embodiment does not limit this.
In order to increase the speed of determining whether each bit of a memory block is zero, in some examples, the determining whether the memory block to be released is a zero block includes:
pre-judging whether each bit of the memory block to be released is zero or not;
if the bit position which is not zero exists in the memory block to be released is judged in advance, determining that the memory block to be released is not a zero block;
if it is judged that all the bits of the memory block to be released are zero, after the read-write permission of the memory block to be released is changed into read-only, whether the memory block to be released is a zero block is judged.
In this embodiment, it is considered that a certain time is consumed in the process of determining that each bit of the memory block is zero, and the memory block may be updated in the determining process; based on this, in this embodiment, whether each byte of the memory block to be released is zero is pre-determined, for example, the pre-determination step may be executed under the condition that the read-write permission of the memory block to be released is not changed, and if it is pre-determined that each bit of the memory block to be released is not zero, it is determined that each bit of the memory block to be released is not zero. And after pre-judging that all the bits of the memory block to be released are zero, changing the read-write permission of the memory block to be released into read-only, and then further judging whether all the bits of the memory block to be released are zero or not. On one hand, the condition that all bits of the memory block to be released are not all zero is judged, and the permission of the memory block does not need to be changed by pre-judgment, so that the expense can be reduced; if all the bit positions of the memory block to be released are judged to be zero in advance, the read permission is changed to be the read-only permission, and accurate judgment is further carried out, so that the determining efficiency is improved, and an accurate judgment result can be obtained.
The pre-determination aims to determine that all bits of the memory block to be released are not all zero, and in practical applications, the pre-determination may be implemented in a variety of ways, and in some examples, the pre-determination of whether all bits of the memory block to be released are all zero includes:
selecting n bits from the memory block to be released, and determining whether the n bits are all zero; wherein n is a positive integer;
if not, pre-judging that the bit which is not zero exists in the memory block to be released;
if yes, pre-judging whether other bits of the memory block to be released except the n bits are all zero.
In this embodiment, in order to increase the speed of the pre-determination, part of the bits may be selected first and determined first, and as long as one of the bits is not zero, it may be pre-determined that all the bits of the memory block to be released are not zero. The selected number n may be configured flexibly as needed, for example, may be determined based on the size of the memory block and the actual overhead requirement, which is not limited in this embodiment.
In some examples, there may be a plurality of ways to select n bits from the memory block to be released, for example, any of the following ways:
selecting n bit positions from the highest bit position of the memory block to be released;
selecting n bit positions from the lowest bit position of the memory block to be released; or the like, or, alternatively,
and randomly selecting n bit positions in the memory block to be released.
Selecting n bit positions from the highest bit position of the memory block to be released or selecting n bit positions from the lowest bit position of the memory block to be released, wherein the two selection modes are quick; and the probability of making accurate pre-judgment can be improved by randomly selecting n bits in the memory block to be released.
Considering that, in the case of a block that is not a zero block, the memory block to be released needs to be further subjected to data comparison with the candidate memory block, and in order to increase the speed of subsequent comparison, in some examples, a determination may also be performed in an encoding manner, where the n bits are bits of n set positions, and the determining whether the n bits are all zeros includes:
generating current coding information of the bits of the n set positions by using a preset coding algorithm, wherein the preset coding algorithm is used for generating different coding information for different inputs;
comparing the current coding information with preset coding information, and determining whether the bit positions of the n set positions are all zero according to a comparison result; the preset coding information is information obtained by coding n bits which are all zero in advance by using the preset coding algorithm.
The n setting positions may be configured as needed, for example, n bits from the low bit to the head of the height of the memory block, or n bits from the tail of the memory block, or other setting positions, and so on. In this embodiment, the information of the n bits taken out may be encoded to obtain current encoding information of the memory block to be released, and the current encoding information is compared with preset encoding information, where the preset encoding information is obtained by encoding n zeros, and based on this, whether the current encoding information is the same as the preset encoding information is compared, it may be determined whether all the n bits taken out from the memory block to be released are zeros. In practical applications, various encoding methods may be adopted, such as various existing Hash (Hash) algorithms, including but not limited to MD5 (Message-Digest Algorithm), SHA (Secure Hash Algorithm), and so on. Alternatively, a custom encoding algorithm is also optional, as long as the encoding algorithm can generate unique encoding information for any input.
In some examples, the method further comprises:
if the memory block to be released is not a zero block, acquiring a candidate memory block set, and determining whether a target memory block storing the same data as the memory block to be released exists in the candidate memory block set;
if so, mapping the virtual address of the memory block to be released to the physical address of the target memory block, creating metadata corresponding to the virtual address, writing the physical address of the target memory block in the metadata, and releasing the memory block to be released.
In this embodiment, if it is determined that the memory block to be released is not a zero block, a candidate memory block set is used to determine whether a target memory block storing data that is the same as the data stored in the memory block to be released exists in the candidate memory block set. The candidate memory block set includes merged memory blocks or non-merged memory blocks. For example, if the memory block to be released taken out of the cold page set does not find a mergeable memory block, the memory block to be released is regarded as an uncombined memory block, and a next memory block to be released is waited to be compared with the memory block to be released. If the memory block to be released finds a memory block having the same content as the memory block to be released, the foregoing releasing operation of mapping the virtual address of the memory block to be released to the physical address of the target memory block, creating metadata corresponding to the virtual address, and writing the metadata into the physical address of the target memory block may be performed on the memory block to be released.
In some examples, to improve management efficiency, the memory chunks extracted from the cold page set may find merged memory chunks, or may temporarily fail to find a mergeable memory chunk, based on which the merged candidate memory chunks and the non-merged candidate memory chunks may be managed separately. As an example, two tree structures may be maintained, one merged and one un-merged. The memory blocks managed in the merged tree are all the memory blocks (at least two same memory blocks) which are already merged, and the attribute of the page table is changed into read-only permission; the uncombined tree is a memory block which is taken from the cold page set and can not be combined yet, and the page table attribute is still read and written. That is, the merged candidate memory block set is maintained by using the merged tree, and the non-merged candidate memory block set is maintained by using the non-merged tree.
The merged tree and the uncombined tree may be designed as needed, for example, nodes of the tree are used for associating the memory blocks, and each node in the merged tree is used for associating the memory blocks with the same content, as shown in fig. 2D, which is a schematic diagram of the merged tree shown in this specification according to an exemplary embodiment, where the tree includes 7 nodes (N1 to N7), taking node N2 and node N3 as an example, node N2 is used for associating metadata to 3 memory blocks, node N3 is used for associating metadata to 2 memory blocks, and each memory block under the same node stores the same data. As shown in fig. 2E, which is a schematic diagram of an unmerged tree, it can be understood that each node in the unmerged tree is associated with metadata of one memory block, for example, the node N14 in the graph is associated with the metadata 14 of one memory block.
Based on this, when the memory block to be released is taken out each time, whether there is a candidate memory block which can be merged or not can be found through the merging tree, and after the merged tree is traversed and not found, the candidate memory block is searched from the non-merged tree until the non-merged tree is traversed and not found. If the candidate memory blocks that can be merged are found in the merged tree, for example, the memory block that stores the same data as the memory block to be released and is found in fig. 2D is associated with the node N3, the metadata of the memory block to be released may be associated with the node N3 again, so that the node N3 is updated to associate 3 memory blocks, and after the update, as shown in fig. 2F, the node N3 is associated with more metadata 6 of the memory block to be released.
If the same target memory block is found in the uncombined tree, the node associated with the target memory block is deleted from the uncombined tree, and a new node is added in the combined tree, where the node is used to associate the target memory block with the memory block to be released, for example, the memory block associated with N14 is found in fig. 2E, the node N14 is deleted from the tree, as shown in fig. 2G, a new node N8 is added in the combined tree, and the metadata 14 of the memory block associated with the original N14 and the metadata 15 of the memory block associated with the memory block to be released are associated.
As in the foregoing embodiment of determining the zero block, similarly, in order to increase the speed of determining whether the data stored in the two memory blocks are the same, in some examples, each candidate memory block corresponds to candidate encoding information obtained by encoding information of bits at n set positions of the candidate memory block by using the preset encoding algorithm;
whether the candidate memory block stores the same data as the memory block to be released is determined by the following method:
comparing the candidate coding information of the candidate memory block with the current coding information of the memory block to be released;
if the candidate memory block and the to-be-released memory block are different, determining that the candidate memory block and the to-be-released memory block store different data;
if the read-write permission of the memory block to be released and the read-write permission of the candidate memory block are the same, comparing whether the data stored in the candidate memory block is the same as the data stored in the memory block to be released.
In this embodiment, the data stored in the two memory blocks are the same, that is, the information of each bit of the two memory blocks is the same. Considering that a certain time is consumed for comparing information stored in each bit of the two memory blocks, and the memory blocks may be updated in the determining process; based on this, in this embodiment, the coding information of a part of bits is taken first to perform pre-determination, for example, the pre-determination step may be executed under the condition that the read-write permission of the two memory blocks is not changed, and if the pre-determination is different, it may be determined that the two memory blocks are different; if the two are the same, further judgment is carried out. The embodiment can reduce the overhead, improve the processing efficiency and obtain an accurate judgment result.
Based on the design of the candidate coding information, in some examples, in order to improve the search efficiency, each node in the merge tree records a physical address of one merged candidate memory block and candidate coding information; each node in the non-merged tree records a physical address and candidate encoding information of one non-merged candidate memory block. That is, the nodes in the tree store candidate encoding information of the memory blocks, and since data comparison between the memory blocks to be released and the memory blocks managed by the nodes in the merged tree or the uncombined tree is required to be performed each time the memory blocks to be released are taken out, the comparison efficiency can be significantly improved through the candidate encoding information stored in the nodes.
For the merged tree for the merged memory blocks, since each merged memory block is read-only, the stored data of the merged memory block is not changed. The memory block associated with the node in the unmerged tree is still in the read-write permission although the memory block belongs to the cold block, so that if the memory block is updated, the corresponding candidate encoding information may be updated. In some examples, the uncombined tree may be updated, so that when a memory block is searched from the uncombined tree, data comparison can be accurately performed, and accuracy of memory combination is ensured. Exemplarily, the method of this embodiment further includes: according to a set period, updating the non-merged tree as follows:
determining whether a node in the non-merged tree needs to be deleted; wherein, whether deletion is needed or not comprises: according to the physical address of the non-merged candidate memory block recorded by the node, accessing the non-merged candidate memory block and taking out the bit positions of the n set positions; generating coding information of the bits of the n set positions, and judging whether the generated coding information is the same as the candidate coding information; determining that deletion is required under different conditions; and after the nodes which need to be deleted are deleted, reorganizing the non-merged tree according to the rest nodes.
As an example, the present embodiment may initialize a root node of the unmerged tree, so as to destroy an organization manner of each node in the unmerged tree, and reorganize each node in the unmerged tree. When organizing, aiming at the metadata of the memory block associated with each node, accessing the memory block to acquire the bit positions of n set positions of the memory block and generate coding information, and judging whether the generated coding information is the same as the candidate coding information or not; determining that deletion is required under different conditions; and after the nodes which need to be deleted are deleted, reorganizing the non-merged tree according to the rest nodes.
Fig. 2H is a flowchart of a memory recovery method according to an exemplary embodiment shown in this specification, and includes the following steps:
in step 210, in response to the memory recovery request, according to the virtual address carried in the memory recovery request, metadata corresponding to the virtual address is obtained.
In step 212, the memory block to be written is allocated from the memory.
In step 214, if the metadata includes preset flag information, accessing a preset zero block data.
In step 216, if the remaining reference zero block is determined by the metadata associated with the preset zero block data, the read-only permission of the reference zero block is modified to be the read-write permission, the memory block to be written is released, the preset zero block data is emptied, and the metadata is deleted.
In step 218, if at least two zero blocks are determined according to the metadata associated with the preset zero block data, the virtual address is mapped to the physical address of the memory block to be written, and the metadata is deleted.
In the memory recovery based on memory consolidation of this embodiment, that is, the memory block originally allocated to the process is released when the memory is consolidated, and the process can only read the memory block but cannot write data of the memory block. When a process needs to write to a memory block, the operating system needs to reallocate the memory block to the process. It will be appreciated that the physical addresses of the recovered memory chunks and the memory chunks originally allocated to the process are not necessarily the same.
The memory recovery request may be initiated when the operating system finds that the process needs to write to the merged memory block. Alternatively, the memory recovery may be performed actively by the operating system.
The memory recovery request carries a virtual address, and in some examples, corresponding metadata may be obtained according to the virtual address and the virtual address carried by the memory recovery request. Whether the memory block needing to be recovered is a zero block can be determined by whether the metadata always comprises preset marking information. And if the residual reference zero block is determined through the metadata associated with the preset zero block data, modifying the read-only permission of the reference zero block in the preset zero block data into the read-write permission, releasing the memory block to be written, emptying the preset zero block data and deleting the metadata. And if at least two zero blocks are determined by the metadata related to the preset zero block data, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata. It can be seen from the above embodiments that, during memory recovery, for recovery of zero blocks, data does not need to be copied in the recovery operation, and the speed of memory recovery is increased.
In some examples, the method further comprises: if the metadata does not include preset flag information, accessing the target memory block according to the physical address of the target memory block mapped by the virtual address recorded in the metadata, writing the data of the target memory block into the memory block to be written, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
The following examples are provided to illustrate the present invention.
(1) Take memory merge as an example
1. And acquiring a memory block ms (memory section) to be released.
For example, a memory block ms to be released may be selected from the cold page set; if not, it fails, otherwise it continues. For example, a batch processing scenario may also be adopted, where a plurality of memory blocks to be released are acquired at a time, and then each memory block is serially processed.
2. Inquiring memory allocation data mmap to which the memory block ms to be released belongs, conveniently converting a physical address (physical address) of the memory block into a virtual address (virtual address) in the follow-up process, and then obtaining page table entry information pmd and the like through the virtual address.
Optionally, mmaps of all processes are in one piece of data (e.g., a linked list, etc.), so that when querying mmaps through ms, locks need to be held, that is, the linked list needs to be temporarily locked, and then mmaps corresponding to ms are obtained through traversal query. Optionally, in the batch processing scenario of the foregoing embodiment, a copy may be established for the entire mmap data as needed. Specifically, for the first memory block ms to be released in batch processing, a lock is firstly held on the whole mmap during query, and the lock is released after a copy is established. And querying the corresponding mmap by using the copy for other memory blocks to be released in batch processing.
3. According to the mmap, the virtual address vaddr corresponding to the memory block ms to be released and the corresponding page table entry pmd (taking 2m granularity as an example) may be obtained, and the metadata item for managing the merge information of the memory block may be established.
Illustratively, the information recorded in the item may include: the virtual address of the user mode corresponding to the ms, the virtual address of the kernel mode corresponding to the ms, the physical address of the memory block mapped after the data stored in the ms are combined, the page table entry pmd corresponding to the ms, the preset mark information after combination, and the like, so that the subsequent memory blocks can be conveniently used after combination, and can also be found when the memory blocks are recovered.
4. Judging whether the memory block ms to be replaced is a suspicious zero page or not, and if so, jumping to the step 5; otherwise, jump to step 9.
The judgment in this step is in a pre-judgment stage, and instead of judging the whole memory block, the judgment is made by taking part of the content of the memory block.
5. Reading the whole memory block, and judging whether the memory block is a zero page or not; if yes, jumping to step 6; otherwise, jump to step 9.
The judgment in this step is the total judgment of the memory block, i.e. each bit of the memory block needs to be read. The judgment of this step is still in the pre-judgment stage, because the read-write permission of the memory block is not modified when the memory block is judged, the memory block may be updated in the judgment process.
6. And changing the read-write permission of the memory block ms into read-only, and judging whether the page is zero again. If yes, go to step 7, otherwise go to step 9.
7. Judging whether a zero memory block zms (zero memory section) exists or not through preset zero block data, and if not, establishing a zero-page node by taking the memory block as a reference zero block; recording the information of the memory block in the metadata, adding preset mark information representing a zero block in the metadata item, and skipping to the step 22; otherwise, jumping to step 8.
8. Updating the page table entry information of the memory block to be released, specifically, changing the physical address information in the page table entry pmd corresponding to the virtual address vaddr of the memory block into the physical address information corresponding to the reference zero page zms, adding preset mark information indicating a zero block to the metadata item, and going to step 22.
9. It is determined whether the memory blocks to be released can be merged through the merged tree (the step is executed to indicate that the memory block is not a zero block), the merged tree is traversed first, one memory block sms is taken out from the merged tree, and the step 10 is executed.
10. Comparing the md5 value recorded for the memory block sms with the md5 value of the memory block ms to be merged, if the md5 values are the same, jumping to step 11, otherwise jumping to step 14.
In this embodiment, md5 is generated according to n bits of the header of the memory block, and the page table attribute of the current memory block is still the read-write permission.
11. Changing the page table attribute of ms into read-only, comparing the contents of sms and ms in full quantity, if the contents are the same, jumping to step 12, otherwise jumping to step 14;
12. and if the matching is successful, updating the address of the page table entry pmd corresponding to the ms to the address corresponding to the sms, recording the address into the node corresponding to the merge tree sms, and marking the updated item in the merge tree.
13. And if the matching is unsuccessful, judging whether the merged tree is traversed or not, returning to the step 9 if the merging is not completed, and otherwise, jumping to the step 14.
14. Thus, it is explained that the same memory block is not found in the merged tree, the memory block ums is taken out from the un-merged tree for searching, and the step 15 is skipped.
15. Similarly, whether the memory blocks ums taken out are consistent with the md5 of the memory blocks ms to be combined is judged, if so, the step 16 is skipped, otherwise, the step 21 is skipped.
16. Comparing whether the content of ums and ms is consistent, if so, jumping to the step 17, otherwise, jumping to the step 21.
17. At this moment, ums and ms which are suspected to be found are pages with the same content, the page table attributes of the ums and the ms are changed into read-only in the step, the judgment is carried out again, and if the ums and the ms are consistent, the step is skipped to the step 19; otherwise, go to step 18.
18. In this step, it indicates that the ums or ms content is changed during the merging process, and the merging fails, the page table attributes of the two memory blocks are changed back to read and write, and the process skips to step 21.
19. In this step, the content of ums and ms is the same, ms is sent to the merge tree, a node is newly built in the merge tree, and the step 20 is skipped.
20. Deleting ums from the merged tree, then sending the ums into the merged tree, mounting the ums below the node where ms is located, and skipping to the step 22;
21. judging whether the non-merged tree is traversed or not, if so, indicating that no memory block capable of being merged is found, sending ms as a new node into the non-merged tree, updating the item mark in the non-merged tree, and then directly abandoning memory merging or continuously sending the page into other secondary storages for exchange; otherwise, skipping to step 14, and taking out the next uncombined memory block;
22. the memory blocks ms are successfully merged, the corresponding management metadata item is sent to the associated process related structure, and the whole memory merging process is completed;
(2) Update embodiments for non-merged trees
1. Periodically scanning the non-merged trees, triggering scanning when a timer expires, and setting a scanning quota (such as the number of scanned memory blocks or scanning time); the scan configuration is set in such a way that the non-merged tree is likely to be large and the total scan is too time-consuming, so that the scan configuration can be executed in batches based on the scan quota, and the influence on other tasks is reduced.
2. Reinitializing an uncombined root node; that is, the method is an organization method for destroying nodes in the original uncombined tree.
3. And taking out one process a from all the structures participating in the merging process to start scanning. There may be many processes for merging memory blocks, and in this embodiment, each process may be scanned one by one with the process as a granularity. Each process corresponds to a piece of metadata, which is used to record information of each memory block in the process, such as information whether the memory blocks are merged, information that the memory blocks belong to a merged tree, information that the memory blocks belong to an uncombined tree, and the like.
4. Scanning all memory blocks participating in merging below the process a, skipping merged memory blocks through metadata records, and only judging memory blocks belonging to non-merged trees; one memory block is taken from the uncombined tree.
5. Judging whether md5 is updated, if the md5 is updated, indicating that the memory block may frequently change and is not suitable for merging, updating md5, and going to step 7.
6. Otherwise, the memory block is sent to the uncombined tree again.
7. Judging whether the scanning quota is reached, and jumping to the step 9; otherwise, jumping to step 8.
8. Judging whether the pages participating in the process and merging are scanned completely, finishing the step 3, or else, finishing the step 4;
9. the scanning is ended.
(3) Memory recovery embodiments
1. Determining a process a to be replaced into the memory, and finding out memory allocation information mmap of the process a for subsequent query and update;
2. selecting a virtual memory vaddr which is merged from the memory allocation information mmap of the process a;
steps 1 and 2 in the above embodiment are described by taking active recovery of the operating system as an example, and in practical application, the operating system may be triggered when a page fault exception occurs.
3. And searching the metadata item corresponding to the vaddr from the process-related structure.
4. Allocating one memory block ms from main storage (i.e., memory), failing to exit, otherwise continuing.
5. Judging whether the merged memory block belongs to a zero block or a merged tree according to the item; if the block is zero, jump to step 6. Otherwise, jumping to step 8.
6. And judging whether the page is the zero page of the last node reference of the zero page, if so, directly changing the page into reading and writing, releasing the pre-allocated memory block ms, and jumping to the step 11. Otherwise, jump to step 7.
7. The address information of the page table entry pmd corresponding to the address vaddr is changed into the address corresponding to ms, the page table is changed into a readable and writable state, and the step 11 is skipped.
8. Finding out the associated node of the item in the merged tree through the item, changing the physical address of the page table item pmd corresponding to vaddr into the address corresponding to ms, copying the data of the merged block into new ms, and changing the page table attribute into read-write.
9. Judging that there are more residual merged memory blocks in the merged tree node, and if the number of the residual merged memory blocks in the merged tree node is more than one, skipping to the step 11; otherwise, jumping to step 10;
10. and deleting the merging node from the merging tree, changing the attribute of the sms page table of the memory block into reading and writing, and sending the attribute into the non-merging tree.
11. And at this point, the merged page is recovered from the merged node, the metadata item is released, and the page recovery process is finished.
Corresponding to the embodiments of the foregoing memory releasing method and memory recovering method, this specification also provides embodiments of a memory releasing device/a memory recovering device and a terminal applied thereto.
The embodiments of the memory releasing/restoring apparatus in this specification can be applied to a computer device, such as a server or a terminal device. The apparatus embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. In the case of software implementation, as a logical device, the corresponding computer program instructions in the nonvolatile memory are read into the memory by the processor in which the device is located and executed. From a hardware aspect, as shown in fig. 3, it is a hardware structure diagram of a computer device in which the memory releasing apparatus/the memory recovering apparatus are located in this specification, except for the processor 310, the memory 330, the network interface 320, and the nonvolatile memory 340 shown in fig. 3, in an embodiment, the computer device in which the memory releasing apparatus/the memory recovering apparatus 331 is located may also include other hardware according to an actual function of the computer device, which is not described again.
As shown in fig. 4, fig. 4 is a block diagram of a memory releasing device according to an exemplary embodiment, where the device includes:
a request module 41, configured to: responding to a memory release request, and determining a memory block to be released;
a zero block determination module 42 for: determining whether the memory block to be released is a zero block, wherein each bit of the memory block characterized by the zero block is zero;
an access module 43 for: if the memory block to be released is a zero block, accessing preset zero block data;
a release handling module 44 configured to: if the preset zero block data is not written with information, determining not to release the memory block to be released, updating the memory block to be released as a read-only permission, taking the memory block to be released as a reference zero block, writing a physical address of the memory block to be released in the preset zero block data, creating metadata corresponding to the virtual address of the memory block to be released, and associating the metadata with the preset zero block data; if the preset zero block data records the physical address of a reference zero block, mapping the virtual address corresponding to the memory block to be released to the physical address, creating metadata corresponding to the virtual address and associating the metadata with the preset zero block data, and then releasing the memory block to be released; wherein the metadata includes preset flag information.
In some examples, the zero block determination module is further configured to:
pre-judging whether each bit of the memory block to be released is zero or not;
if the bit position which is not zero exists in the memory block to be released is judged in advance, determining that the memory block to be released is not a zero block;
if all the bits of the memory block to be released are judged to be zero in advance, after the read-write permission of the memory block to be released is changed into read-only, whether the memory block to be released is a zero block is judged.
In some examples, the zero block determination module is further configured to:
selecting n bits from the memory block to be released, and determining whether the n bits are all zero; wherein n is a positive integer;
if not, pre-judging that a bit which is not zero exists in the memory block to be released;
if yes, pre-judging whether other bits of the memory block to be released except the n bits are all zero.
In some examples, the n bits are n bits of the set position, and the zero block determination module is further configured to:
generating current coding information of the bit positions of the n set positions by using a preset coding algorithm, wherein the preset coding algorithm is used for generating different coding information for different inputs;
comparing the current coding information with preset coding information, and determining whether the bit positions of the n set positions are all zero or not according to a comparison result; the preset coding information is information obtained by coding n bits which are all zero by using the preset coding algorithm in advance.
In some examples, the apparatus further includes a non-zero block processing module to:
if the memory block to be released is not a zero block, acquiring a candidate memory block set, and determining whether a target memory block storing the same data as the memory block to be released exists in the candidate memory block set;
if yes, mapping the virtual address of the memory block to be released to the physical address of the target memory block, creating metadata corresponding to the virtual address, and releasing the memory block to be released after the physical address of the target memory block is written in the metadata.
In some examples, each candidate memory block corresponds to candidate encoding information, where the candidate encoding information is obtained by encoding, by using the preset encoding algorithm, information of bits at n set positions of the candidate memory block;
the non-zero block processing module is further configured to: determining whether the candidate memory block stores the same data as the memory block to be released by the following method:
comparing the candidate coding information of the candidate memory block with the current coding information of the memory block to be released;
if the candidate memory block and the to-be-released memory block are different, determining that the candidate memory block and the to-be-released memory block store different data;
if the read-write permission of the memory block to be released and the read-write permission of the candidate memory block are the same, comparing whether the data stored in the candidate memory block is the same as the data stored in the memory block to be released.
In some examples, the set of candidate memory chunks includes a merged set of candidate memory chunks and an uncombined set of candidate memory chunks;
maintaining the merged candidate memory block set by adopting a merging tree, wherein each node in the merging tree records a physical address and candidate encoding information of one merged candidate memory block;
the non-merged candidate memory block set is maintained by using a non-merged tree, and each node in the non-merged tree records a physical address and candidate encoding information of one non-merged candidate memory block.
In some examples, the apparatus further comprises an update module to: according to a set period, updating the non-merged tree as follows:
determining that each node in the non-merged tree needs to be deleted; wherein, whether deletion is needed or not comprises: according to the physical address of the non-merged candidate memory block recorded by the node, accessing the non-merged candidate memory block and taking out the bit positions of the n set positions; generating coding information of the bits of the n set positions, and judging whether the generated coding information is the same as the candidate coding information; determining that deletion is required under different conditions;
and after the nodes which need to be deleted are deleted, reorganizing the non-merged tree according to the rest nodes.
As shown in fig. 5, fig. 5 is a block diagram of a memory recovery apparatus according to an exemplary embodiment, where the apparatus includes:
a request module 51, configured to: responding to a memory recovery request, and acquiring metadata corresponding to a virtual address according to the virtual address carried by the memory recovery request;
an assignment module 52 to: allocating memory blocks to be written from the memory;
an access module 53 for: if the metadata comprises preset mark information, accessing preset zero block data;
a recovery processing module 54 for: if the residual reference zero block is determined through the metadata related to the preset zero block data, modifying the read-only permission of the reference zero block in the preset zero block data into the read-write permission, releasing the memory block to be written, emptying the preset zero block data and deleting the metadata; and if at least two zero blocks are determined by the metadata related to the preset zero block data, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
In some examples, the apparatus further comprises a non-zero block processing module to:
if the metadata does not include preset flag information, accessing the target memory block according to the physical address of the target memory block mapped by the virtual address recorded in the metadata, writing the data of the target memory block into the memory block to be written, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
Accordingly, the present specification further provides a computer readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the method embodiment of the foregoing first aspect.
Embodiments of the present specification further provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method embodiment of the foregoing first aspect when executing the computer program.
The implementation process of the function and action of each module in the above device is specifically described in the implementation process of the corresponding step in the above memory release/memory recovery method, and is not described herein again.
Accordingly, embodiments of the present specification further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the foregoing memory release/memory recovery method embodiment are implemented.
Accordingly, embodiments of the present specification further provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the memory release/memory recovery method embodiments when executing the program.
Accordingly, embodiments of the present specification further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the memory release/memory recovery method embodiments.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, wherein the modules described as separate parts may or may not be physically separate, and the parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in the present specification. One of ordinary skill in the art can understand and implement it without inventive effort.
The above embodiments may be applied to one or more electronic devices, which are a device capable of automatically performing numerical calculation and/or information processing according to instructions set or stored in advance, and the hardware of the electronic device includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), an intelligent wearable device, and the like.
The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of hosts or network servers.
The Network where the electronic device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the present application to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
The description herein of "particular examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the specification. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Other embodiments of the present description will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations of the specification following, in general, the principles of the specification and including such departures from the present disclosure as come within known or customary practice within the art to which the specification pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the specification being indicated by the following claims.
It will be understood that the present description is not limited to the precise arrangements described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the description is limited only by the appended claims.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims (14)

1. A method of memory release, the method comprising:
responding to a memory release request, and determining a memory block to be released;
determining whether the memory block to be released is a zero block, wherein each bit of the memory block characterized by the zero block is zero;
if the memory block to be released is a zero block, accessing preset zero block data;
if the preset zero block data is not written with information, determining not to release the memory block to be released, updating the memory block to be released as a read-only permission, taking the memory block to be released as a reference zero block, writing a physical address of the memory block to be released in the preset zero block data, creating metadata corresponding to the virtual address of the memory block to be released, and associating the metadata with the preset zero block data;
if the preset zero block data records the physical address of a reference zero block, mapping the virtual address corresponding to the memory block to be released to the physical address, creating metadata corresponding to the virtual address and associating the metadata with the preset zero block data, and then releasing the memory block to be released; wherein the metadata includes preset flag information.
2. The method according to claim 1, wherein the determining whether the to-be-released memory chunk is a zero chunk includes:
pre-judging whether each bit of the memory block to be released is zero or not;
if the bit position which is not zero exists in the memory block to be released is judged in advance, determining that the memory block to be released is not a zero block;
if it is judged that all the bits of the memory block to be released are zero, after the read-write permission of the memory block to be released is changed into read-only, whether the memory block to be released is a zero block is judged.
3. The method according to claim 2, wherein the pre-determining whether each bit of the memory block to be released is zero includes:
selecting n bits from the memory block to be released, and determining whether the n bits are all zero; wherein n is a positive integer;
if not, pre-judging that a bit which is not zero exists in the memory block to be released;
if yes, pre-judging whether other bits of the memory block to be released except the n bits are all zero.
4. The method of claim 3, the n bits being n set-position bits, the determining whether the n bits are all zeros comprising:
generating current coding information of the bit positions of the n set positions by using a preset coding algorithm, wherein the preset coding algorithm is used for generating different coding information for different inputs;
comparing the current coding information with preset coding information, and determining whether the bit positions of the n set positions are all zero according to a comparison result; the preset coding information is obtained by coding n bits which are all zero by using the preset coding algorithm in advance.
5. The method of claim 1, further comprising:
if the memory block to be released is not a zero block, acquiring a candidate memory block set, and determining whether a target memory block storing the same data as the memory block to be released exists in the candidate memory block set;
if yes, mapping the virtual address of the memory block to be released to the physical address of the target memory block, creating metadata corresponding to the virtual address, and releasing the memory block to be released after the physical address of the target memory block is written in the metadata.
6. The method according to claim 5, wherein each of the candidate memory blocks corresponds to candidate encoding information obtained by encoding, by using the preset encoding algorithm, information of bits at n set positions of the candidate memory block;
whether the candidate memory block stores the same data as the memory block to be released is determined by the following method:
comparing the candidate coding information of the candidate memory block with the current coding information of the memory block to be released;
if the candidate memory block and the to-be-released memory block are different, determining that the candidate memory block and the to-be-released memory block store different data;
if the read-write permission of the memory block to be released and the read-write permission of the candidate memory block are the same, comparing whether the data stored in the candidate memory block is the same as the data stored in the memory block to be released.
7. The method according to claim 6, wherein the set of candidate memory chunks includes a merged set of candidate memory chunks and an uncombined set of candidate memory chunks;
maintaining the merged candidate memory block set by adopting a merging tree, wherein each node in the merging tree records a physical address and candidate encoding information of one merged candidate memory block;
the non-merged candidate memory block set is maintained by using a non-merged tree, and each node in the non-merged tree records a physical address and candidate encoding information of one non-merged candidate memory block.
8. The method of claim 7, further comprising:
according to a set period, updating the non-merged tree as follows:
determining whether nodes in the non-merged tree need to be deleted; wherein, whether deletion is required or not includes: according to the physical address of the non-merged candidate memory block recorded by the node, accessing the non-merged candidate memory block and taking out the bit positions of the n set positions; generating coding information of the bits of the n set positions, and judging whether the generated coding information is the same as the candidate coding information; determining that deletion is required under different conditions;
and after the nodes which need to be deleted are deleted, reorganizing the non-merged tree according to the rest nodes.
9. A method of memory recovery, the method comprising:
responding to a memory recovery request, and acquiring metadata corresponding to a virtual address according to the virtual address carried by the memory recovery request;
allocating memory blocks to be written from the memory;
if the metadata comprises preset mark information, accessing preset zero block data;
if the residual reference zero block is determined through the metadata related to the preset zero block data, modifying the read-only permission of the reference zero block into the read-write permission, releasing the memory block to be written, emptying the preset zero block data and deleting the metadata;
and if at least two zero blocks are determined by the metadata associated with the preset zero block data, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
10. The method of claim 9, further comprising:
if the metadata does not include preset mark information, accessing the target memory block according to the physical address of the target memory block mapped by the virtual address recorded in the metadata, writing the data of the target memory block into the memory block to be written, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
11. A memory release device, the device comprising:
a request module to: responding to a memory release request, and determining a memory block to be released;
a zero block determination module to: determining whether the memory block to be released is a zero block, wherein each bit of the memory block characterized by the zero block is zero;
an access module to: if the memory block to be released is a zero block, accessing preset zero block data;
a release handling module to: if the preset zero block data is not written with information, determining not to release the memory block to be released, updating the memory block to be released as a read-only permission, taking the memory block to be released as a reference zero block, writing a physical address of the memory block to be released in the preset zero block data, creating metadata corresponding to the virtual address of the memory block to be released, and associating the metadata with the preset zero block data; if the preset zero block data records the physical address of a reference zero block, mapping the virtual address corresponding to the memory block to be released to the physical address, creating metadata corresponding to the virtual address and associating the metadata with the preset zero block data, and then releasing the memory block to be released; wherein the metadata includes preset flag information.
12. A memory recovery apparatus, the apparatus comprising:
a request module to: responding to a memory recovery request, and acquiring metadata corresponding to a virtual address according to the virtual address carried by the memory recovery request;
an assignment module to: allocating memory blocks to be written from the memory;
an access module to: if the metadata comprises preset mark information, accessing preset zero block data;
a recovery processing module to: if the residual reference zero block is determined through the metadata associated with the preset zero block data, modifying the read-only permission of the reference zero block into the read-write permission, releasing the memory block to be written, emptying the preset zero block data and deleting the metadata; and if at least two zero blocks are determined by the metadata associated with the preset zero block data, mapping the virtual address to the physical address of the memory block to be written, and deleting the metadata.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any of claims 1 to 10 are implemented when the computer program is executed by the processor.
14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 10.
CN202211406834.3A 2022-11-10 2022-11-10 Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium Pending CN115756838A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211406834.3A CN115756838A (en) 2022-11-10 2022-11-10 Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211406834.3A CN115756838A (en) 2022-11-10 2022-11-10 Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115756838A true CN115756838A (en) 2023-03-07

Family

ID=85369112

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211406834.3A Pending CN115756838A (en) 2022-11-10 2022-11-10 Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115756838A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116107925A (en) * 2023-04-10 2023-05-12 阿里云计算有限公司 Data storage unit processing method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116107925A (en) * 2023-04-10 2023-05-12 阿里云计算有限公司 Data storage unit processing method
CN116107925B (en) * 2023-04-10 2023-09-26 阿里云计算有限公司 Data storage unit processing method

Similar Documents

Publication Publication Date Title
US10983955B2 (en) Data unit cloning in memory-based file systems
TWI661301B (en) Memory system and control method for controlling non-volatile memory
JP6122557B2 (en) File migration method and apparatus, and storage device
US9501421B1 (en) Memory sharing and page deduplication using indirect lines
CN107066498B (en) Key value KV storage method and device
CN110018998B (en) File management method and system, electronic equipment and storage medium
US9069477B1 (en) Reuse of dynamically allocated memory
US10552377B2 (en) Data discard method for journaling file system and memory management apparatus thereof
CN110858162B (en) Memory management method and device and server
CN107329704B (en) Cache mirroring method and controller
WO2024099448A1 (en) Memory release method and apparatus, memory recovery method and apparatus, and computer device and storage medium
CN113254365A (en) Method, apparatus, device, medium, and program product for managing storage space
CN115617542A (en) Memory exchange method and device, computer equipment and storage medium
CN115599544A (en) Memory management method and device, computer equipment and storage medium
CN114327917A (en) Memory management method, computing device and readable storage medium
CN116302491A (en) Memory management method, device, computer equipment and storage medium
CN115756838A (en) Memory release method, memory recovery method, memory release device, memory recovery device, computer equipment and storage medium
CN113835639B (en) I/O request processing method, device, equipment and readable storage medium
CN114546898A (en) TLB management method, device, equipment and storage medium
CN110955488A (en) Virtualization method and system for persistent memory
US10416901B1 (en) Storage element cloning in presence of data storage pre-mapper with multiple simultaneous instances of volume address using virtual copies
CN116225693A (en) Metadata management method, device, computer equipment and storage medium
US10209909B1 (en) Storage element cloning in presence of data storage pre-mapper
CN116302376A (en) Process creation method, process creation device, electronic equipment and computer readable medium
US11954328B2 (en) Storage management device, storage management method, and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination