CN116860439A - Memory management method and device, electronic equipment and storage medium - Google Patents

Memory management method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116860439A
CN116860439A CN202310764892.1A CN202310764892A CN116860439A CN 116860439 A CN116860439 A CN 116860439A CN 202310764892 A CN202310764892 A CN 202310764892A CN 116860439 A CN116860439 A CN 116860439A
Authority
CN
China
Prior art keywords
data
target object
access
pool
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310764892.1A
Other languages
Chinese (zh)
Inventor
涂云山
陈嘉华
赵裕众
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Oceanbase Technology Co Ltd
Original Assignee
Beijing Oceanbase Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Oceanbase Technology Co Ltd filed Critical Beijing Oceanbase Technology Co Ltd
Priority to CN202310764892.1A priority Critical patent/CN116860439A/en
Publication of CN116860439A publication Critical patent/CN116860439A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24552—Database cache management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

One or more embodiments of the present disclosure provide a memory management method and apparatus, an electronic device, and a storage medium, where the method includes: determining a first target object in a first object pool in a memory in response to first data in a permanent storage device being accessed, and caching partial or all fields of the first data in the first target object, wherein the first object pool comprises a plurality of first objects with the same capacity, and the first target object is one first object in the first object pool; and in response to the first target object being filled with a portion of the fields of the first data and other fields of the first data being accessed, determining a second target object in a second object pool in the memory, and caching all the fields of the first data in the second target object, wherein the second object pool comprises a plurality of second objects with the same capacity, the capacity of the second object is larger than that of the first object, and the second target object is one second object in the second object pool.

Description

Memory management method and device, electronic equipment and storage medium
Technical Field
One or more embodiments of the present disclosure relate to the field of database technologies, and in particular, to a memory management method and apparatus, an electronic device, and a storage medium.
Background
Today, the development of the internet and informatization is rapid, and the generation of data is explosively increasing, so that the requirements for databases and management thereof are increasing. When the data in the database is accessed, the data needs to be loaded into the memory for access, and because the data in the database is more, the limited memory often cannot load all the data, and the data needs to be swapped in and out in the memory. In the related art, the management effect of the memory is poor, so that the utilization rate of the memory is low.
Disclosure of Invention
In view of this, one or more embodiments of the present disclosure provide a memory management method and apparatus, an electronic device, and a storage medium.
In order to achieve the above object, one or more embodiments of the present disclosure provide the following technical solutions:
according to a first aspect of one or more embodiments of the present disclosure, a memory management method is provided, where the method includes:
determining a first target object in a first object pool in a memory in response to first data in a permanent storage device being accessed, and caching part or all of fields of the first data in the first target object, wherein the first object pool comprises a plurality of first objects with the same capacity, and the first target object is one first object in the first object pool;
And in response to the first target object being filled with a part of fields of the first data and other fields of the first data being accessed, determining a second target object in a second object pool in a memory, and caching all fields of the first data in the second target object, wherein the second object pool comprises a plurality of second objects with the same capacity, the capacity of the second object is larger than that of the first object, and the second target object is one second object in the second object pool.
In one embodiment of the present specification, the method further comprises:
determining an access heat score value of a first object according to an access source and access time in response to the first object in the first object pool being accessed;
the determining a first target object in the first object pool in the memory includes:
and in response to the first object in the first object pool without the idle state, determining a first target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, and deleting the data cached in the first target object.
In one embodiment of the present specification, the method further comprises:
determining an access heat score value of a second object according to an access source and access time in response to the second object in the second object pool being accessed;
determining a second target object in the second object pool in the memory, including:
and in response to the second object in the second object pool without the idle state, determining a second target object in at least one second object in the second object pool according to the access heat scoring value and the access time of the at least one second object, and transferring part of data cached in the second target object to the first object in the first object pool.
In one embodiment of the present disclosure, the transferring the part of the data cached in the second target object to the first object in the first object pool includes:
and transferring partial data with highest access heat grading value cached in the first target object to a first object in the first object pool.
In one embodiment of the present disclosure, the transferring the part of the data cached in the second target object to the first object in the first object pool includes:
And responding to the first object in the first object pool without the idle state, determining a third target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, deleting the data cached in the third target object, and transferring part of the data cached in the second target object to the third target object, wherein the third target object is one first object in the first object pool.
In one embodiment of the present specification, the method further comprises:
responding to the failure of deleting the data cached in the first target object, and re-determining the first target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the first target object is successfully deleted; and/or the number of the groups of groups,
responding to the data transfer failure of the cache in the second target object, and re-determining the second target object according to the access heat scoring value and the access time of at least one second object in the second object pool until the data transfer of the cache in the second target object is successful; and/or the number of the groups of groups,
And responding to the failure of deleting the data cached in the third target object, and re-determining the third target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the third target object is successfully deleted.
In one embodiment of the present specification, the determining the access heat score value of the first object according to the access source and the access time includes:
determining an access popularity grading system of the first object according to the user priority and the access time under the condition that the access source is a user;
when the access source is a system and the first object is not accessed by a user, determining an access heat scoring value of the first object according to the system priority and the access time;
when the access source is a system and the first object is accessed by a user, maintaining the access heat scoring value of the first object unchanged; and/or the number of the groups of groups,
the determining the access heat scoring value of the second object according to the access source and the access time comprises the following steps:
determining an access heat scoring value of the second object according to the user priority and the access time under the condition that the access source is a user;
When the access source is a system and the second object is not accessed by a user, determining an access heat scoring value of the second object according to the system priority and the access time;
and when the access source is a system and the second object is accessed by a user, keeping the access heat scoring value of the second object unchanged.
In one embodiment of the present disclosure, the determining, according to the access heat score value and the access time of at least one first object in the first object pool, a first target object in the at least one first object includes:
determining whether the access heat degree score value of each first object in the first object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a first object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a first target object;
the determining a second target object in at least one second object in the second object pool according to the access heat scoring value and the access time of the at least one second object comprises the following steps:
And determining whether the access heat degree score value of each second object in the second object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a second object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a second target object.
In one embodiment of the present disclosure, in response to the first target object being filled with a portion of the fields of the first data and other fields of the first data being accessed, determining a second target object within a second object pool in memory and caching all of the fields of the first data within the second target object, includes:
and in response to the first target object being filled with a part of fields of the first data, other fields of the first data being accessed, and the access frequency of the other fields of the first data being higher than a preset frequency threshold, determining a second target object in a second object pool in the memory, and caching all fields of the first data in the second target object.
In one embodiment of the present specification, the method further comprises:
In response to second data being created, saving the second data to the persistent storage device;
determining a fourth target object in a first object pool in a memory in response to the capacity of the second data being smaller than or equal to the capacity of the first object, and caching all fields of the second data in the fourth target object, wherein the fourth target object is one first object in the first object pool;
and in response to the capacity of the second data being greater than the capacity of the first object, determining a fifth target object in a second object pool in the memory, and caching all fields of the second data in the fifth target object, wherein the fifth target object is one second object in the second object pool.
In one embodiment of the present specification, the first data and the second data are metadata of a data slice.
According to a second aspect of one or more embodiments of the present disclosure, there is provided a memory management device, the device including:
the first caching module is used for responding to that first data in the permanent storage device are accessed, determining a first target object in a first object pool in a memory, and caching part or all fields of the first data in the first target object, wherein the first object pool comprises a plurality of first objects with the same capacity, and the first target object is one first object in the first object pool;
And the second caching module is used for responding to the fact that the first target object is filled with part of fields of the first data, other fields of the first data are accessed, determining a second target object in a second object pool in a memory, and caching all fields of the first data in the second target object, wherein the second object pool comprises a plurality of second objects with the same capacity, the capacity of the second object is larger than that of the first object, and the second target object is one second object in the second object pool.
In one embodiment of the present specification, the apparatus further comprises a scoring module for:
determining an access heat score value of a first object according to an access source and access time in response to the first object in the first object pool being accessed;
the first cache module is configured to, when determining a first target object in a first object pool in the memory, specifically:
and in response to the first object in the first object pool without the idle state, determining a first target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, and deleting the data cached in the first target object.
In one embodiment of the present specification, the apparatus further comprises a scoring module for:
determining an access heat score value of a second object according to an access source and access time in response to the second object in the second object pool being accessed;
the second cache module is configured to, when determining a second target object in a second object pool in the memory, specifically:
and in response to the second object in the second object pool without the idle state, determining a second target object in at least one second object in the second object pool according to the access heat scoring value and the access time of the at least one second object, and transferring part of data cached in the second target object to the first object in the first object pool.
In one embodiment of the present disclosure, the second buffer module is configured to, when transferring the part of the data buffered in the second target object to the first object in the first object pool, specifically:
and transferring partial data with highest access heat grading value cached in the first target object to a first object in the first object pool.
In one embodiment of the present disclosure, the second buffer module is configured to, when transferring the part of the data buffered in the second target object to the first object in the first object pool, specifically:
And responding to the first object in the first object pool without the idle state, determining a third target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, deleting the data cached in the third target object, and transferring part of the data cached in the second target object to the third target object, wherein the third target object is one first object in the first object pool.
In one embodiment of the present specification, the first buffer module is further configured to: responding to the failure of deleting the data cached in the first target object, and re-determining the first target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the first target object is successfully deleted; and/or the number of the groups of groups,
the second cache module is further configured to: responding to the data transfer failure of the cache in the second target object, and re-determining the second target object according to the access heat scoring value and the access time of at least one second object in the second object pool until the data transfer of the cache in the second target object is successful; and/or the number of the groups of groups,
The second cache module is further configured to: and responding to the failure of deleting the data cached in the third target object, and re-determining the third target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the third target object is successfully deleted.
In one embodiment of the present disclosure, the scoring module is configured to, when determining the access heat score value of the first object according to the access source and the access time, specifically:
determining an access popularity grading system of the first object according to the user priority and the access time under the condition that the access source is a user;
when the access source is a system and the first object is not accessed by a user, determining an access heat scoring value of the first object according to the system priority and the access time;
when the access source is a system and the first object is accessed by a user, maintaining the access heat scoring value of the first object unchanged; and/or the number of the groups of groups,
the scoring module is used for determining the access heat scoring value of the second object according to the access source and the access time, and is specifically used for:
Determining an access heat scoring value of the second object according to the user priority and the access time under the condition that the access source is a user;
when the access source is a system and the second object is not accessed by a user, determining an access heat scoring value of the second object according to the system priority and the access time;
and when the access source is a system and the second object is accessed by a user, keeping the access heat scoring value of the second object unchanged.
In one embodiment of the present disclosure, the first buffer module is configured to, when determining a first target object in at least one first object in the first object pool according to an access heat score value and an access time of the at least one first object, specifically be:
determining whether the access heat degree score value of each first object in the first object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a first object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a first target object;
The second cache module is configured to determine, according to the access heat score value and the access time of at least one second object in the second object pool, a second target object in the at least one second object, where the second target object is specifically configured to:
and determining whether the access heat degree score value of each second object in the second object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a second object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a second target object.
In one embodiment of the present disclosure, the second buffer module is specifically configured to:
and in response to the first target object being filled with a part of fields of the first data, other fields of the first data being accessed, and the access frequency of the other fields of the first data being higher than a preset frequency threshold, determining a second target object in a second object pool in the memory, and caching all fields of the first data in the second target object.
In one embodiment of the present specification, the apparatus further comprises an allocation module for:
In response to second data being created, saving the second data to the persistent storage device;
determining a fourth target object in a first object pool in a memory in response to the capacity of the second data being smaller than or equal to the capacity of the first object, and caching all fields of the second data in the fourth target object, wherein the fourth target object is one first object in the first object pool;
and in response to the capacity of the second data being greater than the capacity of the first object, determining a fifth target object in a second object pool in the memory, and caching all fields of the second data in the fifth target object, wherein the fifth target object is one second object in the second object pool.
In one embodiment of the present specification, the first data and the second data are metadata of a data slice.
According to a third aspect of one or more embodiments of the present specification, there is provided an electronic device comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor implements the method of the first aspect by executing the executable instructions.
According to a fourth aspect of one or more embodiments of the present description, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method as described in the first aspect.
The technical scheme provided by the embodiment of the specification can comprise the following beneficial effects:
the memory management method provided by the embodiment of the present disclosure presets a first object pool and a second object pool in a memory, where the capacity of a first object in the first object pool is smaller than the capacity of a second object in the second object pool; therefore, when the first data in the permanent storage device is accessed and needs to be loaded, a first object can be selected from the first object pool as a first target object, and part or all of fields of the first data are cached (i.e. loaded) in the first target object; and when the first target object is filled with a part of fields of the first data and other fields of the first data are accessed, a second object can be selected from the second object pool as a second target object, and all fields of the first data are cached in the second target object. When certain data is needed to be loaded, the first object with smaller capacity is allocated first, and if loading requirements exist on the unloaded part of the data in the first object, the data can be upgraded to the second object with larger capacity, so that the memory can be reasonably allocated for loading the data, the memory waste caused by uniformly allocating the objects with larger capacity is avoided, meanwhile, the memory fragments are avoided from being generated due to the preset of the object pool, and the utilization rate of the memory is further improved.
Drawings
Fig. 1 is a flowchart of a memory management method according to an exemplary embodiment.
Fig. 2 is a schematic diagram of a first elimination linked list provided by an example embodiment.
Fig. 3 is a schematic diagram of a second elimination linked list provided by an example embodiment.
FIG. 4 is a flowchart of a method for allocating memory objects provided by an exemplary embodiment.
FIG. 5 is a flow chart of a method for eliminating memory objects according to an exemplary embodiment.
Fig. 6 is a schematic diagram of an apparatus according to an exemplary embodiment.
Fig. 7 is a block diagram of a memory management device according to an exemplary embodiment.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with aspects of one or more embodiments of the present description as detailed in the accompanying claims.
It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.
Today, the development of the internet and informatization is rapid, and the generation of data is explosively increasing, so that the requirements for databases and management thereof are increasing. When the data in the database is accessed, the data needs to be loaded into the memory for access, and because the data in the database is more, the limited memory often cannot load all the data, and the data needs to be swapped in and out in the memory. In the related art, the management effect of the memory is poor, so that the utilization rate of the memory is low.
Based on this, in a first aspect, at least one embodiment of the present disclosure provides a memory management method, which is configured to manage memory resources to control data loaded on a memory, so as to improve a utilization rate of the memory. It will be appreciated that the method may manage the overall memory resources or the memory resources used to load specific data, such as the memory resources used to load metadata for data shards within a database.
For example, a first object pool (for example, a normal object pool) and a second object pool (for example, a large object pool) may be built in advance in a memory, where the first object pool may include a plurality of first objects (for example, normal objects) with the same capacity (size), for example, a first object with the same capacity of 4kB, and the second object pool may include a plurality of second objects (for example, large objects) with the same capacity, where the capacity of the second object is greater than that of the first object, for example, the capacity of the second object may be 64kB; the capacity of the second object may be determined according to the maximum capacity (max size) of the memory resources managed by the method for loading the specific data, i.e. the second object is capable of loading the specific data of the maximum capacity, e.g. the method manages the memory resources for loading the metadata of the data fragments in the database, the capacity of the second object is the capacity of the metadata of the data fragments of the maximum capacity.
The first object and the second object are specific spaces in the memory; the memory capacity occupied by the second object pool may be preset, for example, the number of the first objects is preset when the capacity of the first objects is known, and the memory capacity occupied by the second object pool may be preset, for example, the number of the second objects is preset when the capacity of the second objects is known; the ratio between the memory capacity occupied by the second object pool and the memory capacity occupied by the first object pool may be preset, preferably, the content capacity occupied by the second object pool is smaller than the memory capacity occupied by the first object pool, for example, the ratio between the memory capacity occupied by the second object pool and the memory capacity occupied by the first object pool may be 1:16.
Referring to fig. 1, a flow of the method is schematically shown, and includes steps S101 to S102.
In step S101, in response to the first data in the persistent storage device being accessed, a first target object is determined in a first object pool in the memory, and part or all of fields of the first data are cached in the first target object, where the first object pool includes a plurality of first objects with the same capacity, and the first target object is one first object in the first object pool.
The permanent storage device may be a storage device such as a magnetic disk. The first data may be metadata of a data fragment. The data on the persistent storage device may be accessed by a user, such as a user's query request, etc.; the data on the persistent storage device may also be accessed by the system (i.e., database management system) in the background, such as by the system traversing the data on the persistent storage device for data statistics, etc.
For example, when determining the first target object in the first object pool in the memory, in response to the first object in the first object pool having the idle state, determining the first object in any idle state as the first target object, for example, determining the first object in the first idle state as the first target object. The system may maintain a first cull list as shown in fig. 2, after loading data in a certain first object in the first object pool, the first object may be recorded in the first cull list, after releasing a certain first object in the first object pool, the first object may be deleted from the first cull list, i.e. after determining that a first object in an idle state is a first target object, the first object in the idle state may be recorded in the cull list.
It will be appreciated that the first object loaded with data may be accessed by a user or a system, and that access conditions such as access source, access time, etc. may affect the access popularity of the first object, so that the access popularity score value of the first object may be determined according to the access source and the access time in response to the first object being accessed in the first object pool. For example, different access sources have different priorities, e.g., user priority is 1 and system priority is 0; further, the access heat score value of the first object may be determined in different ways in the following three cases, respectively:
first, in the case that the access source is a user, determining an access heat score value of the first object according to a user priority and the access time. Specifically, the following formula may be used to calculate the access heat score value of the first object:
Score＝(1-cpriority)*tlast+cpriority*(tlast-INT64_Max)
where Score is the access heat Score, cpriority is the user priority, tlast is the access time, and INT64_Max is the maximum value of the time recording scheme.
Second, in the case that the access source is a system and the first object (the data currently loaded in the system) is not accessed by the user, determining an access heat score value of the first object according to the system priority and the access time. Specifically, the following formula may be used to calculate the access heat score value of the first object:
Score＝(1-cpriority’)tlast+cpriority’*(tlast-INT64_Max)
Where Score is the access heat Score, cpriority' is the system priority, tlast is the access time, and INT64_Max is the maximum value of the time recording scheme.
Third, in the case where the access source is a system and the first object (the data currently loaded in) is accessed by the user, the access heat score value of the first object is kept unchanged. Since the user priority is higher than the system priority, after the user accesses the first object, the access heat score value of the first object is updated only when the user accesses the first object again, and the access heat score value of the first object is not updated when the system accesses the first object; in other words, the access heat score value of the first object is updated only in the rising direction, and is not updated in the falling direction.
When determining the first target object in the first object pool in the memory based on the update of the access heat score value of the first object, the first target object may be determined in at least one first object in the first object pool according to the access heat score value and the access time of the at least one first object in the first object pool in response to the first object in the first object pool having no idle state, and the data cached in the first target object is deleted.
For example, it is sequentially determined (according to the order in the first elimination chain table) whether the access popularity score value of each first object in the first object pool is smaller than a first score threshold (e.g. 0), whether the access time is greater than a first time length threshold (e.g. 5 s) from the current time, and the first object in the first object pool, where the access popularity score value is smaller than the first score threshold, or the access time is greater than the first time length threshold, is determined as the first target object.
For example, in response to successful deletion of the cached data within the first target object, caching a portion or all of the fields of the first data within the first target object; and responding to the failure of deleting the data cached in the first target object, and re-determining the first target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the first target object is successfully deleted. For example, if the first target object is being accessed, the data cached in the first target object cannot be deleted and the deletion fails. For another example, when the first target object is redetermined, the manner in the above example may be continuously adopted, that is, whether the access heat score value of each first object that is not traversed in the first object pool is sequentially determined to be smaller than a first score threshold (for example, 0), whether the access time is greater than a first time length threshold (for example, 5 s) from the current time, and the first object in the first object pool, where the access heat score value is smaller than the first score threshold, or the access time is greater than the first time length threshold, is determined to be the first target object. In one possible scenario, when the first target object is determined for the first time, traversing is started by the first object until the access heat score value of the fifth first object is not smaller than the first score threshold value, so that the fifth first object is determined to be the first target object; however, if the deletion of the data cached in the fifth first object fails, continuing to traverse by the sixth first object until the access time distance of the ninth first object is less than the current time and is less than the first time threshold, thereby determining that the ninth first object is the first target object; however, if the deletion of the data cached in the ninth first object fails, the traversal is continued from the tenth first object, and the access time of the tenth first object is just less than the first time threshold from the current time, so that the tenth first object is determined to be the first target object, and the deletion of the data cached in the tenth first object is successful, so that all or part of the fields of the first data are cached in the tenth first object.
Further exemplary, when caching a part or all of the fields of the first data in the first target object, if the capacity of all of the fields of the first data is less than or equal to the capacity of the first object, caching all of the fields of the first data in the first target object; if the capacity of all the fields of the first data is greater than the capacity of the first object, then a portion of the fields of the first data are cached in the first target object, such as a plurality of fields including the currently accessed field, and the cached fields are utilized to fill the first target object.
In step S102, in response to the first target object being filled with a portion of the fields of the first data and other fields of the first data being accessed, a second target object is determined in a second object pool in the memory, and all fields of the first data are cached in the second target object, where the second object pool includes a plurality of second objects with the same capacity, the capacity of the second object is greater than the capacity of the first object, and the second target object is one second object in the second object pool.
The function of this step is to upgrade the memory object of the first data from a first object (e.g. normal object) with smaller capacity to a second object (e.g. large object) when a part of the fields of the first data are filled with the first target object and other fields of the first data also have loading requirements. It will be appreciated that the data within the first target object may be emptied after this step has been performed.
Alternatively, the other fields of the first data may be determined directly to have a load requirement when accessed, for example, when any of the other fields of the first data is accessed.
Optionally, when the other fields of the first data are accessed and the access frequency of the other fields of the first data is higher than the preset frequency threshold, it may be determined that the other fields of the first data have a loading requirement, for example, when any one of the other fields of the first data is accessed and the number of accesses (including the current access) exceeds the preset number threshold (e.g., 5) within a certain period of time (e.g., 10 s) recently, it is determined that the other fields of the first data have a loading requirement. That is, the step may determine a second target object in a second object pool in the memory and cache all fields of the first data in the second target object in response to the first target object being filled with a portion of the fields of the first data, other fields of the first data being accessed, and the frequency of access of the other fields of the first data being above a preset frequency threshold.
For example, when determining the second target object in the second object pool in the memory, in response to the second object in the second object pool having the idle state, determining the second object in any idle state as the second target object, for example, determining the second object in the first idle state as the second target object. The system may maintain a second cull list as shown in fig. 3, after loading data in a certain second object in the second object pool, the second object may be recorded in the second cull list, after releasing a certain second object in the second object pool, the second object may be deleted from the second cull list, i.e. after determining that a second object in an idle state is a second target object, the second object in the idle state may be recorded in the cull list.
It will be appreciated that the second object loaded with data may be accessed by a user or a system, and that access conditions such as access source, access time, etc. may affect the access popularity of the second object, so that the access popularity score value of the second object may be determined according to the access source and the access time in response to the second object being accessed in the second object pool. For example, different access sources have different priorities, e.g., user priority is 1 and system priority is 0; further, the access heat score value of the second object may be determined in different ways in the following three cases, respectively:
First, in the case that the access source is a user, determining an access heat score value of the second object according to a user priority and the access time. Specifically, the following formula may be used to calculate the access heat score value of the second object:
Score＝(1-cpriority)*tlast+cpriority*(tlast-INT64_Max)
where Score is the access heat Score, cpriority is the user priority, tlast is the access time, and INT64_Max is the maximum value of the time recording scheme.
Second, in the case that the access source is a system and the second object (the data currently loaded in the system) is not accessed by the user, determining an access heat score value of the second object according to the system priority and the access time. Specifically, the following formula may be used to calculate the access heat score value of the second object:
Score＝(1-cpriority’)tlast+cpriority’*(tlast-INT64_Max)
where Score is the access heat Score, cpriority' is the system priority, tlast is the access time, and INT64_Max is the maximum value of the time recording scheme.
Third, in the case where the access source is a system and the second object (the data currently loaded in the memory) is accessed by the user, the access heat score value of the second object is kept unchanged. Since the user priority is higher than the system priority, after the user accesses the second object, the access heat score value of the second object is updated only when the user accesses the second object again, and the access heat score value of the second object is not updated when the system accesses the second object; in other words, the access heat score value of the second object is updated only in the rising direction, and is not updated in the falling direction.
And when determining the second target object in the second object pool in the memory based on the update of the access heat score value of the second object, determining the second target object in at least one second object according to the access heat score value and the access time of at least one second object in the second object pool in response to the fact that the second object in the idle state does not exist in the second object pool, and transferring part of data cached in the second target object to the first object in the first object pool.
For example, it is sequentially determined (in the order of the first elimination chain table) whether the access popularity score value of each second object in the second object pool is smaller than a first score threshold (e.g. 0), whether the access time is greater than a first time threshold (e.g. 5 s) from the current time, and the first access popularity score value in the second object pool is smaller than the first score threshold, or the second object whose access time is greater than the first time threshold from the current time is determined as the second target object.
For example, each time a second object is accessed, the access heat score value of a particular accessed field may be determined based on the access source and access time, and the particular manner in which the access heat score value of the field is determined may be the same as or different from the manner in which the access heat score value of the second object is determined in the above examples. Furthermore, when the partial data cached in the second target object is transferred to the first object in the first object pool, the partial data with the highest access heat score value cached in the first target object is transferred to the first object in the first object pool, that is, the fields are sorted according to the access heat score value, and transfer is started to the first object from the field with the highest access heat score value until the first object is filled up, and then transfer is stopped, and the remaining fields in the second target object are deleted.
For example, when partial data cached in the second target object is transferred to the first object in the first object pool, the partial data cached in the second target object may be transferred to the first object in any idle state in response to the first object in the first object pool having the idle state; and in response to the first object in the first object pool in the idle state not being existed, determining a third target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, deleting the data cached in the third target object, and transferring part of the data cached in the second target object to the third target object, wherein the third target object is one first object in the first object pool. It will be appreciated that the manner of determining the third target object in the first object pool may be the same as or different from the manner of determining the first target object in step S101; in addition, partial data cached in the second target object is transferred to the third target object in response to successful deletion of the data cached in the third target object; and responding to the failure of deleting the data cached in the third target object, and re-determining the third target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the third target object is successfully deleted. For example, if the third target object is being accessed, the data cached in the third target object cannot be deleted and the deletion fails. For another example, when the third target object is redetermined, the manner of redefining the first target object in step S101 may be adopted, or other manners.
For example, in response to successful transfer of the data cached within the second target object, all fields of the first data are cached within the second target object; and responding to the data transfer failure cached in the second target object, and re-determining the second target object according to the access heat scoring value and the access time of at least one second object in the second object pool until the data transfer cached in the second target object is successful. For example, if the second target object is being accessed, the data cached in the second target object cannot be restored and fails to be restored. For another example, when the second target object is redetermined, the manner in the above example may be continuously adopted, that is, whether the access heat score value of each second object that is not traversed in the second object pool is sequentially determined to be smaller than a first score threshold (for example, 0), whether the access time is greater than a first time threshold (for example, 5 s) from the current time, and the first access heat score value in the second object pool is smaller than the first score threshold, or the access time is greater than the first time threshold from the current time is determined to be the second target object. In one possible scenario, when the second target object is determined for the first time, starting traversing by the first second object until the access heat score value of the fifth second object is less than the first score threshold, thereby determining the fifth second object as the second target object; however, if the data cached in the fifth second object is not transferred, continuing to traverse by the sixth second object until the access time distance of the ninth second object is smaller than the current time length threshold value, thereby determining that the ninth second object is a second target object; however, if the data cached in the ninth second objects is not cached, the traversal is continued from the tenth second object, and the access time of the tenth second object is just less than the current time by the first time threshold, so that the tenth second object is determined to be the second target object, and the data cached in the tenth second object is successfully cached, so that all fields of the first data are cached in the tenth second object.
The memory management method provided by the embodiment of the present disclosure presets a first object pool and a second object pool in a memory, where the capacity of a first object in the first object pool is smaller than the capacity of a second object in the second object pool; therefore, when the first data in the permanent storage device is accessed and needs to be loaded, a first object can be selected from the first object pool as a first target object, and part or all of fields of the first data are cached (i.e. loaded) in the first target object; and when the first target object is filled with a part of fields of the first data and other fields of the first data are accessed, a second object can be selected from the second object pool as a second target object, and all fields of the first data are cached in the second target object. When certain data is needed to be loaded, the first object with smaller capacity is allocated first, and if loading requirements exist on the unloaded part of the data in the first object, the data can be upgraded to the second object with larger capacity, so that the memory can be reasonably allocated for loading the data, the memory waste caused by uniformly allocating the objects with larger capacity is avoided, meanwhile, the memory fragments are avoided from being generated due to the preset of the object pool, and the utilization rate of the memory is further improved.
In some embodiments of the present description, when new data is created within a database, such as metadata for a new data fragment, the newly created data may be loaded into a memory resource in a manner as shown in FIG. 4.
First, in response to second data being created, the second data is saved to the persistent storage device. Wherein the second data may be metadata of a data fragment within the database, e.g., when a new data fragment is generated within the database, the metadata of the new data fragment is created; the persistent storage device may be a disk or the like.
Next, in response to the second data having a capacity less than or equal to the capacity of the first object, determining a fourth target object within the first object pool in memory, and caching all fields of the second data within the fourth target object; and in response to the capacity of the second data being greater than the capacity of the first object, determining a fifth target object in a second object pool in the memory, and caching all fields of the second data in the fifth target object, wherein the fifth target object is one second object in the second object pool. Wherein the fourth target object is a first object in the first object pool. For example, in the case where the first object has a capacity of 4kB and the second object has a capacity of 64kB, it may be determined whether the second data is less than or equal to 4kB, if the second data is less than or equal to 4kB, all fields of the second data are loaded to the fourth target object, if the second data is greater than 4kB, it may be further determined whether the second data has a capacity of less than or equal to 64kB, and if the second data is less than or equal to 64kB, all fields of the second data are loaded to the fifth target object.
It will be appreciated that the manner of determining the fourth target object in the first object pool may be the same as or different from the manner of determining the first target object in step S101; the manner of determining the fifth target object in the second object pool may be the same as or different from the manner of determining the second target object in step S102.
In this embodiment, when the second data is created, a suitable memory object is selected according to the capacity size of the second data for loading, so that the newly created data is conveniently accessed, and the efficiency of data access is improved.
In the above embodiments, in the scenario where the memory object is required to load data, if there is an idle memory object, the data loading is more convenient, and if there is no idle memory object, the data in one memory object needs to be eliminated to provide an idle memory object. In order to improve the efficiency of data loading and avoid occupying memory resources by the data with low heat, the memory objects can be eliminated in real time in the process of memory management in the manner shown in fig. 5, so that the existence probability of the memory objects in an idle state is improved.
Step S501: traversing the linked list. For example, the first and second phase-out linked lists are traversed separately.
Step S502: and determining that the first access heat score value is negative or the access time is greater than 5s from the current time.
Step S503: it is determined whether the memory object determined in step S502 is a second object (e.g., a large object).
Step S504: attempts to eliminate the object. For example, data loaded within a first object (e.g., a normal object) is deleted.
Step S505: attempts to demote the object. For example, a portion of the data loaded within the second object is dumped to the first object in the idle state.
Fig. 6 is a schematic block diagram of an apparatus provided in an exemplary embodiment. Referring to fig. 6, at the hardware level, the device includes a processor 602, an internal bus 604, a network interface 606, a memory 608, and a non-volatile storage 610, although other tasks may be performed. One or more embodiments of the present description may be implemented in a software-based manner, such as by the processor 602 reading a corresponding computer program from the non-volatile memory 610 into the memory 608 and then running. Of course, in addition to software implementation, one or more embodiments of the present disclosure do not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or a logic device.
Referring to fig. 7, the memory management apparatus may be applied to the device shown in fig. 6 to implement the technical solution of the present specification. The device comprises:
a first caching module 701, configured to determine a first target object in a first object pool in a memory in response to first data in a persistent storage device being accessed, and cache a part or all of fields of the first data in the first target object, where the first object pool includes a plurality of first objects with the same capacity, and the first target object is one first object in the first object pool;
and a second buffer module 702, configured to determine a second target object in a second object pool in a memory in response to the first target object being filled with a part of the fields of the first data and other fields of the first data being accessed, and buffer all fields of the first data in the second target object, where the second object pool includes a plurality of second objects with the same capacity, and the capacity of the second object is greater than that of the first object, and the second target object is one second object in the second object pool.
In one embodiment of the present specification, the apparatus further comprises a scoring module for:
Determining an access heat score value of a first object according to an access source and access time in response to the first object in the first object pool being accessed;
the first cache module is configured to, when determining a first target object in a first object pool in the memory, specifically:
and in response to the first object in the first object pool without the idle state, determining a first target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, and deleting the data cached in the first target object.
In one embodiment of the present specification, the apparatus further comprises a scoring module for:
determining an access heat score value of a second object according to an access source and access time in response to the second object in the second object pool being accessed;
the second cache module is configured to, when determining a second target object in a second object pool in the memory, specifically:
and in response to the second object in the second object pool without the idle state, determining a second target object in at least one second object in the second object pool according to the access heat scoring value and the access time of the at least one second object, and transferring part of data cached in the second target object to the first object in the first object pool.
In one embodiment of the present disclosure, the second buffer module is configured to, when transferring the part of the data buffered in the second target object to the first object in the first object pool, specifically:
and transferring partial data with highest access heat grading value cached in the first target object to a first object in the first object pool.
In one embodiment of the present disclosure, the second buffer module is configured to, when transferring the part of the data buffered in the second target object to the first object in the first object pool, specifically:
and responding to the first object in the first object pool without the idle state, determining a third target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, deleting the data cached in the third target object, and transferring part of the data cached in the second target object to the third target object, wherein the third target object is one first object in the first object pool.
In one embodiment of the present specification, the first buffer module is further configured to: responding to the failure of deleting the data cached in the first target object, and re-determining the first target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the first target object is successfully deleted; and/or the number of the groups of groups,
The second cache module is further configured to: responding to the data transfer failure of the cache in the second target object, and re-determining the second target object according to the access heat scoring value and the access time of at least one second object in the second object pool until the data transfer of the cache in the second target object is successful; and/or the number of the groups of groups,
the second cache module is further configured to: and responding to the failure of deleting the data cached in the third target object, and re-determining the third target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the third target object is successfully deleted.
In one embodiment of the present disclosure, the scoring module is configured to, when determining the access heat score value of the first object according to the access source and the access time, specifically:
determining an access popularity grading system of the first object according to the user priority and the access time under the condition that the access source is a user;
when the access source is a system and the first object is not accessed by a user, determining an access heat scoring value of the first object according to the system priority and the access time;
When the access source is a system and the first object is accessed by a user, maintaining the access heat scoring value of the first object unchanged; and/or the number of the groups of groups,
the scoring module is used for determining the access heat scoring value of the second object according to the access source and the access time, and is specifically used for:
determining an access heat scoring value of the second object according to the user priority and the access time under the condition that the access source is a user;
when the access source is a system and the second object is not accessed by a user, determining an access heat scoring value of the second object according to the system priority and the access time;
and when the access source is a system and the second object is accessed by a user, keeping the access heat scoring value of the second object unchanged.
In one embodiment of the present disclosure, the first buffer module is configured to, when determining a first target object in at least one first object in the first object pool according to an access heat score value and an access time of the at least one first object, specifically be:
determining whether the access heat degree score value of each first object in the first object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a first object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a first target object;
The second cache module is configured to determine, according to the access heat score value and the access time of at least one second object in the second object pool, a second target object in the at least one second object, where the second target object is specifically configured to:
and determining whether the access heat degree score value of each second object in the second object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a second object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a second target object.
In one embodiment of the present disclosure, the second buffer module is specifically configured to:
and in response to the first target object being filled with a part of fields of the first data, other fields of the first data being accessed, and the access frequency of the other fields of the first data being higher than a preset frequency threshold, determining a second target object in a second object pool in the memory, and caching all fields of the first data in the second target object.
In one embodiment of the present specification, the apparatus further comprises an allocation module for:
In response to second data being created, saving the second data to the persistent storage device;
determining a fourth target object in a first object pool in a memory in response to the capacity of the second data being smaller than or equal to the capacity of the first object, and caching all fields of the second data in the fourth target object, wherein the fourth target object is one first object in the first object pool;
and in response to the capacity of the second data being greater than the capacity of the first object, determining a fifth target object in a second object pool in the memory, and caching all fields of the second data in the fifth target object, wherein the fifth target object is one second object in the second object pool.
In one embodiment of the present specification, the first data and the second data are metadata of a data slice.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or a combination of any of these devices.
In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, read only compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by the computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of related data is required to comply with the relevant laws and regulations and standards of the relevant country and region, and is provided with corresponding operation entries for the user to select authorization or rejection.
It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present description to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
The foregoing description of the preferred embodiment(s) is (are) merely intended to illustrate the embodiment(s) of the present invention, and it is not intended to limit the embodiment(s) of the present invention to the particular embodiment(s) described.

Claims (14)

1. A memory management method, the method comprising:
determining a first target object in a first object pool in a memory in response to first data in a permanent storage device being accessed, and caching part or all of fields of the first data in the first target object, wherein the first object pool comprises a plurality of first objects with the same capacity, and the first target object is one first object in the first object pool;
and in response to the first target object being filled with a part of fields of the first data and other fields of the first data being accessed, determining a second target object in a second object pool in a memory, and caching all fields of the first data in the second target object, wherein the second object pool comprises a plurality of second objects with the same capacity, the capacity of the second object is larger than that of the first object, and the second target object is one second object in the second object pool.
2. The memory management method of claim 1, the method further comprising:
determining an access heat score value of a first object according to an access source and access time in response to the first object in the first object pool being accessed;
the determining a first target object in the first object pool in the memory includes:
and in response to the first object in the first object pool without the idle state, determining a first target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, and deleting the data cached in the first target object.
3. The memory management method of claim 2, the method further comprising:
determining an access heat score value of a second object according to an access source and access time in response to the second object in the second object pool being accessed;
determining a second target object in the second object pool in the memory, including:
and in response to the second object in the second object pool without the idle state, determining a second target object in at least one second object in the second object pool according to the access heat scoring value and the access time of the at least one second object, and transferring part of data cached in the second target object to the first object in the first object pool.
4. The memory management method according to claim 3, wherein the transferring the part of the data cached in the second target object to the first object in the first object pool includes:
and transferring partial data with highest access heat grading value cached in the first target object to a first object in the first object pool.
5. The memory management method of claim 3, the method further comprising: the transferring the part of the data cached in the second target object to the first object in the first object pool includes:
and responding to the first object in the first object pool without the idle state, determining a third target object in at least one first object in the first object pool according to the access heat scoring value and the access time of the at least one first object, deleting the data cached in the third target object, and transferring part of the data cached in the second target object to the third target object, wherein the third target object is one first object in the first object pool.
6. The memory management method of claim 5, the method further comprising:
Responding to the failure of deleting the data cached in the first target object, and re-determining the first target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the first target object is successfully deleted; and/or the number of the groups of groups,
responding to the data transfer failure of the cache in the second target object, and re-determining the second target object according to the access heat scoring value and the access time of at least one second object in the second object pool until the data transfer of the cache in the second target object is successful; and/or the number of the groups of groups,
and responding to the failure of deleting the data cached in the third target object, and re-determining the third target object according to the access heat scoring value and the access time of at least one first object in the first object pool until the data cached in the third target object is successfully deleted.
7. The memory management method according to claim 3, wherein the determining the access heat score value of the first object according to the access source and the access time includes:
determining an access popularity grading system of the first object according to the user priority and the access time under the condition that the access source is a user;
When the access source is a system and the first object is not accessed by a user, determining an access heat scoring value of the first object according to the system priority and the access time;
when the access source is a system and the first object is accessed by a user, maintaining the access heat scoring value of the first object unchanged; and/or the number of the groups of groups,
the determining the access heat scoring value of the second object according to the access source and the access time comprises the following steps:
determining an access heat scoring value of the second object according to the user priority and the access time under the condition that the access source is a user;
when the access source is a system and the second object is not accessed by a user, determining an access heat scoring value of the second object according to the system priority and the access time;
and when the access source is a system and the second object is accessed by a user, keeping the access heat scoring value of the second object unchanged.
8. The memory management method according to claim 3, wherein the determining a first target object in at least one first object in the first object pool according to the access heat score value and the access time of the at least one first object comprises:
Determining whether the access heat degree score value of each first object in the first object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a first object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a first target object;
the determining a second target object in at least one second object in the second object pool according to the access heat scoring value and the access time of the at least one second object comprises the following steps:
and determining whether the access heat degree score value of each second object in the second object pool is smaller than a first score threshold value, whether the access time is greater than a first time threshold value or not, and determining a second object with the first access heat degree score value smaller than the first score threshold value or the access time is greater than the first time threshold value as a second target object.
9. The memory management method of claim 1, wherein responsive to the first target object being filled with a portion of the fields of the first data and other fields of the first data being accessed, determining a second target object within a second object pool in memory and caching all of the fields of the first data within the second target object, comprises:
And in response to the first target object being filled with a part of fields of the first data, other fields of the first data being accessed, and the access frequency of the other fields of the first data being higher than a preset frequency threshold, determining a second target object in a second object pool in the memory, and caching all fields of the first data in the second target object.
10. The memory management method of claim 1, the method further comprising:
in response to second data being created, saving the second data to the persistent storage device;
determining a fourth target object in a first object pool in a memory in response to the capacity of the second data being smaller than or equal to the capacity of the first object, and caching all fields of the second data in the fourth target object, wherein the fourth target object is one first object in the first object pool;
and in response to the capacity of the second data being greater than the capacity of the first object, determining a fifth target object in a second object pool in the memory, and caching all fields of the second data in the fifth target object, wherein the fifth target object is one second object in the second object pool.
11. The memory management method according to any one of claims 1 to 10, wherein the first data and the second data are metadata of a data slice.
12. A memory management apparatus, the apparatus comprising:
the first caching module is used for responding to that first data in the permanent storage device are accessed, determining a first target object in a first object pool in a memory, and caching part or all fields of the first data in the first target object, wherein the first object pool comprises a plurality of first objects with the same capacity, and the first target object is one first object in the first object pool;
and the second caching module is used for responding to the fact that the first target object is filled with part of fields of the first data, other fields of the first data are accessed, determining a second target object in a second object pool in a memory, and caching all fields of the first data in the second target object, wherein the second object pool comprises a plurality of second objects with the same capacity, the capacity of the second object is larger than that of the first object, and the second target object is one second object in the second object pool.
13. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any of claims 1-11 by executing the executable instructions.
14. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any of claims 1-11.
CN202310764892.1A 2023-06-26 2023-06-26 Memory management method and device, electronic equipment and storage medium Pending CN116860439A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310764892.1A CN116860439A (en) 2023-06-26 2023-06-26 Memory management method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310764892.1A CN116860439A (en) 2023-06-26 2023-06-26 Memory management method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116860439A true CN116860439A (en) 2023-10-10

Family

ID=88220847

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310764892.1A Pending CN116860439A (en) 2023-06-26 2023-06-26 Memory management method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116860439A (en)

Similar Documents

Publication Publication Date Title
US10671290B2 (en) Control of storage of data in a hybrid storage system
US10114908B2 (en) Hybrid table implementation by using buffer pool as permanent in-memory storage for memory-resident data
US20200257450A1 (en) Data hierarchical storage and hierarchical query method and apparatus
US11372568B2 (en) System and method for storing and accessing blockchain data
US11586629B2 (en) Method and device of storing data object
US11314689B2 (en) Method, apparatus, and computer program product for indexing a file
US9966152B2 (en) Dedupe DRAM system algorithm architecture
US10409728B2 (en) File access predication using counter based eviction policies at the file and page level
US10956072B2 (en) Accelerating concurrent access to a file in a memory-based file system
US9727479B1 (en) Compressing portions of a buffer cache using an LRU queue
US20130290636A1 (en) Managing memory
JP2005276198A (en) System and method for balancing multiple memory buffer sizes
CN113590045B (en) Data hierarchical storage method, device and storage medium
US11836087B2 (en) Per-process re-configurable caches
US20210200463A1 (en) Method and system for memory swap
US20210089442A1 (en) Dynamically allocating memory pool subinstances
CN112148736A (en) Method, device and storage medium for caching data
CN115168304A (en) Data processing method, device, storage medium and equipment
CN109582233A (en) A kind of caching method and device of data
KR102321346B1 (en) Data journaling method for large solid state drive device
CN111475099A (en) Data storage method, device and equipment
CN113434285A (en) Key value cache system-based memory management method and device
CN116027982A (en) Data processing method, device and readable storage medium
CN116860439A (en) Memory management method and device, electronic equipment and storage medium
CN115509437A (en) Storage system, network card, processor, data access method, device and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination