WO2024259951A1 - 内存数据的迁移方法、装置、非易失性可读存储介质及电子装置 - Google Patents

内存数据的迁移方法、装置、非易失性可读存储介质及电子装置 Download PDF

Info

Publication number
WO2024259951A1
WO2024259951A1 PCT/CN2024/070674 CN2024070674W WO2024259951A1 WO 2024259951 A1 WO2024259951 A1 WO 2024259951A1 CN 2024070674 W CN2024070674 W CN 2024070674W WO 2024259951 A1 WO2024259951 A1 WO 2024259951A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
memory space
target
usage
space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/070674
Other languages
English (en)
French (fr)
Inventor
王云
王兴隆
李金锋
王海梦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co Ltd filed Critical Suzhou Metabrain Intelligent Technology Co Ltd
Priority to US19/137,733 priority Critical patent/US20260111137A1/en
Publication of WO2024259951A1 publication Critical patent/WO2024259951A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/0223User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023Free address space management
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604Improving or facilitating administration, e.g. storage management
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608Saving storage space on storage systems
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • G06F3/0631Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647Migration mechanisms
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0679Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022Mechanisms to release resources
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the embodiments of the present application relate to the field of computers, and in particular, to a method and device for migrating memory data, a non-volatile readable storage medium, and an electronic device.
  • CXL Computer Express Link
  • Embodiments of the present application provide a method and device for migrating memory data, a non-volatile readable storage medium, and an electronic device to at least solve the problem of low efficiency of memory use in the related art.
  • a method for migrating memory data comprising:
  • the target memory space is a memory space allocated for a target application for memory data to be migrated, the target memory space includes a first memory space of a first type and a second memory space of a second type, the memory usage parameter is used to indicate a frequency distribution of use of the target memory space, and an operation efficiency of the target application on the first memory space is higher than an operation efficiency of the target application on the second memory space;
  • the target migration parameter is determined according to an allocation parameter of the target memory space, the allocation parameter is used to indicate a proportion of the first memory space and the second memory space in the target memory space, the target migration parameter is used to indicate a proportion of memory space in the target memory space that is allowed to migrate memory data, and the third memory space is a memory space in the second memory space whose frequency of use is higher than a first frequency threshold;
  • the first memory data stored in the third memory space is migrated to the first memory space.
  • selecting a third memory space from the second memory space according to a memory usage parameter and a target migration parameter includes:
  • Memory units belonging to the second memory space are extracted from the first memory unit set to obtain a third memory space.
  • the method before extracting memory cells satisfying a first ratio from a plurality of memory cells according to the usage parameter from high to low as the first memory cell set, the method further includes:
  • a first proportion is determined according to the allocation parameter.
  • obtaining allocation parameters of the target memory space includes: obtaining a first allocation ratio of the first memory space in the target memory space, wherein the allocation parameters include the first allocation ratio;
  • Determining the first proportion according to the allocation parameter includes: determining the difference between the first allocation proportion and the first preset proportion as the first proportion.
  • selecting a third memory space from the second memory space according to a memory usage parameter and a target migration parameter includes:
  • the memory cells satisfying the second ratio in the first part are screened out from low to high to obtain a second memory cell set, wherein the target migration parameters include the target ratio and the second ratio;
  • Memory units belonging to the second memory space are extracted from the second memory unit set to obtain a third memory space.
  • the method before dividing the plurality of memory cells into the first part and the second part according to the usage parameter from high to low, the method further includes:
  • the target proportion and the second proportion are determined according to the allocation parameters.
  • obtaining allocation parameters of the target memory space includes: obtaining a first allocation ratio of the first memory space in the target memory space, and a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the first allocation ratio and the second allocation ratio;
  • Determining the target ratio and the second ratio according to the allocation parameters includes: determining the first allocation ratio and the second allocation ratio as the target ratio; and obtaining the second preset ratio as the second ratio.
  • the method further includes:
  • the second memory data stored in the fourth memory space is migrated to the second memory space.
  • selecting a fourth memory space from the first memory space according to a memory usage parameter and a target migration parameter includes:
  • Memory units belonging to the first memory space are extracted from the third memory unit set to obtain a fourth memory space.
  • the method before extracting memory cells satisfying a third ratio from a plurality of memory cells according to the usage parameter from low to high as a third memory cell set, the method further includes:
  • the third proportion is determined according to the allocation parameters.
  • obtaining allocation parameters of the target memory space includes: obtaining a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the second allocation ratio;
  • Determining the third proportion according to the allocation parameter includes: determining the difference between the second allocation proportion and the second preset proportion as the third proportion.
  • selecting a fourth memory space from the first memory space according to a memory usage parameter and a target migration parameter includes:
  • the memory units satisfying the fourth ratio in the second part are screened out from high to low according to the usage parameters to obtain a fourth memory unit set, wherein the target migration parameters include the target ratio and the fourth ratio;
  • Memory units belonging to the first memory space are extracted from the fourth memory unit set to obtain a fourth memory space.
  • the method before dividing the plurality of memory cells into the first part and the second part according to the usage parameter from high to low, the method further includes:
  • the target ratio and the fourth ratio are determined based on the allocation parameters.
  • obtaining allocation parameters of the target memory space includes: obtaining a first allocation ratio of the first memory space in the target memory space, and a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the first allocation ratio and the second allocation ratio;
  • Determining the target ratio and the fourth ratio according to the allocation parameters includes: determining the first allocation ratio and the second allocation ratio as the target ratio; and obtaining the fourth preset ratio as the fourth ratio.
  • migrating first memory data stored in a third memory space to the first memory space includes:
  • detecting a memory usage parameter of a target memory space includes:
  • the usage parameter corresponding to the target memory unit is calculated according to one or more usage frequencies, and the memory unit and the usage parameter having a corresponding relationship are obtained as the memory usage parameter.
  • calculating a usage parameter corresponding to a target memory unit according to one or more usage frequencies includes:
  • a weighted sum of one or more usage frequencies is calculated as a usage parameter corresponding to the target memory unit.
  • the method before detecting the memory usage parameter of the target memory space, the method further includes:
  • the initial memory space is adjusted to obtain the target memory space.
  • adjusting the initial memory space according to the relationship between the usage rate and the usage rate threshold to obtain the target memory space includes:
  • the usage rate is greater than or equal to the first usage rate threshold, allocating an additional memory space of a first capacity to the target application to obtain a target memory space, wherein the target memory space includes an initial memory space and an additional memory space;
  • the memory space of the second capacity is released from the target memory space to obtain the target memory space.
  • adjusting the initial memory space includes:
  • the initial memory space is adjusted using the first memory tool
  • a second memory tool is used to adjust the initial memory space, wherein the adjustment efficiency of the second memory tool for the memory space is lower than the adjustment efficiency of the first memory tool for the memory space, but the operating load of the second memory tool is less than the operating load of the first memory tool.
  • a device for migrating memory data comprising:
  • a first detection module configured to detect a memory usage parameter of a target memory space, wherein the target memory space is a memory space allocated for a target application for memory data to be migrated, the target memory space includes a first memory space of a first type and a second memory space of a second type, the memory usage parameter is used to indicate a frequency distribution of use of the target memory space, and an operation efficiency of the target application on the first memory space is higher than an operation efficiency of the target application on the second memory space;
  • a first screening module is configured to screen out a third memory space from the second memory space according to a memory usage parameter and a target migration parameter, wherein the target migration parameter is determined according to an allocation parameter of the target memory space, the allocation parameter is used to indicate a proportion of the first memory space and the second memory space in the target memory space, the target migration parameter is used to indicate a proportion of memory space in the target memory space that is allowed to migrate memory data, and the third memory space is a memory space in the second memory space whose frequency of use is higher than a first frequency threshold;
  • the first migration module is configured to migrate the first memory data stored in the third memory space to the first memory space.
  • a computer non-volatile readable storage medium in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
  • an electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
  • the target memory space used by the target application includes the first memory space and the second memory space. According to the frequency distribution of the target memory space used and the allocation parameters of the target memory space, the memory space with high frequency of use in the second memory space is screened out from the target memory space, that is, the third memory space that needs to be migrated. Since the target application's operating efficiency on the first memory space is higher than the target application's operating efficiency on the second memory space, by migrating the first memory data stored in the part of the memory space with high use efficiency in the second memory space to the first memory space, the target application's operating efficiency on the memory data with high use efficiency is improved, thereby improving the operating efficiency of the target application. Therefore, the problem of low memory use efficiency can be solved, and the effect of improving the efficiency of memory use can be achieved.
  • FIG1 is a hardware structure block diagram of a mobile terminal according to a method for migrating memory data in an embodiment of the present application
  • FIG2 is a flow chart of a method for migrating memory data according to an embodiment of the present application.
  • FIG3 is a schematic diagram of a memory usage parameter determination process according to an embodiment of the present application.
  • FIG. 4 is a flow chart of allocating initial memory space for a target application according to an embodiment of the present application
  • FIG5 is a flow chart of dynamically allocating memory space according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of screening a fourth memory space from a first memory space according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of screening a third memory space from a second memory space according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of performing memory data migration in a first memory space and a second memory space according to an embodiment of the present application
  • FIG. 9 is a structural block diagram of a memory data migration device according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of an electronic device according to an embodiment of the present application.
  • FIG1 is a hardware structure block diagram of a mobile terminal according to a method for migrating memory data in an embodiment of the present application.
  • the mobile terminal may include one or more (only one is shown in FIG1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 configured to store data, wherein the mobile terminal may also include a transmission device 106 configured to have a communication function and an input/output device 108.
  • processors 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA
  • a transmission device 106 configured to have a communication function and an input/output device 108.
  • FIG1 is only for illustration and does not limit the structure of the mobile terminal.
  • the mobile terminal may also include more or fewer components than those shown in FIG1 , or have a configuration different from that shown in FIG1 .
  • the memory 104 may be configured to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the migration method of memory data in the embodiment of the present application.
  • the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method.
  • the memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory 104 may include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the transmission device 106 is configured to receive or send data via a network.
  • the above network examples may include a wireless network provided by a communication provider of the mobile terminal.
  • the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet.
  • the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is configured to communicate with the Internet wirelessly.
  • RF Radio Frequency
  • FIG. 2 is a flow chart of a method for migrating memory data according to an embodiment of the present application. As shown in FIG. 2 , the process includes the following steps:
  • Step S202 detecting a memory usage parameter of a target memory space, wherein the target memory space is a memory space allocated for a target application of memory data to be migrated, the target memory space includes a first memory space of a first type and a second memory space of a second type, the memory usage parameter is used to indicate a frequency distribution of use of the target memory space, and an operation efficiency of the target application on the first memory space is higher than an operation efficiency of the target application on the second memory space;
  • Step S204 selecting a third memory space from the second memory space according to the memory usage parameter and the target migration parameter, wherein the target migration parameter is determined according to the allocation parameter of the target memory space, the allocation parameter is used to indicate the proportion of the first memory space and the second memory space in the target memory space, the target migration parameter is used to indicate the proportion of the memory space in the target memory space that is allowed to migrate memory data, and the third memory space is a memory space in the second memory space whose frequency of use is higher than the first frequency threshold;
  • Step S206 Migrate the first memory data stored in the third memory space to the first memory space.
  • the target memory space used by the target application includes the first memory space and the second memory space. According to the frequency distribution of the target memory space used and the allocation parameters of the target memory space, the memory space with high frequency of use in the second memory space is screened out from the target memory space, that is, the third memory space that needs to be migrated. Since the target application's operating efficiency on the first memory space is higher than the target application's operating efficiency on the second memory space, by migrating the first memory data stored in the part of the memory space with high use efficiency in the second memory space to the first memory space, the target application's operating efficiency on the memory data with high use efficiency is improved, thereby improving the operating efficiency of the target application. Therefore, the problem of low memory use efficiency can be solved, and the effect of improving memory use efficiency can be achieved.
  • the above memory data migration method can be but is not limited to realizing memory expansion of the server hardware system through the CXL protocol.
  • the server hardware system realizes remote sharing of non-local DRAM (Dynamic Random Access Memory) and shared IO devices through the PCIe 5 protocol (Peripheral Component Interconnect Express 5, a high-speed serial computer expansion bus standard (fifth generation)), expands the data transmission method between the processor and DDR DRAM (Double Data Rate Dynamic Random Access Memory) in the computer, and realizes remote shared memory to provide the system with higher memory capacity and performance.
  • PCIe 5 protocol Peripheral Component Interconnect Express 5
  • DDR DRAM Double Data Rate Dynamic Random Access Memory
  • the memory data migration method may be applied to, but not limited to, a server hardware system supporting the CXL protocol
  • the server hardware system may include, but not limited to, a CPU (Central Processing Unit) supporting the CXL protocol, a chipset supporting the CXL protocol, a hardware device supporting the CXL protocol, and an operating system supporting the CXL protocol, etc.
  • the CXL protocol may include, but not limited to, CXL1.1 protocol, CXL1.0 protocol, CXL2.0 protocol, etc.
  • the above memory data migration method can be applied to, but not limited to, operating systems that support the CXL protocol, such as: Linux system (GNU/Linux), Windows operating system (Microsoft Windows, an operating system developed by Microsoft Corporation based on a graphical user interface), VMware ESXi system (a virtualization software that supports Windows platform operation), etc.
  • Linux system GNU/Linux
  • Windows operating system Microsoft Windows, an operating system developed by Microsoft Corporation based on a graphical user interface
  • VMware ESXi system a virtualization software that supports Windows platform operation
  • the Linux system can manage the extended CXL memory in, but not limited to, the following ways:
  • the target application may be, but is not limited to, running in a server hardware system supporting the CXL protocol.
  • the server hardware system supporting the CXL protocol is capable of running one or more target applications.
  • One or more applications running in the server hardware system supporting the CXL protocol may be, but is not limited to, being determined as target applications.
  • the target memory space can be obtained by allocating memory spaces from two different sources to the target application at the stage of system startup, but is not limited to.
  • the target application's memory space can be adjusted in real time according to the target application's demand for memory space during the running of the target application, but is not limited to, so as to obtain a new target memory space.
  • the first memory space belonging to the first type and the second memory space belonging to the second type can be connected to the system where the target application is located in two different ways, but not limited to, and the first type and the second type are used to indicate that the ways in which the first memory space and the second memory space are connected to the system where the target application is located are different.
  • the first memory space belonging to the first type may be, but is not limited to, the local memory of the system where the target application is located.
  • the second memory space belonging to the second type may be, but is not limited to, the external memory of the system where the target application is located, such as CXL memory, and may be, but is not limited to, using the remote memory as the local memory through CXL technology.
  • CXL memory the external memory of the system where the target application is located
  • CXL memory the external memory of the system where the target application is located
  • the remote memory such as CXL memory
  • the external memory due to the physical distance between the closer CXL memory, i.e., the host processor and the remote memory, it can provide faster data transmission speed and lower latency, while the farther CXL memory may produce higher access latency and slower data transmission speed. Therefore, the target application's operating efficiency for the first memory space located locally is higher than the target application's operating efficiency for the farther second memory space.
  • the second type of second memory space mentioned above can be connected to the system where the target application is located in the following manner but not limited to: the CXL memory (i.e., the second memory space) is connected to the system server through the PCIe interface, and after the server is turned on, the BIOS (Basic Input Output System) recognizes the CXL memory and records the relevant CXL memory mapping address and attributes to the SRAT table (Static Resource Affinity Table), and records the NUMA node distance.
  • the CXL memory i.e., the second memory space
  • the BIOS Basic Input Output System
  • the target memory space may include but is not limited to multiple memory pages, data storage operations may be performed on the memory pages, and the first memory space of the first type and the second memory space of the second type may include but are not limited to multiple memory pages of different sizes, such as a memory page of 2G, a memory page of 1M, a memory page of 2KB, etc.
  • the target application may be allocated memory pages of corresponding sizes and quantities according to the amount of data that the target application needs to store, but is not limited to.
  • the memory usage parameter of the target memory space is used to indicate the frequency distribution of the target memory space being used.
  • the memory usage parameter of the target memory space can be determined by, but is not limited to, the following method: taking each memory page in the target memory space as a unit, detecting the number of times each memory page in the target memory space is used within a period of time, and then sorting the memory pages according to the number of times each memory page is used within a period of time, obtaining memory pages in the target memory space that are distributed in a high frequency interval, memory pages in the target memory space that are distributed in a low frequency interval, and memory pages in the target memory space that are distributed in a relatively stable frequency interval, and determining the interval obtained according to the number of times used as the memory usage parameter of the target memory space.
  • the memory usage parameters of the target space can be detected in the following manner but is not limited to: collecting the usage frequency of the target memory unit in each time period in one or more time periods to obtain one or more usage frequencies, wherein the target memory unit is each memory unit among multiple memory units that have stored data in the target memory space; calculating the usage parameters corresponding to the target memory unit based on the one or more usage frequencies to obtain memory units and usage parameters with corresponding relationships as memory usage parameters.
  • the target application may include but is not limited to multiple memory units, and a memory page for storing data may be determined as the above memory unit, but is not limited to.
  • a memory page storing data in the target application may be determined as the target memory unit, but is not limited to.
  • the number of times a memory page storing data in a target application is used in each of one or more time periods may be determined as the usage frequency of the target memory unit, but is not limited to, for example, the target application includes N memory pages storing data, and the N memory pages storing data are used M times in one time period, so M times is determined as the usage frequency of the target memory unit.
  • the target application includes N memory pages storing data, and the N memory pages storing data are used M1...MP times in each of P time periods, so M1 times...MP times are all determined as the usage frequency of the target memory unit.
  • the usage parameter corresponding to the target memory unit can be determined according to, but not limited to, the usage frequency of the target memory unit.
  • the target application includes N memory pages storing data, and the N memory pages storing data are used M times in a time period, so M times are determined as the usage frequency of the target memory unit, and the corresponding usage parameter M is determined according to the usage frequency M.
  • the target application includes N memory pages storing data, and the N memory pages storing data are used M1...MP times in each of P time periods, so M1 times...MP times are determined as the usage frequency of the target memory unit, and the corresponding usage parameter is determined according to the usage frequencies M1...MP as the sum of M1...MP.
  • the usage parameter corresponding to the target memory unit can be calculated based on one or more usage frequencies in the following manner but is not limited to: assigning a weight to each time period based on the distance between each time period and the current time, wherein the smaller the distance between each time period and the current time, the greater the weight corresponding to each time period; calculating the weighted sum of one or more usage frequencies as the usage parameter corresponding to the target memory unit.
  • the weight of each time period can be determined based on but not limited to the distance between each time period and the current time.
  • the target application includes N memory pages storing data, and the N memory pages storing data are used M1, M2 and M3 times in time periods P1, P2 and P3 respectively, and their corresponding weights are T1, T2 and T3. Since time period P1 is closest to the current time, the distance between time period P2 and the current time is longer than the distance between time period P1 and the current time, and time period P3 is the farthest from the current time, so T1>T2>T3.
  • FIG3 is a schematic diagram of a process for determining memory usage parameters according to an embodiment of the present application.
  • the memory usage parameters of the target memory space may be determined in the following manners, but are not limited to:
  • Collect the usage frequency of the target memory unit within 180 seconds from the current time determine the usage frequency of each time period of 30 seconds, and obtain the usage frequency P of each time period within 180 seconds from the current time, including: P30, P60, P90, P120, P150 and P180;
  • T30, T60, T90, T120, T150 and T180 Assign a weight T to each time period according to the distance between each time period and the current time: T30, T60, T90, T120, T150 and T180, where T30 is greater than T60, T60 is greater than T90, T90 is greater than T120, T120 is greater than T150, and T150 is greater than T180;
  • the target memory space before detecting the memory usage parameters of the target memory space, can be obtained in the following manner but is not limited to: detecting the usage rate of the initial memory space allocated to the target application; adjusting the initial memory space according to the relationship between the usage rate and the usage rate threshold to obtain the target memory space.
  • the system where the target application is located may, but is not limited to, automatically allocate initial memory space for the target application during the startup phase, and the initial memory space may, but is not limited to, include memory space belonging to the first type and memory space belonging to the second type.
  • the initial memory space of the target application may be, but is not limited to, a specific size, for example, the initial memory space may be, but is not limited to, a first type of memory space of size M and a second type of memory space of size M.
  • a first type of memory space of size M and a second type of memory space of size N where M and N are natural numbers greater than or equal to zero and M is not equal to N.
  • the initial memory space of the above-mentioned target application can be, but is not limited to, allocating appropriate initial memory space for it based on the historical memory occupancy of the target application, for example: based on the maximum occupancy of the first type of memory space and the second type of memory space when the target application is used for P times recently, the initial memory space is allocated to the target application according to the median of its maximum occupancy.
  • the usage rate of the initial memory space can be determined by, but not limited to, the memory space occupation by the target application, for example, by managing the memory through a Bitmap and tracking whether the memory blocks in the target application are in use or free state.
  • the usage rate of the initial memory space may be determined based on, but not limited to, whether the initial memory space demand of the target application exceeds the initial memory space.
  • the usage rate of the initial memory space may be determined based on, but not limited to, whether the initial memory space demand of the target application is much smaller than the initial memory space.
  • the above-mentioned usage rate threshold may be, but is not limited to, predetermined, or modified in real time according to the memory situation of the target application.
  • the above-mentioned usage rate threshold may be, but is not limited to, used to determine whether the occupation of the initial memory space by the above-mentioned target application is in a balanced state, for example: when the usage rate of the initial memory space of the target application exceeds the usage rate threshold, it is determined that the target application's demand for the initial memory space is greater than the capacity of the initial memory space, which is used to indicate that an expansion operation needs to be performed on the initial memory space.
  • the usage rate of the initial memory space of the target application is less than the usage rate threshold, it is determined that the target application's demand for the initial memory space is much less than the capacity of the initial memory space, which is used to indicate that an operation of reducing a certain content space can be performed on the initial memory space.
  • FIG4 is a flowchart of allocating initial memory space for a target application according to an embodiment of the present application.
  • the initial memory space for the target application may be allocated in the following manners, but is not limited to:
  • pre-allocated memory (initial memory space) is allocated for the target application according to the set fixed value, including local memory capacity (first type of memory space) and CXL memory capacity (second type of memory space).
  • the allocated memory is implemented in a red-black tree data structure, that is, the CXL memory with a short distance is used first.
  • the application table records the maximum usage of local memory and CXL memory for each application and calculates the median. In addition, the average number of cores used by each application is also recorded.
  • the memory pool table records the number and offset of size_class (indicates the size that needs to be aligned for the memory size allocated in the initial memory space) of each memory page in the initial memory space.
  • the allocated capacity is the median data of the maximum occupancy of the first type of memory space and the second type of memory space by the target application saved in the application table. If there is no median data, the default setting is 200MB of local memory and 200MB of CXL memory.
  • the initial memory space is adjusted according to the relationship between the usage rate and the usage rate threshold, and the target memory space can be obtained in the following manner but is not limited to: when the usage rate is greater than or equal to the first usage rate threshold, additional memory space of a first capacity is allocated to the target application to obtain the target memory space, wherein the target memory space includes the initial memory space and the additional memory space; when the usage rate is less than or equal to the second usage rate threshold, memory space of a second capacity is released from the target memory space to obtain the target memory space.
  • the first usage rate threshold may be, but is not limited to, predetermined, or modified in real time according to the memory situation of the target application.
  • the first usage threshold can be but is not limited to being used to determine whether the target application's occupancy of the initial memory space is in an overloaded state. For example, when the usage rate of the target application's initial memory space exceeds or is equal to the first usage threshold, it is determined that the target application's demand for the initial memory space is greater than the capacity of the initial memory space, and an expansion operation needs to be performed on the initial memory space.
  • the first capacity of the additional memory space may be, but is not limited to, determined according to the occupation of the initial memory space by the target application.
  • a fixed capacity of additional memory space is allocated to the target application, for example, when the usage rate is greater than or equal to the first usage rate threshold, 200 MB of additional memory space is allocated to the target application.
  • the additional memory space may be obtained from a memory space of the first type, or may be obtained from a memory space of the second type.
  • the second usage rate threshold may be, but is not limited to, predetermined, or modified in real time according to the memory situation of the target application.
  • the second usage rate threshold may be, but is not limited to, used to indicate the memory occupation situation of the target application within a certain period of time, for example, if the target application does not use part of the initial memory space within a certain period of time, it can be considered that the initial memory space is in an idle state.
  • the difference between the memory in the initial memory space in a certain period of time that is less than or equal to the total capacity of the initial memory space and the part of the memory space that is not used by the initial memory space within a certain period of time is determined as the second usage rate threshold.
  • the second usage threshold can be but is not limited to being used to determine whether the target application's occupancy of the initial memory space is in an idle state. For example, when the usage rate of the initial memory space of the target application is less than or equal to the second usage threshold, it is determined that the capacity of the initial memory space is much larger than the target application's demand for memory space, and an operation of reducing part of the initial memory space is required.
  • the second capacity may be, but is not limited to, determined based on the occupation of the initial memory space by the target application.
  • the fixed capacity memory space in the target application is reduced, for example, when the usage rate is less than or equal to the second usage rate threshold, 200MB of memory space is reduced from the target application.
  • FIG. 5 is a flowchart of dynamically allocating the memory space according to an embodiment of the present application.
  • the initial memory space of the target application can be dynamically allocated through, but not limited to, the following process, wherein the two groups of BitMaps are group A BitMap and group B BitMap respectively:
  • Group A BitMap detects in real time whether the memory is being used.
  • the application can flexibly allocate memory when running. It can, but is not limited to, monitor the memory occupancy in the memory pool in real time. For example, when the memory used in the memory pool (including the first type of memory space and the second type of memory space) exceeds 90% (the first usage threshold) of the corresponding capacity in the memory pool, the memory allocation API is called to increase the memory capacity by 200MB (the first capacity).
  • Group B BitMap manages the usage of memory within a certain period of time. If there is unused memory space in the memory pool within 5 minutes, and the occupied memory in the memory pool (including the first type of memory space and the second type of memory space) is less than or equal to the difference (second usage threshold) between the total capacity of the memory pool and 90% (second usage threshold) of the unused memory space within 5 minutes, the unused useless memory (second capacity) is released.
  • the initial memory space can be adjusted in the following manner but is not limited to: detecting the target number of processor cores used by the target application; when the target number is greater than or equal to a number threshold, using a first memory tool to adjust the initial memory space; when the target number is less than the number threshold, using a second memory tool to adjust the initial memory space, wherein the adjustment efficiency of the second memory tool for the memory space is lower than the adjustment efficiency of the first memory tool for the memory space, but the operating load of the second memory tool is less than the operating load of the first memory tool.
  • the quantity threshold may be, but is not limited to, pre-set and is used to select a tool for adjusting the memory space according to a target number of processor cores used by a target application.
  • the adjustment efficiency of the second memory tool for the memory space is lower than that of the first memory tool, but the running load of the second memory tool is smaller than that of the first memory tool.
  • the first memory tool may be, but is not limited to, tcmalloc allocator (an efficient memory allocator that can perform memory allocation and release operations more quickly and efficiently)
  • the second memory tool may be, but is not limited to, jemalloc allocator (a general, thread-safe memory allocator).
  • a process for adjusting the initial memory space is provided, which may be but is not limited to adjusting the initial memory space in the following manner: recording the average of the processor cores used by each application in a table, using tcmalloc allocator when the number of processor cores used by the application is not more than 3 (quantity threshold), using jemalloc allocator when the number of processor cores used by the application is more than 3, and using tcmalloc allocator by default if there is no record in the application table.
  • the following methods can be used but are not limited to: selecting a fourth memory space from the first memory space based on the memory usage parameters and the target migration parameters, wherein the fourth memory space is a memory space in the first memory space whose usage frequency is lower than a second frequency threshold; and migrating the second memory data stored in the fourth memory space to the second memory space.
  • the target memory space can be divided into N intervals by target migration parameters, which are used to indicate the proportion of memory data allowed to be migrated in the target memory space.
  • the target migration parameters of the target memory space can be determined according to the proportion of the first memory space and the second memory space in the target memory space, for example: the proportion of the first memory space in the target memory space is A, the proportion of the second memory space in the target memory space is 100-A, and the target migration parameter is (A-10):20:(90-A).
  • the proportion of the first memory space in the target memory space is A, the proportion of the second memory space in the target memory space is 100-A, and the target migration parameter is A:100-A).
  • the memory usage parameters can be but are not limited to being determined based on the allocation parameters of the target memory space, and can be but are not limited to determining the proportion of the first memory space and the second memory space in the target memory space as the allocation parameters of the target memory space.
  • the above-mentioned target memory space allocation parameters can be but are not limited to being determined when the target memory space is allocated to the target application.
  • the target memory space allocated to the target application includes a first memory space with a capacity of A and a second memory space with a capacity of B
  • the allocation parameters of the target memory space include the proportion of the first memory space in the target memory space A/(A+B) and the proportion of the second memory space in the target memory space B/(A+B).
  • the second frequency threshold may be, but is not limited to, predetermined and used to select a memory space in the second memory space that is used more frequently.
  • a portion of the memory space in the first memory space that is used more frequently can be determined as the fourth memory space, but is not limited to it. Since the target application has a higher operating efficiency on the first memory space than on the second memory space, a portion of the memory space in the first memory space that is used less frequently is migrated to the second memory space to increase the operating efficiency of the target application.
  • the fourth memory space can be screened out from the first memory space according to the memory usage parameters and the target migration parameters in the following manner, but is not limited to: extracting memory cells that meet the third ratio from multiple memory cells in ascending order according to the usage parameters as a third memory cell set, wherein the multiple memory cells are memory cells that have stored data in the target memory space, the memory usage parameters include memory cells and usage parameters with corresponding relationships, the usage parameters are used to indicate the frequency with which the corresponding memory cells are used, and the target migration parameters include the third ratio; extracting memory cells belonging to the first memory space from the third memory cell set to obtain the fourth memory space.
  • the above-mentioned usage parameters may be but are not limited to the usage frequency of each memory unit in the target memory space of the target application.
  • the target memory space includes multiple memory units, and each time an operation (including but not limited to adding, deleting, searching and modifying, etc.) is performed on the memory unit in one or more time periods, the number of times the memory unit is used is increased by 1, and the total number of times the memory unit is used in one or more time periods is obtained, and then a weight is given to the sum of the number of times the operation is performed according to the distance between the time period in which the operation is performed and the current time, thereby obtaining the usage parameter of each memory unit.
  • the plurality of memory units may be arranged according to the usage parameter but is not limited thereto, for example, each memory unit is arranged from high to low according to the usage parameter. Alternatively, each memory unit is arranged from low to high according to the usage parameter.
  • the third proportion is used to indicate the memory units that meet a certain proportion in the target memory unit, and the third proportion may be determined, but not limited to, according to the target migration parameter.
  • the target memory space standard migration parameter is A: (100-A)
  • (100-A) may be determined as the third proportion, but not limited to.
  • the target memory space standard migration parameter is (A-10): 20: (90-A)
  • (90-A) may be determined as the third proportion, but not limited to.
  • the above-mentioned third memory cell set is the memory cells with the lowest usage frequency in the target memory cells and satisfying the third proportion.
  • the memory cells are arranged in order from the memory cells with the lowest usage frequency to the memory cells with the highest usage frequency according to the usage frequency of the memory cells, and the 20 memory cells with the highest usage frequency are selected from low to high as the third memory cell set.
  • the memory cells belonging to the first memory space among the most frequently used memory cells can be determined as the fourth memory space, but are not limited to it.
  • the third memory cell set including 100 memory cells 50 memory cells belonging to the first memory space are found from the third memory cell set, and the 50 memory cells in the third memory cell set belonging to the first memory space are determined as the fourth memory space.
  • the third ratio can be determined in the following manner but is not limited to: obtaining allocation parameters of the target memory space; determining the third ratio based on the allocation parameters.
  • the memory units may be arranged according to the usage parameters, 100% - C% - A% is the third proportion, and the lowest 100% - C% - A% is selected from high to low as the third memory unit set.
  • the allocation parameters of the target memory space can be obtained in the following manner, but not limited to: obtaining a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the second allocation ratio; the third ratio can be determined based on the allocation parameters in the following manner, but not limited to: determining the difference between the second allocation ratio and the second preset ratio as the third ratio.
  • the proportion of the second memory space in the target memory space may be determined as the second allocation proportion but is not limited to it.
  • the second preset proportion may be, but is not limited to, predetermined and used to determine the first proportion from the second allocated proportion.
  • the third proportion can be determined based on, but not limited to, the proportion of the second memory space in the target memory space and the second preset proportion. For example, taking the second preset proportion of 20% as an example, the proportion of the second memory space in the target memory space is A%, and the third proportion can be, but not limited to, A%-20%.
  • FIG6 is a schematic diagram of selecting a fourth memory space from a first memory space according to an embodiment of the present application. As shown in FIG6, taking the second preset proportion of 20% as an example, the fourth memory space can be determined by, but is not limited to, the following process:
  • the allocation parameter of the target memory space is 40%:60% for the first memory space and 40%:60% for the second memory space.
  • the target migration parameter is determined to be 40%:60%.
  • the target memory space is arranged from low to high.
  • the memory units belonging to the first memory space are obtained from the third memory unit set as the fourth memory space.
  • the fourth memory space can be screened out from the first memory space according to the memory usage parameters and the target migration parameters in the following manner, but is not limited to: dividing multiple memory units into a first part and a second part according to the usage parameters from high to low, wherein the first part and the second part are in a target ratio, and the multiple memory units are memory units that have stored data in the target memory space, and the memory usage parameters include memory units and usage parameters with a corresponding relationship, and the usage parameters are used to indicate the frequency with which the corresponding memory units are used; filtering out the memory units in the second part that meet the fourth ratio according to the usage parameters from high to low to obtain a fourth memory unit set, wherein the target migration parameters include the target ratio and the fourth ratio; extracting the memory units belonging to the first memory space from the fourth memory unit set to obtain the fourth memory space.
  • each memory unit in the target memory space may be divided according to the usage frequency of each memory unit in the target memory space, but is not limited to, to obtain a first part with a higher usage frequency and a second part with a lower usage frequency.
  • the target ratio of the first part to the second part can be but is not limited to being determined based on the proportion of the first memory space and the second memory space in the target memory space.
  • the target memory space can be but is not limited to being arranged from high to low according to the usage parameters, the first A% is divided into the first part, and the last B% is divided into the second part.
  • the second proportion may be, but is not limited to, a portion of the memory space based on the proportion of the first memory space and the second memory space in the target memory space.
  • the fourth memory unit set can be obtained by filtering out the memory units with the fourth proportion with the highest usage parameter from the second part but is not limited to it.
  • the target memory space is arranged from high to low according to the usage parameter, the first A% is divided into the first part, and the last B% is divided into the second part, and then 10% is filtered out from the second part from high to low to obtain the fourth memory unit set.
  • the memory units in the fourth memory unit set that belong to the first memory space may be determined as the fourth memory space, but is not limited to it.
  • the following method before dividing multiple memory units into the first part and the second part according to the usage parameters from high to low, the following method can be used but is not limited to: obtaining the allocation parameters of the target memory space; determining the target ratio and the fourth proportion according to the allocation parameters.
  • the fourth proportion can be obtained by, but not limited to, dividing the proportion of the first memory space and the second memory space in the target memory space.
  • the first memory space proportion in the target memory space is A%
  • it can be, but not limited to, dividing C% from the first memory space proportion A% as the fourth proportion.
  • it can be, but not limited to, dividing C% from the second memory space proportion (100% - A%) as the fourth proportion.
  • it can be, but not limited to, dividing C% from the first memory space proportion A% and then dividing D% from the second memory space proportion (100% - A%), and C% and D% are used as the fourth proportion.
  • the allocation parameters of the target memory space can be obtained in the following manner, but not limited to: obtaining a first allocation ratio of the first memory space in the target memory space, and a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the first allocation ratio and the second allocation ratio; the target ratio and the fourth ratio can be determined according to the allocation parameters in the following manner, but not limited to: determining the first allocation ratio and the second allocation ratio as the target ratio; obtaining a fourth preset ratio as the fourth ratio.
  • the proportion of the first memory space in the target memory space can be determined as the first allocation proportion
  • the proportion of the second memory space in the target memory space can be determined as the second allocation proportion.
  • the first allocation proportion and the second allocation proportion can be determined as allocation parameters, but are not limited to.
  • the fourth preset ratio may be, but is not limited to, predetermined, and the fourth preset ratio may be, but is not limited to, used to determine the fourth ratio.
  • the fourth proportion can be determined based on, but not limited to, the proportion of the first memory space in the target memory space, the proportion of the second memory space in the target memory space and the fourth preset proportion. For example, taking the fourth preset proportion of 20% as an example, the proportion of the first memory space in the target memory space is A%-10%, and the proportion of the second memory space in the target memory space is 90%-A%.
  • the fourth proportion can be determined to be 20%, but not limited to.
  • the proportion of the first memory space and the second memory space in the target memory space can be determined as the allocation parameters of the target memory space, but is not limited to.
  • the allocation parameters of the above target memory space can be determined, but are not limited to, when the target memory space is allocated to the target application.
  • the target memory space allocated to the target application includes a first memory space with a capacity of A and a second memory space with a capacity of B.
  • the allocation parameters of the target memory space include the proportion of the first memory space in the target memory space A/(A+B) and the proportion of the second memory space in the target memory space B/(A+B).
  • the target memory space can be divided into N intervals by target migration parameters, which are used to indicate the proportion of memory data allowed to be migrated in the target memory space.
  • the target migration parameters of the target memory space can be determined according to the proportion of the first memory space and the second memory space in the target memory space, for example: the proportion of the first memory space in the target memory space is A, the proportion of the second memory space in the target memory space is 100-A, and the target migration parameters are (A-10) ⁇ 20 ⁇ (90-A).
  • the proportion of the first memory space in the target memory space is A
  • the proportion of the second memory space in the target memory space is 100-A
  • the target migration parameters are A ⁇ (100-A).
  • the first frequency threshold may be, but is not limited to, predetermined, and the first frequency threshold may be, but is not limited to, used to select a memory space in the second memory space that is used more frequently.
  • a portion of the second memory space that is used more frequently can be determined as the third memory space, but is not limited to. Since the target application has a higher operating efficiency on the first memory space than on the second memory space, the portion of the second memory space that is used more frequently is migrated to the first memory space to increase the operating efficiency of the target application.
  • the third memory space can be screened out from the second memory space according to the memory usage parameters and the target migration parameters in the following manner, but is not limited to: extracting memory cells that meet the first ratio from multiple memory cells in descending order according to the usage parameters as a first memory cell set, wherein the multiple memory cells are memory cells that have stored data in the target memory space, the memory usage parameters include memory cells and usage parameters with corresponding relationships, the usage parameters are used to indicate the frequency with which the corresponding memory cells are used, and the target migration parameters include the first ratio; extracting memory cells belonging to the second memory space from the first memory cell set to obtain the third memory space.
  • the above-mentioned usage parameters may be but are not limited to the usage frequency of each memory unit in the target memory space of the target application.
  • the target memory space includes multiple memory units, and each time an operation (including but not limited to adding, deleting, searching and modifying, etc.) is performed on the memory unit in one or more time periods, the number of times the memory unit is used is increased by 1, and the total number of times the memory unit is used in one or more time periods is obtained, and then a weight is given to the sum of the number of times the operation is performed according to the distance between the time period in which the operation is performed and the current time, thereby obtaining the usage parameter of each memory unit.
  • the plurality of memory units may be arranged according to the usage parameter but is not limited thereto, for example, each memory unit is arranged from high to low according to the usage parameter. Alternatively, each memory unit is arranged from low to high according to the usage parameter.
  • the first proportion is used to indicate the memory units that meet a certain proportion in the target memory unit, and the first proportion may be determined, but not limited to, according to the target migration parameter.
  • the target memory space standard migration parameter is A: (100-A)
  • A may be determined, but not limited to, as the first proportion.
  • the target memory space standard migration parameter is (A-10): 20: (90-A)
  • (A-10) may be determined, but not limited to, as the first proportion.
  • the above-mentioned first memory unit set is the first percentage of memory units that are used most frequently in the target memory units. For example, taking the first percentage as 20%, and the target memory space including 100 memory units as an example, the memory units are arranged in order from the memory units with the highest frequency of use to the memory units with the lowest frequency of use according to the frequency of use of the memory units, and the 20 memory units with the highest frequency of use are selected from high to low as the first memory unit set.
  • the memory cells belonging to the second memory space among the most frequently used memory cells can be determined as the third memory space, but are not limited to it.
  • the first memory cell set including 100 memory cells 50 memory cells belonging to the second memory space are found from the first memory cell set, and the 50 memory cells belonging to the second memory space in the first memory cell set are determined as the third memory space.
  • the first ratio can be determined in the following manner but is not limited to: obtaining allocation parameters of the target memory space; determining the first ratio based on the allocation parameters.
  • the allocation parameters of the target memory space include A%-C%: 2C%: 100%-C%-A% as an example
  • the memory units may be arranged according to the usage parameters, A%-C% is the first proportion, and the first A%-C% is selected from high to low as the first memory unit set.
  • the allocation parameters of the target memory space can be obtained in the following manner, but not limited to: obtaining a first allocation ratio of the first memory space in the target memory space, wherein the allocation parameters include the first allocation ratio; the first ratio can be determined according to the allocation parameters in the following manner, but not limited to: determining the difference between the first allocation ratio and the first preset ratio as the first ratio.
  • the proportion of the first memory space in the target memory space may be determined as the first allocation proportion but is not limited to it.
  • the first preset proportion may be, but is not limited to, predetermined and used to determine the first proportion from the first allocated proportion.
  • the first proportion can be determined based on, but not limited to, the proportion of the first memory space in the target memory space and the first preset proportion. For example, taking the first preset proportion of 20% as an example, the proportion of the first memory space in the target memory space is A%, and the first proportion is A%-20%.
  • FIG. 7 is a schematic diagram of selecting a third memory space from a second memory space according to an embodiment of the present application. As shown in FIG. 7 , taking the first preset proportion of 20% as an example, the third memory space can be determined by, but is not limited to, the following process:
  • the allocation parameter of the target memory space is 60%:40% for the first memory space and 60%:40% for the second memory space.
  • the target migration parameter is determined to be 60%:40%.
  • the target memory spaces are arranged from high to low.
  • the memory units belonging to the second memory space are obtained from the first memory unit set as the third memory space.
  • the third memory space can be screened out from the second memory space according to the memory usage parameters and the target migration parameters in the following manner, but is not limited to: dividing the multiple memory units into a first part and a second part according to the usage parameters from high to low, wherein the first part and the second part are in a target ratio, and the multiple memory units are memory units that have stored data in the target memory space, and the memory usage parameters include memory units and usage parameters with a corresponding relationship, and the usage parameters are used to indicate the frequency with which the corresponding memory units are used; filtering out the memory units that meet the second ratio in the first part according to the usage parameters from low to high to obtain a second memory unit set, wherein the target migration parameters include the target ratio and the second ratio; extracting the memory units belonging to the second memory space from the second memory unit set to obtain the third memory space.
  • each memory unit in the target memory space may be divided according to the usage frequency of each memory unit in the target memory space, but is not limited to, to obtain a first part with a higher usage frequency and a second part with a lower usage frequency.
  • the target ratio of the first part to the second part can be but is not limited to being determined based on the proportion of the first memory space and the second memory space in the target memory space.
  • the target memory space can be but is not limited to being arranged from high to low according to the usage parameters, the first A% is divided into the first part, and the last B% is divided into the second part.
  • the second proportion may be, but is not limited to, a portion of the memory space based on the proportion of the first memory space and the second memory space in the target memory space.
  • the second memory unit set can be obtained by filtering out the second percentage memory units with the lowest usage parameter from the first part but is not limited to it.
  • the target memory space is arranged from high to low according to the usage parameter, the first A% is divided into the first part, and the last B% is divided into the second part, and then 10% is filtered out from the first part from low to high to obtain the second memory unit set.
  • the memory units in the second memory unit set that belong to the second memory space may be determined as the third memory space, but is not limited to it.
  • the target ratio and the second proportion can be determined in the following manner but is not limited to: obtaining allocation parameters of the target memory space; determining the target ratio and the second proportion according to the allocation parameters.
  • the second proportion can be obtained by, but not limited to, dividing the proportion of the first memory space and the second memory space in the target memory space.
  • the first memory space proportion in the target memory space is A%
  • it can be, but not limited to, dividing C% from the first memory space proportion A% as the second proportion.
  • it can be, but not limited to, dividing C% from the second memory space proportion (100% - A%) as the second proportion.
  • it can be, but not limited to, dividing C% from the first memory space proportion A% and then dividing D% from the second memory space proportion (100% - A%), and C% and D% are used as the second proportion.
  • the allocation parameters of the target memory space can be obtained in the following manner, but not limited to: obtaining a first allocation ratio of the first memory space in the target memory space, and a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the first allocation ratio and the second allocation ratio; the target ratio and the second ratio can be determined according to the allocation parameters in the following manner, but not limited to: determining the first allocation ratio and the second allocation ratio as the target ratio; obtaining a second preset ratio as the second ratio.
  • the proportion of the first memory space in the target memory space can be determined as the first allocation proportion
  • the proportion of the second memory space in the target memory space can be determined as the second allocation proportion.
  • the first allocation proportion and the second allocation proportion can be determined as allocation parameters, but are not limited to.
  • the second proportion can be determined based on, but not limited to, the proportion of the first memory space in the target memory space, the proportion of the second memory space in the target memory space and the second preset proportion. For example, taking the second preset proportion of 20% as an example, the proportion of the first memory space in the target memory space is A%-10%, and the proportion of the second memory space in the target memory space is 90%-A%. The second proportion can be determined to be 20%, but not limited to.
  • the first memory data stored in the third memory space may be copied to the first memory space but is not limited to, so as to migrate the first memory data stored in the third memory space to the first memory space.
  • the first memory data stored in the third memory space can be migrated to the first memory space in the following manner but is not limited to: detecting a reference remaining capacity of the first memory space; when the amount of data of the first memory data is greater than the reference remaining capacity, screening target memory data whose data amount meets the reference remaining capacity from all used memory cells in the first memory data; and migrating the target memory data to the first memory space.
  • the reference remaining capacity may be, but is not limited to, unused memory space of the first memory space, or the amount of data that can be accommodated in the first memory space without affecting the use of the first memory space.
  • the first 90% of the first memory data may be determined as the target memory data, but is not limited to, or the memory data with the highest memory usage parameter in the first memory data may be determined as the target memory data, but is not limited to.
  • FIG8 is a schematic diagram of performing memory data migration in a first memory space and a second memory space according to an embodiment of the present application.
  • the present application sets warm memory and keeps the warm memory from being migrated, thereby reducing the number of memory migrations of memory pages in the middle position that are used frequently during the use of the Linux system.
  • the Linux system includes SDK (Software Development Kit) and HW (Hardware). Local memory and CXL memory are deployed in HW.
  • the SDK includes Linux kernel module, memory pooling module, memory tiering module, memory migration module, memory identification module and API module.
  • the Linux kernel module can be, but is not limited to, based on Linux 6.2 development drivers and CXL memory related interfaces. Including a memory identification module, which can be, but is not limited to, identifying the CXL memory capacity and address through the Linux kernel module, for example: the Linux kernel module identifies the CXL memory as RAM during the startup of the operating system, identifies and modifies the CXL memory attribute to soft reserved, maps the local memory and CXL memory into different groups (Normal, CxlMemory) in buddyinfo, reads the CXL memory distance, records the number of slots used by the CXL memory and calculates the bandwidth that the CXL memory can provide based on this, and provides an API for modifying the NUMA node.
  • a memory identification module which can be, but is not limited to, identifying the CXL memory capacity and address through the Linux kernel module, for example: the Linux kernel module identifies the CXL memory as RAM during the startup of the operating system, identifies and modifies the CXL memory
  • the memory pooling module is configured to allocate memory to each application when the system starts.
  • the memory pooling module may, but is not limited to, use a memory allocation mode that uses both tcmalloc allocator and jemalloc allocator, for example, actively configuring to allocate 1GB of local memory and 1GB of CXL memory to the application.
  • the memory pooling module includes an application table configured to store the maximum usage of local memory and CXL memory for each application, and may, but is not limited to, allocate space to the application based on the maximum usage stored in the application table.
  • the memory tiering module may be, but is not limited to, configured to record memory usage parameters of the target memory space, and filter out the memory space to be migrated according to the memory usage parameters and the target migration parameters.
  • the memory migration module is configured to migrate the memory space that needs to be migrated and is screened out by the memory layering module.
  • the memory data migration can be performed in the first memory space and the second memory space in the following manners, but not limited to:
  • the remaining capacity a of the local memory in the memory pool (target memory space) is calculated, and the portion of the hot memory data (third memory space) of the CXL memory that does not exceed 90% of a is packaged into data blocks, which are uniformly copied into the local memory.
  • the hierarchical table used to record the memory allocation and the BitMap used to manage the memory are converted and integrated accordingly, and the corresponding position of the hot memory in the original CXL memory that has been copied is cleared to 0; when the local memory usage reaches 90%, the storage is stopped, and the cold memory data (fourth memory space) of the local memory is copied and stored in the CXL memory instead;
  • the remaining capacity b of the CXL memory in the calculation pool (target memory space) is calculated, and the cold memory data of the local memory that does not exceed 90% of b is packaged into data blocks, and uniformly copied into the CXL memory. Then, the hierarchical table used to record the memory allocation and the BitMap used to manage the memory are converted and integrated accordingly, and the corresponding position of the cold memory of the original local memory that has been copied is cleared to 0;
  • the storage is suspended and the hot memory data of the CXL memory is stored in the local memory instead; until all the hot memory data of the CXL memory is stored in the local memory and all the cold data of the local memory is stored in the CXL memory.
  • the migration granularity of the memory page 1 is first set to 256Byte (bytes), and then the used memory block is found according to the Offset (offset) allocated in the offset migration page 1, and it is migrated to the local memory, and then the migration granularity is updated in sequence, and then other used memory blocks are found according to the Offset allocated in page 1, and they are migrated to the local memory, until the migration of the current memory page 1 is completed.
  • Offset Offset allocated in the offset migration page 1
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.
  • the technical solution of the present application, or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a non-volatile readable storage medium (such as ROM/RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.
  • a non-volatile readable storage medium such as ROM/RAM, magnetic disk, optical disk
  • a terminal device which can be a mobile phone, computer, server, or network device, etc.
  • a migration device for memory data is also provided, which is configured to implement the above-mentioned embodiments and optional implementation modes, and the descriptions that have been made will not be repeated.
  • the term "module” can be a combination of software and/or hardware that implements a predetermined function.
  • the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
  • FIG. 9 is a structural block diagram of a memory data migration device according to an embodiment of the present application. As shown in FIG. 9 , the device includes:
  • a first detection module 92 is configured to detect a memory usage parameter of a target memory space, wherein the target memory space is a memory space allocated for a target application of memory data to be migrated, the target memory space includes a first memory space of a first type and a second memory space of a second type, the memory usage parameter is used to indicate a frequency distribution of use of the target memory space, and an operation efficiency of the target application on the first memory space is higher than an operation efficiency of the target application on the second memory space;
  • a first screening module 94 is configured to screen out a third memory space from the second memory space according to a memory usage parameter and a target migration parameter, wherein the target migration parameter is determined according to an allocation parameter of the target memory space, the allocation parameter is used to indicate a proportion of the first memory space and the second memory space in the target memory space, the target migration parameter is used to indicate a proportion of memory space in the target memory space that is allowed to migrate memory data, and the third memory space is a memory space in the second memory space whose frequency of use is higher than a first frequency threshold;
  • the first migration module 96 is configured to migrate the first memory data stored in the third memory space to the first memory space.
  • the target memory space used by the target application includes the first memory space and the second memory space. According to the frequency distribution of the target memory space used and the allocation parameters of the target memory space, the memory space with high frequency of use in the second memory space is screened out from the target memory space, that is, the third memory space that needs to be migrated. Since the target application's operating efficiency on the first memory space is higher than the target application's operating efficiency on the second memory space, by migrating the first memory data stored in the part of the memory space with high use efficiency in the second memory space to the first memory space, the target application's operating efficiency on the memory data with high use efficiency is improved, thereby improving the operating efficiency of the target application. Therefore, the problem of low memory use efficiency can be solved, and the effect of improving memory use efficiency can be achieved.
  • the first screening module includes:
  • a first extraction unit is configured to extract memory cells that meet a first proportion from a plurality of memory cells according to a usage parameter from high to low as a first memory cell set, wherein the plurality of memory cells are memory cells that have stored data in a target memory space, the memory usage parameter includes memory cells and usage parameters that have a corresponding relationship, the usage parameter is used to indicate a frequency at which a corresponding memory cell is used, and the target migration parameter includes the first proportion;
  • the second extraction unit is configured to extract memory units belonging to the second memory space from the first memory unit set to obtain a third memory space.
  • the apparatus further comprises:
  • a first acquisition module is configured to acquire allocation parameters of a target memory space
  • the first determining module is configured to determine a first proportion according to the allocation parameter.
  • the first acquisition module includes:
  • a first acquisition unit is configured to acquire a first allocation ratio of the first memory space in the target memory space, wherein the allocation parameters include the first allocation ratio;
  • the first determining module includes:
  • the first determining unit is configured to determine a difference between the first allocated proportion and the first preset proportion as the first proportion.
  • the first screening module includes:
  • a first division unit is configured to divide the plurality of memory cells into a first part and a second part according to the usage parameter from high to low, wherein the first part and the second part are in a target ratio, the plurality of memory cells are memory cells storing data in the target memory space, the memory usage parameter includes memory cells and usage parameters having a corresponding relationship, and the usage parameter is used to indicate a frequency at which the corresponding memory cell is used;
  • a first screening unit is configured to screen out memory cells that meet a second proportion in the first part from low to high according to the usage parameter to obtain a second memory cell set, wherein the target migration parameter includes a target proportion and a second proportion;
  • the third extraction unit is configured to extract memory units belonging to the second memory space from the second memory unit set to obtain a third memory space.
  • the apparatus further comprises:
  • a second acquisition module is configured to acquire allocation parameters of a target memory space
  • the second determination module is configured to determine the target ratio and the second ratio according to the allocation parameters.
  • the second acquisition module includes:
  • a second acquisition unit is configured to acquire a first allocation ratio of the first memory space in the target memory space and a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the first allocation ratio and the second allocation ratio;
  • the second determining module includes:
  • the second determining unit is configured to determine the first allocation ratio and the second allocation ratio as a target ratio; and obtain a second preset ratio as a second ratio.
  • the apparatus further comprises:
  • a second screening module is configured to screen out a fourth memory space from the first memory space according to the memory usage parameter and the target migration parameter, wherein the fourth memory space is a memory space in the first memory space whose usage frequency is lower than the second frequency threshold;
  • the second migration module is configured to migrate the second memory data stored in the fourth memory space to the second memory space.
  • the second screening module includes:
  • a fourth extraction unit configured to extract memory units that meet a third proportion from a plurality of memory units according to a usage parameter from low to high as a third memory unit set, wherein the plurality of memory units are memory units that have stored data in a target memory space, the memory usage parameter includes memory units and usage parameters that have a corresponding relationship, the usage parameter is used to indicate a frequency at which a corresponding memory unit is used, and the target migration parameter includes the third proportion;
  • the fifth extraction unit is configured to extract memory units belonging to the first memory space from the third memory unit set to obtain a fourth memory space.
  • the apparatus further comprises:
  • a third acquisition module is configured to acquire allocation parameters of a target memory space
  • the third determination module is configured to determine a third proportion according to the allocation parameter.
  • the third acquisition module includes:
  • a third acquisition unit is configured to acquire a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameter includes the second allocation ratio;
  • the third determining module includes:
  • the third determining unit is configured to determine a difference between the second allocated proportion and the second preset proportion as a third proportion.
  • the second screening module includes:
  • a second division unit is configured to divide the plurality of memory cells into a first part and a second part according to the usage parameter from high to low, wherein the first part and the second part are in a target ratio, the plurality of memory cells are memory cells in which data has been stored in the target memory space, the memory usage parameter includes memory cells and usage parameters having a corresponding relationship, and the usage parameter is used to indicate a frequency at which the corresponding memory cell is used;
  • a second screening unit is configured to screen out memory units that meet a fourth ratio in the second part from high to low according to the usage parameter to obtain a fourth memory unit set, wherein the target migration parameter includes a target ratio and a fourth ratio;
  • the sixth extraction unit is configured to extract memory units belonging to the first memory space from the fourth memory unit set to obtain a fourth memory space.
  • the apparatus further comprises:
  • a fourth acquisition module is configured to acquire allocation parameters of a target memory space
  • the fourth determination module is configured to determine the target ratio and the fourth proportion according to the allocation parameters.
  • the fourth acquisition module includes:
  • a fourth acquisition unit is configured to acquire a first allocation ratio of the first memory space in the target memory space and a second allocation ratio of the second memory space in the target memory space, wherein the allocation parameters include the first allocation ratio and the second allocation ratio;
  • the fourth determining module includes:
  • the fourth determining unit is configured to determine the first allocation ratio and the second allocation ratio as a target ratio; and obtain a fourth preset ratio as a fourth ratio.
  • the first migration module includes:
  • a first detection unit is configured to detect a reference remaining capacity of the first memory space
  • a screening unit is configured to screen target memory data whose data amount meets the reference remaining capacity from all used memory cells in the first memory data when the data amount of the first memory data is greater than the reference remaining capacity;
  • the migration unit is configured to migrate the target memory data to the first memory space.
  • the first detection module includes:
  • a collection unit is configured to collect the usage frequency of a target memory unit in each of one or more time periods to obtain one or more usage frequencies, wherein the target memory unit is each memory unit in a plurality of memory units storing data in a target memory space;
  • the calculation unit is configured to calculate the usage parameter corresponding to the target memory unit according to one or more usage frequencies, and obtain the memory unit and the usage parameter having a corresponding relationship as the memory usage parameter.
  • the computing unit is further configured to: assign a weight to each time period based on the distance between each time period and the current time, wherein the smaller the distance between each time period and the current time, the greater the weight corresponding to each time period; and calculate the weighted sum of one or more usage frequencies as the usage parameter corresponding to the target memory unit.
  • the apparatus further comprises:
  • a second detection module is configured to detect a usage rate of an initial memory space allocated for a target application
  • the adjustment module is configured to adjust the initial memory space according to the relationship between the usage rate and the usage rate threshold to obtain the target memory space.
  • the adjustment module includes:
  • an allocation unit configured to allocate an additional memory space of a first capacity to a target application when a usage rate is greater than or equal to a first usage rate threshold, to obtain a target memory space, wherein the target memory space includes an initial memory space and an additional memory space;
  • the releasing unit is configured to release the memory space of the second capacity from the target memory space to obtain the target memory space when the usage rate is less than or equal to the second usage rate threshold.
  • the adjustment module includes:
  • a second detection unit configured to detect a target number of processor cores used by a target application
  • a first adjustment unit is configured to adjust the initial memory space using a first memory tool when the target quantity is greater than or equal to a quantity threshold;
  • the second adjustment unit is configured to use a second memory tool to adjust the initial memory space when the target quantity is less than a quantity threshold, wherein the adjustment efficiency of the second memory tool for the memory space is lower than the adjustment efficiency of the first memory tool for the memory space, but the operating load of the second memory tool is less than the operating load of the first memory tool.
  • the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
  • An embodiment of the present application further provides a computer non-volatile readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
  • the above-mentioned computer non-volatile readable storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.
  • FIG. 10 is a schematic diagram of an electronic device according to an embodiment of the present application. As shown in Figure 10, it includes a memory and a processor. A computer program is stored in the memory. The processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
  • the electronic device may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
  • modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation.
  • the present application is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Memory System Of A Hierarchy Structure (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)

Abstract

本申请实施例提供了一种内存数据的迁移方法、装置、非易失性可读存储介质及电子装置,其中,该方法包括:检测目标内存空间的内存使用参数,其中,目标内存空间是为待迁移内存数据的目标应用所分配的内存空间,目标内存空间中包括属于第一类型的第一内存空间和属于第二类型的第二内存空间,内存使用参数用于指示目标内存空间被使用的频率分布,目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率;根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间;将第三内存空间中所存储的第一内存数据迁移至第一内存空间。通过本申请,解决了内存使用的效率较低的问题,进而达到了提高内存使用的效率的效果。

Description

内存数据的迁移方法、装置、非易失性可读存储介质及电子装置
相关申请的交叉引用
本申请要求于2023年06月21日提交中国专利局,申请号为202310743575.1,申请名称为“内存数据的迁移方法、装置、存储介质及电子装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及计算机领域,特别的,涉及一种内存数据的迁移方法、装置、非易失性可读存储介质及电子装置。
背景技术
随着超大型数据中心出现破记录的增长趋势,内存需求问题应运而生。在这种情况下,为了满足应用在实现更丰富的功能时,能够提供足够的内存空间,CXL(Compute Express Link,计算快速连接)技术应运而生,CXL是一种开放的行业标准互连,在主机处理器和加速器、内存缓冲区和智能I/O设备(Input/Output,输入/输出)等等设备之间提供高带宽、低延迟的连接。目前,CXL技术最流行的应用领域便是拓展内存容量,但在完成了对内存容量的扩展后,对设备的本地内存与扩展的内存之间仍然缺少针对应用使用的过程中进行动态可伸缩式内存的管理和分层的方法。
针对相关技术中,内存使用的效率较低等问题,尚未提出有效的解决方案。
发明内容
本申请实施例提供了一种内存数据的迁移方法、装置、非易失性可读存储介质及电子装置,以至少解决相关技术中内存使用的效率较低的问题。
根据第一方面,提供了一种内存数据的迁移方法,包括:
检测目标内存空间的内存使用参数,其中,目标内存空间是为待迁移内存数据的目标应用所分配的内存空间,目标内存空间中包括属于第一类型的第一内存空间和属于第二类型的第二内存空间,内存使用参数用于指示目标内存空间被使用的频率分布,目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率;
根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间,其中,目标迁移参数是根据目标内存空间的分配参数确定的,分配参数用于指示第一内存空间和第二内存空间在目标内存空间中的占比,目标迁移参数用于指示目标内存空间中允许迁移内存数据的内存空间的占比,第三内存空间是第二内存空间中被使用的频率高于第一频率阈值的内存空间;
将第三内存空间中所存储的第一内存数据迁移至第一内存空间。
在一个示例性实施例中,根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间,包括:
按照使用参数由高至低提取多个内存单元中满足第一占比的内存单元作为第一内存单元集合,其中,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率,目标迁移参数包括第一占比;
从第一内存单元集合中提取属于第二内存空间的内存单元,得到第三内存空间。
在一个示例性实施例中,在按照使用参数由高至低提取多个内存单元中满足第一占比的内存单元作为第一内存单元集合之前,方法还包括:
获取目标内存空间的分配参数;
根据分配参数确定第一占比。
在一个示例性实施例中,获取目标内存空间的分配参数,包括:获取第一内存空间在目标内存空间中的第一分配占比,其中,分配参数包括第一分配占比;
根据分配参数确定第一占比,包括:将第一分配占比与第一预设占比的差值确定为第一占比。
在一个示例性实施例中,根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间,包括:
按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,第一部分与第二部分呈目标比例,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率;
按照使用参数由低至高筛除第一部分中满足第二占比的内存单元得到第二内存单元集合,其中,目标迁移参数包括目标比例和第二占比;
从第二内存单元集合中提取属于第二内存空间的内存单元,得到第三内存空间。
在一个示例性实施例中,在按照使用参数由高至低将多个内存单元划分为第一部分和第二部分之前,方法还包括:
获取目标内存空间的分配参数;
根据分配参数确定目标比例和第二占比。
在一个示例性实施例中,获取目标内存空间的分配参数,包括:获取第一内存空间在目标内存空间中的第一分配占比,和第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第一分配占比和第二分配占比;
根据分配参数确定目标比例和第二占比,包括:将第一分配占比与第二分配占比确定为目标比例;获取第二预设占比作为第二占比。
在一个示例性实施例中,在检测目标内存空间的内存使用参数之后,方法还包括:
根据内存使用参数和目标迁移参数从第一内存空间中筛选出第四内存空间,其中,第四内存空间是第一内存空间中被使用的频率低于第二频率阈值的内存空间;
将第四内存空间中所存储的第二内存数据迁移至第二内存空间。
在一个示例性实施例中,根据内存使用参数和目标迁移参数从第一内存空间中筛选出第四内存空间,包括:
按照使用参数由低至高提取多个内存单元中满足第三占比的内存单元作为第三内存单元集合,其中,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率,目标迁移参数包括第三占比;
从第三内存单元集合中提取属于第一内存空间的内存单元得到第四内存空间。
在一个示例性实施例中,在按照使用参数由低至高提取多个内存单元中满足第三占比的内存单元作为第三内存单元集合之前,方法还包括:
获取目标内存空间的分配参数;
根据分配参数确定第三占比。
在一个示例性实施例中,获取目标内存空间的分配参数,包括:获取第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第二分配占比;
根据分配参数确定第三占比,包括:将第二分配占比与第二预设占比的差值确定为第三占比。
在一个示例性实施例中,根据内存使用参数和目标迁移参数从第一内存空间中筛选出第四内存空间,包括:
按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,第一部分与第二部分呈目标比例,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率;
按照使用参数由高至低筛除第二部分中满足第四占比的内存单元得到第四内存单元集合,其中,目标迁移参数包括目标比例和第四占比;
从第四内存单元集合中提取属于第一内存空间的内存单元得到第四内存空间。
在一个示例性实施例中,在按照使用参数由高至低将多个内存单元划分为第一部分和第二部分之前,方法还包括:
获取目标内存空间的分配参数;
根据分配参数确定目标比例和第四占比。
在一个示例性实施例中,获取目标内存空间的分配参数,包括:获取第一内存空间在目标内存空间中的第一分配占比,和第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第一分配占比和第二分配占比;
根据分配参数确定目标比例和第四占比,包括:将第一分配占比与第二分配占比确定为目标比例;获取第四预设占比作为第四占比。
在一个示例性实施例中,将第三内存空间中所存储的第一内存数据迁移至第一内存空间,包括:
检测第一内存空间的参考剩余容量;
在第一内存数据的数据量大于参考剩余容量的情况下,从第一内存数据中全部被使用的内存单元中筛选数据量满足参考剩余容量的目标内存数据;
将目标内存数据迁移至第一内存空间。
在一个示例性实施例中,检测目标内存空间的内存使用参数,包括:
采集目标内存单元在一个或者多个时间段中每个时间段内的使用频次,得到一个或者多个使用频次,其中,目标内存单元是目标内存空间中已存储数据的多个内存单元中的每个内存单元;
根据一个或者多个使用频次计算目标内存单元所对应的使用参数,得到具有对应关系的内存单元和使用参数作为内存使用参数。
在一个示例性实施例中,根据一个或者多个使用频次计算目标内存单元所对应的使用参数,包括:
根据每个时间段与当前时间的距离为每个时间段分配权重,其中,每个时间段与当前时间的距离越小,每个时间段所对应的权重越大;
计算一个或者多个使用频次的加权和作为目标内存单元所对应的使用参数。
在一个示例性实施例中,在检测目标内存空间的内存使用参数之前,方法还包括:
检测为目标应用所分配的初始内存空间的使用率;
根据使用率与使用率阈值之间的关系,调整初始内存空间,得到目标内存空间。
在一个示例性实施例中,根据使用率与使用率阈值之间的关系,调整初始内存空间,得到目标内存空间,包括:
在使用率大于或者等于第一使用率阈值的情况下,为目标应用分配第一容量的附加内存空间,得到目标内存空间,其中,目标内存空间包括初始内存空间和附加内存空间;
在使用率小于或者等于第二使用率阈值的情况下,从目标内存空间中释放第二容量的内存空间,得到目标内存空间。
在一个示例性实施例中,调整初始内存空间,包括:
检测目标应用所使用的处理器核心的目标数量;
在目标数量大于或者等于数量阈值的情况下,使用第一内存工具调整初始内存空间;
在目标数量小于数量阈值的情况下,使用第二内存工具调整初始内存空间,其中,第二内存工具对内存空间的调整效率低于第一内存工具对内存空间的调整效率,但第二内存工具的运行负载小于第一内存工具的运行负载。
根据第二方面,提供了一种内存数据的迁移装置,包括:
第一检测模块,被设置为检测目标内存空间的内存使用参数,其中,目标内存空间是为待迁移内存数据的目标应用所分配的内存空间,目标内存空间中包括属于第一类型的第一内存空间和属于第二类型的第二内存空间,内存使用参数用于指示目标内存空间被使用的频率分布,目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率;
第一筛选模块,被设置为根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间,其中,目标迁移参数是根据目标内存空间的分配参数确定的,分配参数用于指示第一内存空间和第二内存空间在目标内存空间中的占比,目标迁移参数用于指示目标内存空间中允许迁移内存数据的内存空间的占比,第三内存空间是第二内存空间中被使用的频率高于第一频率阈值的内存空间;
第一迁移模块,被设置为将第三内存空间中所存储的第一内存数据迁移至第一内存空间。
根据第三方面,还提供了一种计算机非易失性可读存储介质,计算机非易失性可读存储介质中存储有计算机程序,其中,计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
根据第四方面,还提供了一种电子设备,包括存储器和处理器,存储器中存储有计算机程序,处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
通过本申请,目标应用使用的目标内存空间包括第一内存空间和第二内存空间,根据目标内存空间中被使用的频率分布,以及目标内存空间的分配参数从目标内存空间中筛选出第二内存空间中被使用的频率高的内存空间即需要迁移的第三内存空间,由于目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率,通过将第二内存空间中使用效率高的部分内存空间中存储的第一内存数据迁移至第一内存空间,提高了目标应用对使用效率高的内存数据的操作效率,从而提高了目标应用的操作效率。因此,可以解决内存使用的效率较低问题,达到提高内存使用的效率的效果。
附图说明
图1是根据本申请实施例的一种内存数据的迁移方法的移动终端的硬件结构框图;
图2是根据本申请实施例的一种内存数据的迁移方法的流程图;
图3是根据本申请实施例的一种内存使用参数确定过程的示意图;
图4是根据本申请实施例的为目标应用分配初始内存空间的流程图;
图5是根据本申请实施例的对内存空间进行动态分配的流程图;
图6是根据本申请实施例的从第一内存空间中筛选第四内存空间的示意图;
图7是根据本申请实施例的从第二内存空间中筛选第三内存空间的示意图;
图8是根据本申请实施例的在第一内存空间和第二内存空间中执行内存数据迁移的示意图;
图9是根据本申请实施例的一种内存数据的迁移装置的结构框图;
图10是根据本申请实施例的电子设备的示意图。
具体实施方式
下文中将参考附图并结合实施例来详细说明本申请的实施例。
需要说明的是,本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。
本申请实施例中所提供的方法实施例可以在移动终端、计算机终端或者类似的运算装置中执行。以运行在移动终端上为例,图1是根据本申请实施例的一种内存数据的迁移方法的移动终端的硬件结构框图。如图1所示,移动终端可以包括一个或多个(图1中仅示出一个)处理器102(处理器102可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)和被设置为存储数据的存储器104,其中,上述移动终端还可以包括被设置为通信功能的传输设备106以及输入输出设备108。本领域普通技术人员可以理解,图1所示的结构仅为示意,其并不对上述移动终端的结构造成限定。例如,移动终端还可包括比图1中所示更多或者更少的组件,或者具有与图1所示不同的配置。
存储器104可被设置为存储计算机程序,例如,应用软件的软件程序以及模块,如本申请实施例中的内存数据的迁移方法对应的计算机程序,处理器102通过运行存储在存储器104内的计算机程序,从而执行各种功能应用以及数据处理,即实现上述的方法。存储器104可包括高速随机存储器,还可包括非易失性存储器,如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中,存储器104可包括相对于处理器102远程设置的存储器,这些远程存储器可以通过网络连接至移动终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输设备106被设置为经由一个网络接收或者发送数据。上述的网络实例可包括移动终端的通信供应商提供的无线网络。在一个实例中,传输设备106包括一个网络适配器(Network Interface Controller,简称为NIC),其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中,传输设备106可以为射频(Radio Frequency,简称为RF)模块,其被设置为通过无线方式与互联网进行通讯。
在本实施例中提供了一种运行于上述移动终端的内存数据的迁移方法,图2是根据本申请实施例的一种内存数据的迁移方法的流程图,如图2所示,该流程包括如下步骤:
步骤S202,检测目标内存空间的内存使用参数,其中,目标内存空间是为待迁移内存数据的目标应用所分配的内存空间,目标内存空间中包括属于第一类型的第一内存空间和属于第二类型的第二内存空间,内存使用参数用于指示目标内存空间被使用的频率分布,目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率;
步骤S204,根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间,其中,目标迁移参数是根据目标内存空间的分配参数确定的,分配参数用于指示第一内存空间和第二内存空间在目标内存空间中的占比,目标迁移参数用于指示目标内存空间中允许迁移内存数据的内存空间的占比,第三内存空间是第二内存空间中被使用的频率高于第一频率阈值的内存空间;
步骤S206,将第三内存空间中所存储的第一内存数据迁移至第一内存空间。
通过上述步骤,目标应用使用的目标内存空间包括第一内存空间和第二内存空间,根据目标内存空间中被使用的频率分布,以及目标内存空间的分配参数从目标内存空间中筛选出第二内存空间中被使用的频率高的内存空间即需要迁移的第三内存空间,由于目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率,通过将第二内存空间中使用效率高的部分内存空间中存储的第一内存数据迁移至第一内存空间,提高了目标应用对使用效率高的内存数据的操作效率,从而提高了目标应用的操作效率。因此,可以解决内存使用的效率较低问题,达到提高内存使用的效率的效果。
在上述步骤S202提供的技术方案中,上述内存数据的迁移方法可以但不限于是通过CXL协议实现对服务器硬件系统的内存扩展,比如:服务器硬件系统通过PCIe 5协议(Peripheral Component Interconnect Express 5,高速串行计算机扩展总线标准(第五代))实现远程共享非本地的DRAM(Dynamic Random Access Memory,动态随机存取存储器)以及共享IO设备,扩展计算机中处理器与DDR DRAM(Double Data Rate Dynamic Random Access Memory,双倍速率动态随机存取存储器)之间传输数据的方式,实现了远程共享内存为系统提供更高的内存容量和性能。
可选地,在本实施例中,上述内存数据的迁移方法可以但不限于应用于支持CXL协议的服务器硬件系统中,上述服务器硬件系统可以但不限于包括支持CXL协议的CPU(Central Processing Unit,中央处理器),支持CXL协议的芯片组,支持CXL协议的硬件设备和支持CXL协议的操作系统等等。上述CXL协议可以但不限于包括:CXL1.1协议,CXL1.0协议和CXL2.0协议等等。
可选地,在本实施例中,上述内存数据的迁移方法可以但不限于应用于支持CXL协议的操作系统,比如:Linux系统(GNU/Linux),Windows操作系统(Microsoft Windows,美国微软公司以图形用户界面为基础研发的操作系统),VMware ESXi系统(一款虚拟化支持windows平台运行的软件)等等。以Linux系统支持CXL协议且Linux系统包括本地内存和扩展的CXL内存为例,Linux系统可以但不限于通过以下方式管理扩展的CXL内存:
基于Linux6.2开发相应的驱动和与CXL内存相关的接口,根据SRAT(Static Resource Affinity Table,静态资源关联表)和e820(一种用于描述系统内存布局的数据结构)识别CXL内存容量和地址;在启动Linux操作系统时,将外接的CXL内存自动识别为RAM(Random Access Memory,随机存取存储器),自动识别并修改CXL内存属性为soft reserved(一种内存属性,表示该内存区域已被分配但尚未使用);使用buddyinfo(一个Linux命令,用于查看系统中的Buddy Allocator信息)将本地内存与CXL内存进行区分,映射为不同的组别(NormalMemory,CxlMemory)即(本地内存,CXL内存);使用SLIT(Serial Linked Interface Technology,一种串行连接接口技术)读取CXL内存与主机处理器之间的距离,记录CXL内存使用的插槽数,根据CXL内存使用的插槽数计算CXL内存可以提供的带宽量;通过修改NUMA(Non-Uniform Memory Access,非均匀内存访问)节点API(Application Programming Interface,应用程序接口),使得用户能够在终端或程序中使用此API修改CXL内存的NUMA节点。
可选地,在本实施例中,上述目标应用可以但不限于是运行在支持CXL协议的服务器硬件系统中,上述支持CXL协议的服务器硬件系统能够运行一个或多个目标应用,可以但不限于将支持CXL协议的服务器硬件系统中运行的一个或者多个应用确定为目标应用。
可选地,在本实施例中,可以但不限于在系统启动的阶段为目标应用分配两种不同来源的内存空间得到目标内存空间。或者,可以但不限于在目标应用运行的过程中,根据目标应用对内存空间的需求,对目标应用的内存空间进行实时的调整,从而得到新的目标内存空间。
可选地,在本实施例中,属于第一类型的第一内存空间和属于第二类型的第二内存空间可以但不限于通过两种不同的方式接入目标应用所在的系统中的,上述第一类型和第二类型用于指示与第一内存空间和第二内存空间接入上述目标应用所在的系统中的方式不同。
可选地,在本实施例中,上述属于第一类型的第一内存空间可以但不限于是目标应用所在的系统的本地内存。上述属于第二类型的第二内存空间可以但不限于是目标应用所在的系统的外接内存,比如:CXL内存,可以但不限于通过CXL技术将远程内存视为本地内存使用。以外接内存为CXL内存为例,由于距离较近的CXL内存即主机处理器和远程内存之间的物理距离,能够提供更快速的数据传输速度和更低延迟,而较远的CXL内存则可能产生更高的访问延迟和更慢的数据传输速度,因此,目标应用对处于本地的第一内存空间的操作效率高于目标应用对距离较远的第二内存空间的操作效率。
可选地,在本实施例中,可以但不限于通过以下方式将上述第二类型的第二内存空间接入目标应用所在的系统中:将CXL内存(即第二内存空间)通过PCIe接口接入系统服务器,在服务器开机后,BIOS(Basic Input Output System,基本输入输出系统)识别CXL内存并记录相关CXL内存映射地址以及属性至SRAT表(Static Resource Affinity Table,网络静态资源关联表),并且记录NUMA节点距离。
可选地,在本实施例中,上述目标内存空间可以但不限于包括多个内存页,可以但不限于在内存页上执行数据的存储操作,上述属于第一类型的第一内存空间和属于第二类型的第二内存空间可以但不限于包括大小不同的多个内存页,比如:大小为2G的内存页,大小为1M的内存页,大小为2KB的内存页等等。可以但不限于根据目标应用需要存储的数据量的不同为目标应用分配相应大小以及数量的内存页。
可选地,在本实施例中,上述目标内存空间的内存使用参数用于指示目标内存空间被使用的频率分布。可以但不限于通过以下方法确定目标内存空间的内存使用参数:以目标内存空间中每个内存页为单位,检测目标内存空间中每个内存页在一段时间内使用的次数,再根据每个内存页在一段时间内使用的次数将内存页进行排序,得到目标内存空间中分布在使用频率较高区间的内存页,目标内存空间中分布在使用频率较低区间的内存页以及目标内存空间中分布在使用频率较为平稳的区间的内存页,将根据使用次数得到的区间确定为上述目标内存空间的内存使用参数。
在一个示例性实施例中,可以但不限于采用以下方式检测目标空间的内存使用参数:采集目标内存单元在一个或者多个时间段中每个时间段内的使用频次,得到一个或者多个使用频次,其中,目标内存单元是目标内存空间中已存储数据的多个内存单元中的每个内存单元;根据一个或者多个使用频次计算目标内存单元所对应的使用参数,得到具有对应关系的内存单元和使用参数作为内存使用参数。
可选地,在本实施例中,目标应用可以但不限于包括多个内存单元,可以但不限于将用于存储数据的内存页确定为上述内存单元。可以但不限于将目标应用中存储了数据的内存页确定为目标内存单元。
可选地,在本实施例中,可以但不限于将目标应用中存储了数据的内存页在一个或者多个时间段中每个时间段内使用的次数确定为目标内存单元的使用频次,比如:目标应用中包括N个存储了数据的内存页,N个存储了数据的内存页在一个时间段内使用了M次,因此将M次确定为目标内存单元的使用频次。或者,目标应用中包括N个存储了数据的内存页,N个存储了数据的内存页在P个时间段中每个时间段内使用了M1……MP次,因此将M1次……MP次都确定为目标内存单元的使用频次。
可选地,在本实施例中,可以但不限于根据目标内存单元的使用频次确定其所对应的使用参数,比如:目标应用中包括N个存储了数据的内存页,N个存储了数据的内存页在一个时间段内使用了M次,因此将M次确定为目标内存单元的使用频次,根据使用频次M确定其所对应的使用参数M。或者,目标应用中包括N个存储了数据的内存页,N个存储了数据的内存页在P个时间段中每个时间段内使用了M1……MP次,因此将M1次……MP次都确定为目标内存单元的使用频次,根据使用频次M1……MP确定其所对应的使用参数为M1……MP之和。
在一个示例性实施例中,可以但不限于采用以下方式根据一个或者多个使用频次计算目标内存单元所对应的使用参数:根据每个时间段与当前时间的距离为每个时间段分配权重,其中,每个时间段与当前时间的距离越小,每个时间段所对应的权重越大;计算一个或者多个使用频次的加权和作为目标内存单元所对应的使用参数。
可选地,在本实施例中,可以但不限于根据每个时间段与当前时间的距离确定每个时间段的权重,比如:目标应用中包括N个存储了数据的内存页,N个存储了数据的内存页在时间段P1,时间段P2和时间段P3分别使用了M1,M2和M3次,其对应的权重为T1,T2和T3,由于时间段P1距离当前时间最近,时间段P2与当前时间的距离长于时间段P1与当前时间的距离,时间段P3与当前时间的距离最远,因此T1>T2>T3。
在一个可选的实施方式中,提供了一种目标内存空间的内存使用参数确定过程的示例。图3是根据本申请实施例的一种内存使用参数确定过程的示意图,如图3所示,可以但不限于通过以下方式确定目标内存空间的内存使用参数:
采集目标内存单元在距当前时刻180秒内,确定每30秒为一个时间段的使用频次,得到距当前时刻180秒内每个时间段的使用频次P包括:P30,P60,P90,P120,P150和P180;
根据每个时间段与当前时间时刻的距离为每个时间段分配权重T:T30,T60,T90,T120,T150和T180,其中,T30大于T60,T60大于T90,T90大于T120,T120大于T150,T150大于T180;
将180秒内每个使用频次P的加权T和作为目标内存单元所对应的使用参数S:S=T180*P180+T150*P150+T120*P120+T90*P90+T60*P60+T30*P30。
在一个示例性实施例中,在检测目标内存空间的内存使用参数之前,可以但不限于采用以下方式得到目标内存空间:检测为目标应用所分配的初始内存空间的使用率;根据使用率与使用率阈值之间的关系,调整初始内存空间,得到目标内存空间。
可选地,在本实施例中,目标应用所在的系统可以但不限于在启动该阶段自动为目标应用分配了初始内存空间,上述初始内存空间可以但不限包括属于第一类型的内存空间和属于第二类型的内存空间。
可选地,在本实施例中,上述目标应用的初始内存空间可以但不限于是特定的大小,比如:上述初始内存空间可以但不限于为M大小的第一类型的内存空间和M大小的第二类型的内存空间。或者,M大小的第一类型的内存空间和N大小的第二类型的内存空间,M和N为大于或者等于零的自然数并且M不等于N。
可选地,在本实施例中,上述目标应用的初始内存空间可以但不限于是根据目标应用占用内存的历史情况为其分配合适的初始内存空间,比如:根据目标应用在近期P次使用时对第一类型的内存空间和第二类型的内存空间的最大占用量,根据其最大占用量的中位数为目标应用分配初始内存空间。
可选地,在本实施例中,可以但不限于通过目标应用对内存空间的占用情况确定初始内存空间的使用率,比如:通过Bitmap(位图)管理内存,跟踪目标应用中内存块处于被使用的状态或者空闲的状态。
可选地,在本实施例中,可以但不限于根据目标应用对初始内存空间需求量是否超过初始内存空间确定初始内存空间的使用率。或者,可以但不限于根据目标应用对初始内存空间需求量是否远小于初始内存空间确定初始内存空间的使用率。
可选地,在本实施例中,上述使用率阈值可以但不限于是预先确定的,或者根据目标应用的内存情况实时进行修改的。上述使用率阈值可以但不限于用于确定上述目标应用对初始内存空间的占用情况是否处于平衡状态,比如:在目标应用的初始内存空间的使用率超过使用率阈值的情况下,确定目标应用对初始内存空间需求量大于初始内存空间的容量,用于指示对初始内存空间需要执行扩容操作。或者,在目标应用的初始内存空间的使用率小于使用率阈值的情况下,确定目标应用对初始内存空间需求量远小于初始内存空间的容量,用于指示可以对初始内存空间执行减少一定内容空间的操作。
在一个可选的实施方式中,提供了一种为目标应用分配初始内存空间的过程。图4是根据本申请实施例的为目标应用分配初始内存空间的流程图,如图4所示,可以但不限于通过以下方式为目标应用初始内存空间:
在系统首次启动时,根据设定的固定值针对目标应用分配预分配内存(初始内存空间)包括本地内存容量(第一类型的内存空间)和CXL内存容量(第二类型的内存空间),分配后的内存以红黑树数据结构实现,也就是优先使用距离近的CXL内存,通过应用表记录每个应用对本地内存和CXL内存最大的使用量并计算其中位数,此外也记录每个应用的使用核数均值。通过内存池表记录初始内存空间中各内存页的size_class(表示对于初始内存空间中分配的内存大小需要对齐的size)的个数和偏移量。
在系统非首次启动时,以从应用表获取目标应用在当前启动之前的P=20次使用时,对第一类型的内存空间和第二类型的内存空间的最大占用量为例,设置自动预分配内存(初始内存空间),分配容量为应用表保存的目标应用对第一类型的内存空间和第二类型的内存空间的最大占用量的中位数数据,若无中位数数据,默认设置为200MB本地内存和200MB的CXL内存。
在一个示例性实施例中,根据使用率与使用率阈值之间的关系,调整初始内存空间,可以但不限于采用以下方式得到目标内存空间:在使用率大于或者等于第一使用率阈值的情况下,为目标应用分配第一容量的附加内存空间,得到目标内存空间,其中,目标内存空间包括初始内存空间和附加内存空间;在使用率小于或者等于第二使用率阈值的情况下,从目标内存空间中释放第二容量的内存空间,得到目标内存空间。
可选地,在本实施例中,第一使用率阈值可以但不限于是预先确定的,或者根据目标应用的内存情况实时进行修改的。
可选地,在本实施例中,第一使用率阈值可以但不限于用于确定目标应用对初始内存空间的占用情况是否处于超载状态,比如:在目标应用的初始内存空间的使用率超过或者等于第一使用率阈值的情况下,确定目标应用对初始内存空间需求量大于初始内存空间的容量,对初始内存空间需要执行扩容操作。
可选地,在本实施例中,上述附加内存空间的第一容量可以但不限于根据目标应用对初始内存空间的占用情况确定的。或者,在使用率大于或者等于第一使用率阈值的情况下,为目标应用分配固定容量的附加内存空间,比如:在使用率大于或者等于第一使用率阈值的情况下,为目标应用分配200MB的附加内存空间。
可选地,在本实施例中,上述附加内存空间可以从第一类型的内存空间中获取的。或者,可以从属于第二类型的内存空间中获取的。
可选地,在本实施例中,第二使用率阈值可以但不限于是预先确定的,或者根据目标应用的内存情况实时进行修改的,第二使用率阈值可以但不限于用于指示一定时间内目标应用对内存的占用情况,比如:在一定时间内目标应用未使用初始内存空间的部分内存空间,即可以认为初始内存空间处于空载状态。或者,将在一定时间内初始内存空间中处于占用状态的内存小于或者等于初始内存空间的总容量与一定时间内初始内存空间未使用的部分内存空间的差值确定为第二使用率阈值。
可选地,在本实施例中,第二使用率阈值可以但不限于用于确定目标应用对初始内存空间的占用情况是否处于空载状态,比如:在目标应用的初始内存空间的使用率小于或者等于第二使用率阈值的情况下,确定初始内存空间的容量远大于目标应用对内存空间的需求量,对初始内存空间需要执行减少部分内存空间的操作。
可选地,在本实施例中,上述第二容量可以但不限于根据目标应用对初始内存空间的占用情况确定的。或者,在使用率小于或者等于第二使用率阈值的情况下,减少目标应用中固定容量的内存空间,比如:在使用率小于或者等于第二使用率阈值的情况下,从目标应用中减少200MB的内存空间。
在一个可选的实施方式中,提供了一种对目标应用的初始内存空间进行动态分配的过程,图5是根据本申请实施例的对内存空间进行动态分配的流程图,如图5所示,以设置2组BitMap管理内存池(初始内存空间)中内存的使用情况为例,可以但不限于通过以下过程对目标应用的初始内存空间进行动态分配,其中,2组BitMap分别为A组BitMap和B组BitMap:
A组BitMap实时检测内存是否被使用,应用运行时可弹性分配内存,可以但不限于实时监测内存池中内存的占用情况,如使用内存池(包括第一类型的内存空间和第二类型的内存空间)中的内存超过内存池中相应容量的90%(第一使用率阈值)时,调用内存分配API,增加200MB内存容量(第一容量)。
B组BitMap管理内存一定时间内的使用情况,如内存池中存在5min内未被使用的内存空间,且内存池(包括第一类型的内存空间和第二类型的内存空间)中处于占用状态的内存小于或者等于内存池的总容量与5min内未被使用的内存空间的90%(第二使用率阈值)的差值(第二使用率阈值),则释放未被使用的无用内存(第二容量)。
在一个示例性实施例中,可以但不限于采用以下方式调整初始内存空间:检测目标应用所使用的处理器核心的目标数量;在目标数量大于或者等于数量阈值的情况下,使用第一内存工具调整初始内存空间;在目标数量小于数量阈值的情况下,使用第二内存工具调整初始内存空间,其中,第二内存工具对内存空间的调整效率低于第一内存工具对内存空间的调整效率,但第二内存工具的运行负载小于第一内存工具的运行负载。
可选地,在本实施例中,上述数量阈值可以但不限于是预先设定的,用于根据目标应用所使用的处理器核心的目标数量选择调整内存空间的工具。
可选地,在本实施例中,第二内存工具对内存空间的调整效率低于第一内存工具对内存空间的调整效率,但第二内存工具的运行负载小于第一内存工具的运行负载,比如:第一内存工具可以但不限于为tcmalloc allocator(一种高效的内存分配器,可以更快速和高效地进行内存分配和释放操作),第二内存工具可以但不限于为jemalloc allocator(一种通用的、线程安全的内存分配器)。
在一个可选的实施方式中,提供了一种调整初始内存空间的过程,可以但不限于通过以下方式调整初始内存空间:通过表格记录每个应用所使用的处理器核心的均值,当应用使用处理器核心的数量不高于3时(数量阈值)使用tcmalloc allocator,当应用使用处理器核心的数量高于3时使用jemalloc allocator,若应用表中无记录则默认使用tcmalloc allocator。
在一个示例性实施例中,在检测目标内存空间的内存使用参数之后,可以但不限于采用以下方式:根据内存使用参数和目标迁移参数从第一内存空间中筛选出第四内存空间,其中,第四内存空间是第一内存空间中被使用的频率低于第二频率阈值的内存空间;将第四内存空间中所存储的第二内存数据迁移至第二内存空间。
可选地,在本实施例中,可以但不限于通过目标迁移参数将目标内存空间分为N个区间,用于指示目标内存空间中允许迁移的内存数据在内存空间中的占比,可以但不限于根据第一内存空间和第二内存空间在目标内存空间中的占比确定目标内存空间的目标迁移参数,比如:第一内存空间在目标内存空间的占比为A,第二内存空间在目标内存空间的占比为100-A,目标迁移参数为(A-10)∶20∶(90-A)。或者,第一内存空间在目标内存空间的占比为A,第二内存空间在目标内存空间的占比为100-A,目标迁移参数为A:100-A)。
可选地,在本实施例中,内存使用参数可以但不限于是根据目标内存空间的分配参数确定的,可以但不限于将第一内存空间和第二内存空间在目标内存空间中的占比确定为目标内存空间的分配参数,上述目标内存空间的分配参数可以但不限于是在为目标应用分配的目标内存空间时确定的,比如:为目标应用分配的目标内存空间包括A容量的第一内存空间以及B容量的第二内存空间,目标内存空间的分配参数包括第一内存空间在目标内存空间的占比A/(A+B)以及第二内存空间在目标内存空间的占比B/(A+B)。
可选地,在本实施例中,上述第二频率阈值可以但不限于是预先确定的,用于选择第二内存空间中被使用的频率较高的内存空间。
可选地,在本实施例中,可以但不限于将第一内存空间中使用频率较高的部分内存空间确定为第四内存空间,由于目标应用对第一内存空间的操作效率高于对第二内存空间的操作效率,因此将第一内存空间中使用频率低的部分内存空间迁移至第二内存空间使得目标应用的操作效率增高。
在一个示例性实施例中,可以但不限于采用以下方式根据内存使用参数和目标迁移参数从第一内存空间中筛选出第四内存空间:按照使用参数由低至高提取多个内存单元中满足第三占比的内存单元作为第三内存单元集合,其中,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率,目标迁移参数包括第三占比;从第三内存单元集合中提取属于第一内存空间的内存单元得到第四内存空间。
可选地,在本实施例中,上述使用参数可以但不限于是目标应用的目标内存空间中的每个内存单元的使用频率,比如:目标内存空间中包括多个内存单元,内存单元在一个或者多个时间段内每进行一次操作(可以但不限于包括增加,删除,查找和修改等等)即对内存单元的使用次数+1,得到内存单元在一个或者多个时间段内使用次数的总和,再根据执行操作所在的时间段与当前时间的距离为执行操作次数之和给予权重,从而得到每个内存单元的使用参数。
可选地,在本实施例中,可以但不限于根据使用参数将多个内存单元进行排列,比如:根据使用参数将每个内存单元从高至低进行排列。或者,根据使用参数将每个内存单元从低至高进行排列。
可选地,在本实施例中,上述第三占比用于指示目标内存单元中满足一定比例的内存单元,可以但不限于根据目标迁移参数确定第三占比,比如:目标内存空间的标迁移参数为A∶(100-A),可以但不限于将(100-A)确定为第三占比。或者,目标内存空间的标迁移参数为(A-10)∶20∶(90-A),可以但不限于将(90-A)确定为第三占比。
可选地,在本实施例中,上述第三内存单元集合是目标内存单元中被使用的频率最低的且满足第三占比的内存单元,比如:以第三占比为20%,目标内存空间包括100个内存单元为例,根据内存单元被使用的频率将内存单元按照被使用的频率低的内存单元至被使用的频率低的顺序进行排列,从低至高选择被使用的频率最高的20个内存单元为第三内存单元集合。
可选地,在本实施例中,可以但不限于将被使用的频率最高内存单元中属于第一内存空间的内存单元确定为第四内存空间,比如:以第三内存单元集合包括100个内存单元为例,从第三内存单元集合中查找得到50个属于第一内存空间的内存单元,将第三内存单元集合中50个属于第一内存空间的内存单元确定为第四内存空间。
在一个示例性实施例中,在按照使用参数由低至高提取多个内存单元中满足第三占比的内存单元作为第三内存单元集合之前,可以但不限于采用以下方式确定第三占比:获取目标内存空间的分配参数;根据分配参数确定第三占比。
可选地,在本实施例中,上述目标内存空间的分配参数可以但不限于是第一内存空间和第二内存空间在目标内存空间中的占比,比如:目标内存空间包括占比为A%的第一内存空间和占比为(100%-A%)的第二内存空间,目标内存空间的分配参数包括A%和B%=(100%-A%)。
可选地,在本实施例中,可以但不限于将目标内存空间的分配参数确定为目标内存空间的第三占比,比如:以目标内存空间的分配参数包括A%和B%=(100%-A%)为例,可以但不限于根据使用参数将内存单元进行排列,(100%-A%)为第三占比,从低至高选择最低的A%作为第三内存单元集合。或者,以目标内存空间的分配参数包括A%-C%∶2C%∶100%-C%-A%为例,可以但不限于根据使用参数将内存单元进行排列,100%-C%-A%为第三占比,从高至低选择最低的100%-C%-A%作为第三内存单元集合。
在一个示例性实施例中,可以但不限于采用以下方式获取目标内存空间的分配参数:获取第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第二分配占比;可以但不限于采用以下方式根据分配参数确定第三占比:将第二分配占比与第二预设占比的差值确定为第三占比。
可选地,在本实施例中,可以但不限于将第二内存空间在目标内存空间中的占比确定为第二分配占比。
可选地,在本实施例中,上述第二预设占比可以但不限于是预先确定的,用于从第二分配占比中确定第一占比。
可选地,在本实施例中,可以但不限于根据第二内存空间在目标内存空间中的占比以及第二预设占比确定第三占比,比如:以第二预设占比为20%为例,目标内存空间内第二内存空间占比为A%,第三占比可以但不限于为A%-20%。
在一个示例性实施例中,提供了一种从第一内存空间中筛选第四内存空间的示例。图6是根据本申请实施例的从第一内存空间中筛选第四内存空间的示意图,如图6所示,以第二预设占比为20%为例,可以但不限于通过以下过程确定第四内存空间:
目标内存空间的分配参数第一内存空间:第二内存空间为40%∶60%,根据目标内存空间的分配参数确定目标迁移参数为40%∶60%,根据内存使用参数将目标内存空间由低到高进行排列;
根据目标迁移参数为40%∶60%以及第二预设占比为20%确定第三占比为60%-20%=40%,将后40%确定为第三内存单元集合;
从第三内存单元集合中获取属于第一内存空间的内存单元为第四内存空间。
在一个示例性实施例中,可以但不限于采用以下方式根据内存使用参数和目标迁移参数从第一内存空间中筛选出第四内存空间:按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,第一部分与第二部分呈目标比例,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率;按照使用参数由高至低筛除第二部分中满足第四占比的内存单元得到第四内存单元集合,其中,目标迁移参数包括目标比例和第四占比;从第四内存单元集合中提取属于第一内存空间的内存单元得到第四内存空间。
可选地,在本实施例中,可以但不限于根据目标内存空间中每个内存单元的使用频率将目标内存空间中每个内存单元进行划分,得到使用频率较高的第一部分与使用频率较低的第二部分。
可选地,在本实施例中,第一部分与第二部分的目标比例可以但不限于是根据第一内存空间和第二内存空间在目标内存空间中的占比确定的,比如:以目标内存空间中的第一内存空间占比A%,第二内存空间占比B%=(100%-A%)为例,可以但不限于将目标内存空间按照使用参数由高至低排列,将前A%划分为第一部分,将后B%划分为第二部分。
可选地,在本实施例中,上述第二占比可以但不限于根据从目标内存空间中第一内存空间以及第二内存空间的占比中的一部分内存空间。
可选地,在本实施例中,可以但不限于从第二部分中筛除使用参数最高的第四占比的内存单元得到第四内存单元集合,比如:以第四占比为10%为例,将目标内存空间按照使用参数由高至低排列,将前A%划分为第一部分,将后B%划分为第二部分,再从高至低从第二部分中筛除10%得到第四内存单元集合。
可选地,在本实施例中,可以但不限于将第四内存单元集合中属于第一内存空间的内存单元确定为第四内存空间。
在一个示例性实施例中,在按照使用参数由高至低将多个内存单元划分为第一部分和第二部分之前,可以但不限于采用以下方式:获取目标内存空间的分配参数;根据分配参数确定目标比例和第四占比。
可选地,在本实施例中,上述目标内存空间的分配参数可以但不限于是第一内存空间和第二内存空间在目标内存空间中的占比,比如:目标内存空间包括占比为A%的第一内存空间和占比为(100%-A%)的第二内存空间,目标内存空间的分配参数包括A%和B%=(100%-A%)。
可选地,在本实施例中,上述第四占比可以但不限于根据从目标内存空间中第一内存空间以及第二内存空间的占比中划分得到的,比如:目标内存空间中的第一内存空间占比A%,第二内存空间占比B%=(100%-A%)为例,可以但不限于从第一内存空间占比A%中划分C%作为第四占比。或者,可以但不限于从第二内存空间占比(100%-A%)中划分C%作为第四占比。或者,可以但不限于从第一内存空间占比A%中划分C%再第二内存空间占比(100%-A%)中划分D%,将C%与D%作为第四占比。
在一个示例性实施例中,可以但不限于采用以下方式获取目标内存空间的分配参数:获取第一内存空间在目标内存空间中的第一分配占比,和第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第一分配占比和第二分配占比;可以但不限于采用以下方式根据分配参数确定目标比例和第四占比,包括:将第一分配占比与第二分配占比确定为目标比例;获取第四预设占比作为第四占比。
可选地,在本实施例中,可以但不限于将第一内存空间在目标内存空间中的占比确定为第一分配占比,将第二内存空间在目标内存空间中的占比确定为第二分配占比,可以但不限于将第一分配占比与第二分配占比确定为分配参数。
可选地,在本实施例中,上述第四预设占比可以但不限于是预先确定的,第四预设占比可以但不限于用于确定第四占比。
可选地,在本实施例中,可以但不限于根据第一内存空间在目标内存空间中的占比,第二内存空间在目标内存空间中的占比以及第四预设占比确定第四占比,比如:以第四预设占比为20%为例,目标内存空间内第一内存空间占比为A%-10%,目标内存空间内第二内存空间占比为90%-A%,可以但不限于确定第四占比为20%。
在上述步骤S204提供的技术方案中,可以但不限于将第一内存空间和第二内存空间在目标内存空间中的占比确定为目标内存空间的分配参数,上述目标内存空间的分配参数可以但不限于是在为目标应用分配的目标内存空间时确定的,比如:为目标应用分配的目标内存空间包括A容量的第一内存空间以及B容量的第二内存空间,目标内存空间的分配参数包括第一内存空间在目标内存空间的占比A/(A+B)以及第二内存空间在目标内存空间的占比B/(A+B)。
可选地,在本实施例中,可以但不限于通过目标迁移参数将目标内存空间分为N个区间,用于指示目标内存空间中允许迁移的内存数据在内存空间中的占比,可以但不限于根据第一内存空间和第二内存空间在目标内存空间中的占比确定目标内存空间的目标迁移参数,比如:第一内存空间在目标内存空间的占比为A,第二内存空间在目标内存空间的占比为100-A,目标迁移参数为(A-10)∶20∶(90-A)。或者,第一内存空间在目标内存空间的占比为A,第二内存空间在目标内存空间的占比为100-A,目标迁移参数为A∶(100-A)。
可选地,在本实施例中,上述第一频率阈值可以但不限于是预先确定的,第一频率阈值可以但不限于用于选择第二内存空间中被使用的频率较高的内存空间。
可选地,在本实施例中,可以但不限于将第二内存空间中使用频率较高的部分内存空间确定为第三内存空间,由于目标应用对第一内存空间的操作效率高于对第二内存空间的操作效率,因此将第二内存空间中使用频率较高的部分内存空间迁移至第一内存空间使得目标应用的操作效率增高。
在一个示例性实施例中,可以但不限于采用以下方式根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间:按照使用参数由高至低提取多个内存单元中满足第一占比的内存单元作为第一内存单元集合,其中,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率,目标迁移参数包括第一占比;从第一内存单元集合中提取属于第二内存空间的内存单元,得到第三内存空间。
可选地,在本实施例中,上述使用参数可以但不限于是目标应用的目标内存空间中的每个内存单元的使用频率,比如:目标内存空间中包括多个内存单元,内存单元在一个或者多个时间段内每进行一次操作(可以但不限于包括增加,删除,查找和修改等等)即对内存单元的使用次数+1,得到内存单元在一个或者多个时间段内使用次数的总和,再根据执行操作所在的时间段与当前时间的距离为执行操作次数之和给予权重,从而得到每个内存单元的使用参数。
可选地,在本实施例中,可以但不限于根据使用参数将多个内存单元进行排列,比如:根据使用参数将每个内存单元从高至低进行排列。或者,根据使用参数将每个内存单元从低至高进行排列。
可选地,在本实施例中,上述第一占比用于指示目标内存单元中满足一定比例的内存单元,可以但不限于根据目标迁移参数确定第一占比,比如:目标内存空间的标迁移参数为A∶(100-A),可以但不限于将A确定为第一占比。或者,目标内存空间的标迁移参数为(A-10)∶20∶(90-A),可以但不限于将(A-10)确定为第一占比。
可选地,在本实施例中,上述第一内存单元集合是目标内存单元中被使用的频率最高的第一占比的内存单元,比如:以第一占比为20%,目标内存空间包括100个内存单元为例,根据内存单元被使用的频率将内存单元按照被使用的频率高的内存单元至被使用的频率低的顺序进行排列,从高至低选择被使用的频率最高的20个内存单元为第一内存单元集合。
可选地,在本实施例中,可以但不限于将被使用的频率最高内存单元中属于第二内存空间的内存单元确定为第三内存空间,比如:以第一内存单元集合包括100个内存单元为例,从第一内存单元集合中查找得到50个属于第二内存空间的内存单元,将第一内存单元集合中50个属于第二内存空间的内存单元确定为第三内存空间。
在一个示例性实施例中,在按照使用参数由高至低提取多个内存单元中满足第一占比的内存单元作为第一内存单元集合之前,可以但不限于采用以下方式确定第一占比:获取目标内存空间的分配参数;根据分配参数确定第一占比。
可选地,在本实施例中,上述目标内存空间的分配参数可以但不限于是第一内存空间和第二内存空间在目标内存空间中的占比,比如:目标内存空间包括占比为A%的第一内存空间和占比为(100%-A%)的第二内存空间,目标内存空间的分配参数包括A%和B%=(100%-A%)。
可选地,在本实施例中,可以但不限于将目标内存空间的分配参数确定为目标内存空间的第一占比,比如:以目标内存空间的分配参数包括A%和B%=(100%-A%)为例,可以但不限于根据使用参数将内存单元进行排列,A%为第一占比,从高至低选择前A%作为第一内存单元集合。或者,以目标内存空间的分配参数包括A%-C%:2C%:100%-C%-A%为例,可以但不限于根据使用参数将内存单元进行排列,A%-C%为第一占比,从高至低选择前A%-C%作为第一内存单元集合。
在一个示例性实施例中,可以但不限于采用以下方式获取目标内存空间的分配参数:获取第一内存空间在目标内存空间中的第一分配占比,其中,分配参数包括第一分配占比;可以但不限于采用以下方式根据分配参数确定第一占比:将第一分配占比与第一预设占比的差值确定为第一占比。
可选地,在本实施例中,可以但不限于将第一内存空间在目标内存空间中的占比确定为第一分配占比。
可选地,在本实施例中,上述第一预设占比可以但不限于是预先确定的,用于从第一分配占比中确定第一占比。
可选地,在本实施例中,可以但不限于根据第一内存空间在目标内存空间中的占比以及第一预设占比确定第一占比,比如:以第一预设占比为20%为例,目标内存空间内第一内存空间占比为A%,第一占比为A%-20%。
在一个示例性实施例中,提供了一种从第二内存空间中筛选第三内存空间的示例。图7是根据本申请实施例的从第二内存空间中筛选第三内存空间的示意图,如图7所示,以第一预设占比为20%为例,可以但不限于通过以下过程确定第三内存空间:
目标内存空间的分配参数第一内存空间:第二内存空间为60%∶40%,根据目标内存空间的分配参数确定目标迁移参数为60%∶40%,根据内存使用参数将目标内存空间由高到低进行排列;
根据目标迁移参数为60%∶40%以及第一预设占比为20%确定第一占比为60%-20%=40%,将前40%确定为第一内存单元集合;
从第一内存单元集合中获取属于第二内存空间的内存单元为第三内存空间。
在一个示例性实施例中,可以但不限于采用以下方式根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间:按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,第一部分与第二部分呈目标比例,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率;按照使用参数由低至高筛除第一部分中满足第二占比的内存单元得到第二内存单元集合,其中,目标迁移参数包括目标比例和第二占比;从第二内存单元集合中提取属于第二内存空间的内存单元,得到第三内存空间。
可选地,在本实施例中,可以但不限于根据目标内存空间中每个内存单元的使用频率将目标内存空间中每个内存单元进行划分,得到使用频率较高的第一部分与使用频率较低的第二部分。
可选地,在本实施例中,第一部分与第二部分的目标比例可以但不限于是根据第一内存空间和第二内存空间在目标内存空间中的占比确定的,比如:以目标内存空间中的第一内存空间占比A%,第二内存空间占比B%=(100%-A%)为例,可以但不限于将目标内存空间按照使用参数由高至低排列,将前A%划分为第一部分,将后B%划分为第二部分。
可选地,在本实施例中,上述第二占比可以但不限于根据从目标内存空间中第一内存空间以及第二内存空间的占比中的一部分内存空间。
可选地,在本实施例中,可以但不限于从第一部分中筛除使用参数最低的第二占比的内存单元得到第二内存单元集合,比如:以第二占比为10%为例,将目标内存空间按照使用参数由高至低排列,将前A%划分为第一部分,将后B%划分为第二部分,再从低至高从第一部分中筛除10%得到第二内存单元集合。
可选地,在本实施例中,可以但不限于将第二内存单元集合中属于第二内存空间的内存单元确定为第三内存空间。
在一个示例性实施例中,在按照使用参数由高至低将多个内存单元划分为第一部分和第二部分之前,可以但不限于采用以下方式确定目标比例和第二占比:获取目标内存空间的分配参数;根据分配参数确定目标比例和第二占比。
可选地,在本实施例中,上述目标内存空间的分配参数可以但不限于是第一内存空间和第二内存空间在目标内存空间中的占比,比如:目标内存空间包括占比为A%的第一内存空间和占比为(100%-A%)的第二内存空间,目标内存空间的分配参数包括A%和B%=(100%-A%)。
可选地,在本实施例中,上述第二占比可以但不限于根据从目标内存空间中第一内存空间以及第二内存空间的占比中划分得到的,比如:目标内存空间中的第一内存空间占比A%,第二内存空间占比B%=(100%-A%)为例,可以但不限于从第一内存空间占比A%中划分C%作为第二占比。或者,可以但不限于从第二内存空间占比(100%-A%)中划分C%作为第二占比。或者,可以但不限于从第一内存空间占比A%中划分C%再第二内存空间占比(100%-A%)中划分D%,将C%与D%作为第二占比。
在一个示例性实施例中,可以但不限于采用以下方式获取目标内存空间的分配参数:获取第一内存空间在目标内存空间中的第一分配占比,和第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第一分配占比和第二分配占比;可以但不限于采用以下方式根据分配参数确定目标比例和第二占比:将第一分配占比与第二分配占比确定为目标比例;获取第二预设占比作为第二占比。
可选地,在本实施例中,可以但不限于将第一内存空间在目标内存空间中的占比确定为第一分配占比,将第二内存空间在目标内存空间中的占比确定为第二分配占比,可以但不限于将第一分配占比与第二分配占比确定为分配参数。
可选地,在本实施例中,可以但不限于根据第一内存空间在目标内存空间中的占比,第二内存空间在目标内存空间中的占比以及第二预设占比确定第二占比,比如:以第二预设占比为20%为例,目标内存空间内第一内存空间占比为A%-10%,目标内存空间内第二内存空间占比为90%-A%,可以但不限于确定第二占比为20%。
在上述步骤S206提供的技术方案中,可以但不限于将第三内存空间中所存储的第一内存数据复制至第一内存空间,实现将第三内存空间中所存储的第一内存数据迁移至第一内存空间。
在一个示例性实施例中,可以但不限于采用以下方式将第三内存空间中所存储的第一内存数据迁移至第一内存空间:检测第一内存空间的参考剩余容量;在第一内存数据的数据量大于参考剩余容量的情况下,从第一内存数据中全部被使用的内存单元中筛选数据量满足参考剩余容量的目标内存数据;将目标内存数据迁移至第一内存空间。
可选地,在本实施例中,上述参考剩余容量可以但不限于为第一内存空间未使用的内存空间,或者是在不影响第一内存空间使用的前提下,第一内存空间能够容纳的数据量。
可选地,在本实施例中,可以但不限于将第一内存数据中前90%确定为上述目标内存数据,或者可以但不限于将第一内存数据中内存使用参数最高的内存数据确定为上述目标内存数据。
在一个示例性实施例中,提供了一种在第一内存空间和第二内存空间中执行内存数据迁移的示例。图8是根据本申请实施例的在第一内存空间和第二内存空间中执行内存数据迁移的示意图,如图8所示,以在Linux系统中执行内存数据迁移操作,第一内存空间为本地内存,第二内存空间为CXL内存为例,为了提高Linux系统的内存使用效率,本申请中了设定温内存并且保持温内存不进行迁移,从而减少了Linux系统使用过程中使用的频次处于中间位置的内存页进行内存迁移的次数。
如图8所示,Linux系统中包括SDK(Software Development Kit,软件开发工具包)和HW(Hardware,硬件),本地内存和CXL内存部署在HW中,SDK中包括Linux内核模块、内存池化模块、内存分层模块、内存迁移模块、内存识别模块以及API模块。
Linux内核模块可以但不限于是基于Linux6.2开发驱动以及CXL内存相关接口。包括内存识别模块,可以但不限于通过Linux内核模块识别CXL内存容量和地址,比如:Linux内核模块在操作系统的启动过程中将CXL内存识别为RAM,识别并动修改CXL内存属性为soft reserved,在buddyinfo中将本地内存与CXL内存区分映射为不同组(Normal,CxlMemory),读取CXL内存距离,记录CXL内存使用的插槽数并据此计算CXL内存可提供带宽量,并提供修改NUMA节点的API。
内存池化模块被设置为在系统启动时为每个应用分配内存,内存池化模块可以但不限于采用tcmalloc allocator与jemalloc allocator并用的内存分配模式,比如:主动设置为应用分配容量为1GB的本地内存与1GB的CXL内存。内存池化模块包括被设置为存储各个应用使用时对本地内存和CXL内存的最大使用量的应用表,可以但不限于根据应用表中存储的最大使用量为应用分配空间。
内存分层模块可以但不限于被设置为记录目标内存空间的内存使用参数,并且根据内存使用参数和目标迁移参数筛选出需要迁移的内存空间。
内存迁移模块被设置为将内存分层模块筛选出的需要迁移的内存空间进行迁移。
以将CXL内存中的热内存数据(频次处于高位的内存页)存入本地内存,可以但不限于通过以下方式在第一内存空间和第二内存空间中执行内存数据迁移:
首先,计算内存池(目标内存空间)中本地内存剩余容量a,将CXL内存的热内存数据(第三内存空间)大小不超过a的90%的部分打包为数据块,统一复制入本地内存,将用于记录内存分配情况的分层表和用于管理内存的BitMap进行相应转换、整合,将已经复制了的原CXL内存中的热内存对应位置清0;当本地内存使用量到达90%时暂停存入,转而将本地内存的冷内存数据(第四内存空间)复制存入CXL内存;
首先,计算池(目标内存空间)中CXL内存剩余容量b,将本地内存的冷内存数据大小不超过b的90%的部分打包为数据块,统一复制入CXL内存,然后将用于记录内存分配情况的分层表和用于管理内存的BitMap进行相应转换、整合,将已经复制了的原本地内存的冷内存对应位置清0;
当CXL内存使用量到达90%时暂停存入,转而继续将CXL内存的热内存数据存入本地内存;直至CXL内存的热内存数据全部存入本地内存且本地内存的冷数据全部存入CXL内存。
在进行内存页迁移的过程中,可以但不限于优先迁移已经全部被使用的内存页,再迁移其他内存页。可以但不限于对内存页的class_size(规模)的记录情况设置迁移颗粒度,根据迁移颗粒度执行内存页的迁移,以需要将CXL内存中的热内存页1迁移至本地内存为例,表1是根据本申请实施例的内存页1的规模的记录表,如表1所示,先设置内存页1迁移颗粒度为256Byte(字节),然后根据偏移量迁移页1中被分配的Offset(偏移量)查找到已经使用了的内存块,将其迁移至本地内存,之后依次更新迁移颗粒度,然后根据页1中被分配的Offset查找到其他已经使用了的内存块,将其迁移至本地内存,直至完成当前内存页1的迁移。
通过使用API为应用预分配内存,开启,关闭以及设置内存分配以及迁移的情况,统计BitMap查看本地内存、CXL内存的使用情况。
表1
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个非易失性可读存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,服务器,或者网络设备等)执行本申请各个实施例的方法。
在本实施例中还提供了一种内存数据的迁移装置,该装置被设置为实现上述实施例及可选实施方式,已经进行过说明的不再赘述。如以下所使用的,术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现,但是硬件,或者软件和硬件的组合的实现也是可能并被构想的。
图9是根据本申请实施例的一种内存数据的迁移装置的结构框图,如图9所示,该装置包括:
第一检测模块92,被设置为检测目标内存空间的内存使用参数,其中,目标内存空间是为待迁移内存数据的目标应用所分配的内存空间,目标内存空间中包括属于第一类型的第一内存空间和属于第二类型的第二内存空间,内存使用参数用于指示目标内存空间被使用的频率分布,目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率;
第一筛选模块94,被设置为根据内存使用参数和目标迁移参数从第二内存空间中筛选出第三内存空间,其中,目标迁移参数是根据目标内存空间的分配参数确定的,分配参数用于指示第一内存空间和第二内存空间在目标内存空间中的占比,目标迁移参数用于指示目标内存空间中允许迁移内存数据的内存空间的占比,第三内存空间是第二内存空间中被使用的频率高于第一频率阈值的内存空间;
第一迁移模块96,被设置为将第三内存空间中所存储的第一内存数据迁移至第一内存空间。
通过上述装置,目标应用使用的目标内存空间包括第一内存空间和第二内存空间,根据目标内存空间中被使用的频率分布,以及目标内存空间的分配参数从目标内存空间中筛选出第二内存空间中被使用的频率高的内存空间即需要迁移的第三内存空间,由于目标应用对第一内存空间的操作效率高于目标应用对第二内存空间的操作效率,通过将第二内存空间中使用效率高的部分内存空间中存储的第一内存数据迁移至第一内存空间,提高了目标应用对使用效率高的内存数据的操作效率,从而提高了目标应用的操作效率。因此,可以解决内存使用的效率较低问题,达到提高内存使用的效率的效果。
在一个示例性实施例中,第一筛选模块,包括:
第一提取单元,被设置为按照使用参数由高至低提取多个内存单元中满足第一占比的内存单元作为第一内存单元集合,其中,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率,目标迁移参数包括第一占比;
第二提取单元,被设置为从第一内存单元集合中提取属于第二内存空间的内存单元,得到第三内存空间。
在一个示例性实施例中,装置还包括:
第一获取模块,被设置为获取目标内存空间的分配参数;
第一确定模块,被设置为根据分配参数确定第一占比。
在一个示例性实施例中,第一获取模块,包括:
第一获取单元,被设置为获取第一内存空间在目标内存空间中的第一分配占比,其中,分配参数包括第一分配占比;
在一个示例性实施例中,第一确定模块,包括:
第一确定单元,被设置为将第一分配占比与第一预设占比的差值确定为第一占比。
在一个示例性实施例中,第一筛选模块,包括:
第一划分单元,被设置为按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,第一部分与第二部分呈目标比例,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率;
第一筛除单元,被设置为按照使用参数由低至高筛除第一部分中满足第二占比的内存单元得到第二内存单元集合,其中,目标迁移参数包括目标比例和第二占比;
第三提取单元,被设置为从第二内存单元集合中提取属于第二内存空间的内存单元,得到第三内存空间。
在一个示例性实施例中,装置还包括:
第二获取模块,被设置为获取目标内存空间的分配参数;
第二确定模块,被设置为根据分配参数确定目标比例和第二占比。
在一个示例性实施例中,第二获取模块,包括:
第二获取单元,被设置为获取第一内存空间在目标内存空间中的第一分配占比,和第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第一分配占比和第二分配占比;
在一个示例性实施例中,第二确定模块,包括:
第二确定单元,被设置为将第一分配占比与第二分配占比确定为目标比例;获取第二预设占比作为第二占比。
在一个示例性实施例中,装置还包括:
第二筛选模块,被设置为根据内存使用参数和目标迁移参数从第一内存空间中筛选出第四内存空间,其中,第四内存空间是第一内存空间中被使用的频率低于第二频率阈值的内存空间;
第二迁移模块,被设置为将第四内存空间中所存储的第二内存数据迁移至第二内存空间。
在一个示例性实施例中,第二筛选模块,包括:
第四提取单元,被设置为按照使用参数由低至高提取多个内存单元中满足第三占比的内存单元作为第三内存单元集合,其中,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率,目标迁移参数包括第三占比;
第五提取单元,被设置为从第三内存单元集合中提取属于第一内存空间的内存单元得到第四内存空间。
在一个示例性实施例中,装置还包括:
第三获取模块,被设置为获取目标内存空间的分配参数;
第三确定模块,被设置为根据分配参数确定第三占比。
在一个示例性实施例中,第三获取模块,包括:
第三获取单元,被设置为获取第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第二分配占比;
在一个示例性实施例中,第三确定模块,包括:
第三确定单元,被设置为将第二分配占比与第二预设占比的差值确定为第三占比。
在一个示例性实施例中,第二筛选模块,包括:
第二划分单元,被设置为按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,第一部分与第二部分呈目标比例,多个内存单元是目标内存空间中已存储数据的内存单元,内存使用参数包括具有对应关系的内存单元和使用参数,使用参数用于指示对应的内存单元被使用的频率;
第二筛除单元,被设置为按照使用参数由高至低筛除第二部分中满足第四占比的内存单元得到第四内存单元集合,其中,目标迁移参数包括目标比例和第四占比;
第六提取单元,被设置为从第四内存单元集合中提取属于第一内存空间的内存单元得到第四内存空间。
在一个示例性实施例中,装置还包括:
第四获取模块,被设置为获取目标内存空间的分配参数;
第四确定模块,被设置为根据分配参数确定目标比例和第四占比。
在一个示例性实施例中,第四获取模块,包括:
第四获取单元,被设置为获取第一内存空间在目标内存空间中的第一分配占比,和第二内存空间在目标内存空间中的第二分配占比,其中,分配参数包括第一分配占比和第二分配占比;
在一个示例性实施例中,第四确定模块,包括:
第四确定单元,被设置为将第一分配占比与第二分配占比确定为目标比例;获取第四预设占比作为第四占比。
在一个示例性实施例中,第一迁移模块,包括:
第一检测单元,被设置为检测第一内存空间的参考剩余容量;
筛选单元,被设置为在第一内存数据的数据量大于参考剩余容量的情况下,从第一内存数据中全部被使用的内存单元中筛选数据量满足参考剩余容量的目标内存数据;
迁移单元,被设置为将目标内存数据迁移至第一内存空间。
在一个示例性实施例中,第一检测模块,包括:
采集单元,被设置为采集目标内存单元在一个或者多个时间段中每个时间段内的使用频次,得到一个或者多个使用频次,其中,目标内存单元是目标内存空间中已存储数据的多个内存单元中的每个内存单元;
计算单元,被设置为根据一个或者多个使用频次计算目标内存单元所对应的使用参数,得到具有对应关系的内存单元和使用参数作为内存使用参数。
在一个示例性实施例中,计算单元,还被设置为:根据每个时间段与当前时间的距离为每个时间段分配权重,其中,每个时间段与当前时间的距离越小,每个时间段所对应的权重越大;计算一个或者多个使用频次的加权和作为目标内存单元所对应的使用参数。
在一个示例性实施例中,装置还包括:
第二检测模块,被设置为检测为目标应用所分配的初始内存空间的使用率;
调整模块,被设置为根据使用率与使用率阈值之间的关系,调整初始内存空间,得到目标内存空间。
在一个示例性实施例中,调整模块,包括:
分配单元,被设置为在使用率大于或者等于第一使用率阈值的情况下,为目标应用分配第一容量的附加内存空间,得到目标内存空间,其中,目标内存空间包括初始内存空间和附加内存空间;
释放单元,被设置为在使用率小于或者等于第二使用率阈值的情况下,从目标内存空间中释放第二容量的内存空间,得到目标内存空间。
在一个示例性实施例中,调整模块,包括:
第二检测单元,被设置为检测目标应用所使用的处理器核心的目标数量;
第一调整单元,被设置为在目标数量大于或者等于数量阈值的情况下,使用第一内存工具调整初始内存空间;
第二调整单元,被设置为在目标数量小于数量阈值的情况下,使用第二内存工具调整初始内存空间,其中,第二内存工具对内存空间的调整效率低于第一内存工具对内存空间的调整效率,但第二内存工具的运行负载小于第一内存工具的运行负载。
需要说明的是,上述各个模块是可以通过软件或硬件来实现的,对于后者,可以通过以下方式实现,但不限于此:上述模块均位于同一处理器中;或者,上述各个模块以任意组合的形式分别位于不同的处理器中。
本申请的实施例还提供了一种计算机非易失性可读存储介质,该计算机非易失性可读存储介质中存储有计算机程序,其中,该计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
在一个示例性实施例中,上述计算机非易失性可读存储介质可以包括但不限于:U盘、只读存储器(Read-Only Memory,简称为ROM)、随机存取存储器(Random Access Memory,简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储计算机程序的介质。
本申请的实施例还提供了一种电子设备,图10是根据本申请实施例的电子设备的示意图,如图10所示,包括存储器和处理器,该存储器中存储有计算机程序,该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
在一个示例性实施例中,上述电子设备还可以包括传输设备以及输入输出设备,其中,该传输设备和上述处理器连接,该输入输出设备和上述处理器连接。
本实施例中的示例可以参考上述实施例及示例性实施方式中所描述的示例,本实施例在此不再赘述。
显然,本领域的技术人员应该明白,上述的本申请的各模块或各步骤可以用通用的计算装置来实现,它们可以集中在单个的计算装置上,或者分布在多个计算装置所组成的网络上,它们可以用计算装置可执行的程序代码来实现,从而,可以将它们存储在存储装置中由计算装置来执行,并且在某些情况下,可以以不同于此处的顺序执行所示出或描述的步骤,或者将它们分别制作成各个集成电路模块,或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样,本申请不限制于任何特定的硬件和软件结合。
以上仅为本申请的可选实施例而已,并不用于限制本申请,对于本领域的技术人员来说,本申请可以有各种更改和变化。凡在本申请的原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。

Claims (23)

  1. 一种内存数据的迁移方法,其特征在于,包括:
    检测目标内存空间的内存使用参数,其中,所述目标内存空间是为待迁移内存数据的目标应用所分配的内存空间,所述目标内存空间中包括属于第一类型的第一内存空间和属于第二类型的第二内存空间,所述内存使用参数用于指示所述目标内存空间被使用的频率分布,所述目标应用对所述第一内存空间的操作效率高于所述目标应用对所述第二内存空间的操作效率;
    根据所述内存使用参数和目标迁移参数从所述第二内存空间中筛选出第三内存空间,其中,所述目标迁移参数是根据所述目标内存空间的分配参数确定的,所述分配参数用于指示所述第一内存空间和所述第二内存空间在所述目标内存空间中的占比,所述目标迁移参数用于指示所述目标内存空间中允许迁移内存数据的内存空间的占比,所述第三内存空间是所述第二内存空间中被使用的频率高于第一频率阈值的内存空间;
    将所述第三内存空间中所存储的第一内存数据迁移至所述第一内存空间。
  2. 根据权利要求1所述的方法,其特征在于,所述根据所述内存使用参数和目标迁移参数从所述第二内存空间中筛选出第三内存空间,包括:
    按照使用参数由高至低提取多个内存单元中满足第一占比的内存单元作为第一内存单元集合,其中,所述多个内存单元是所述目标内存空间中已存储数据的内存单元,所述内存使用参数包括具有对应关系的内存单元和使用参数,所述使用参数用于指示对应的内存单元被使用的频率,所述目标迁移参数包括所述第一占比;
    从所述第一内存单元集合中提取属于所述第二内存空间的内存单元,得到所述第三内存空间。
  3. 根据权利要求2所述的方法,其特征在于,在所述按照使用参数由高至低提取多个内存单元中满足第一占比的内存单元作为第一内存单元集合之前,所述方法还包括:
    获取所述目标内存空间的所述分配参数;
    根据所述分配参数确定所述第一占比。
  4. 根据权利要求3所述的方法,其特征在于,
    所述获取所述目标内存空间的所述分配参数,包括:获取所述第一内存空间在所述目标内存空间中的第一分配占比,其中,所述分配参数包括所述第一分配占比;
    所述根据所述分配参数确定所述第一占比,包括:将所述第一分配占比与第一预设占比的差值确定为所述第一占比。
  5. 根据权利要求1所述的方法,其特征在于,所述根据所述内存使用参数和目标迁移参数从所述第二内存空间中筛选出第三内存空间,包括:
    按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,所述第一部分与所述第二部分呈目标比例,所述多个内存单元是所述目标内存空间中已存储数据的内存单元,所述内存使用参数包括具有对应关系的内存单元和使用参数,所述使用参数用于指示对应的内存单元被使用的频率;
    按照使用参数由低至高筛除所述第一部分中满足第二占比的内存单元得到第二内存单元集合,其中,所述目标迁移参数包括所述目标比例和所述第二占比;
    从所述第二内存单元集合中提取属于所述第二内存空间的内存单元,得到所述第三内存空间。
  6. 根据权利要求5所述的方法,其特征在于,在所述按照使用参数由高至低将多个内存单元划分为第一部分和第二部分之前,所述方法还包括:
    获取所述目标内存空间的所述分配参数;
    根据所述分配参数确定所述目标比例和所述第二占比。
  7. 根据权利要求6所述的方法,其特征在于,
    所述获取所述目标内存空间的所述分配参数,包括:获取所述第一内存空间在所述目标内存空间中的第一分配占比,和所述第二内存空间在所述目标内存空间中的第二分配占比,其中,所述分配参数包括所述第一分配占比和所述第二分配占比;
    所述根据所述分配参数确定所述目标比例和所述第二占比,包括:将所述第一分配占比与所述第二分配占比确定为所述目标比例;获取第二预设占比作为所述第二占比。
  8. 根据权利要求1所述的方法,其特征在于,在所述检测目标内存空间的内存使用参数之后,所述方法还包括:
    根据所述内存使用参数和目标迁移参数从所述第一内存空间中筛选出第四内存空间,其中,所述第四内存空间是所述第一内存空间中被使用的频率低于第二频率阈值的内存空间;
    将所述第四内存空间中所存储的第二内存数据迁移至所述第二内存空间。
  9. 根据权利要求8所述的方法,其特征在于,所述根据所述内存使用参数和目标迁移参数从所述第一内存空间中筛选出第四内存空间,包括:
    按照使用参数由低至高提取多个内存单元中满足第三占比的内存单元作为第三内存单元集合,其中,所述多个内存单元是所述目标内存空间中已存储数据的内存单元,所述内存使用参数包括具有对应关系的内存单元和使用参数,所述使用参数用于指示对应的内存单元被使用的频率,所述目标迁移参数包括所述第三占比;
    从所述第三内存单元集合中提取属于所述第一内存空间的内存单元得到所述第四内存空间。
  10. 根据权利要求9所述的方法,其特征在于,在所述按照使用参数由低至高提取多个内存单元中满足第三占比的内存单元作为第三内存单元集合之前,所述方法还包括:
    获取所述目标内存空间的所述分配参数;
    根据所述分配参数确定所述第三占比。
  11. 根据权利要求10所述的方法,其特征在于,
    所述获取所述目标内存空间的所述分配参数,包括:获取所述第二内存空间在所述目标内存空间中的第二分配占比,其中,所述分配参数包括所述第二分配占比;
    所述根据所述分配参数确定所述第三占比,包括:将所述第二分配占比与第二预设占比的差值确定为所述第三占比。
  12. 根据权利要求8所述的方法,其特征在于,所述根据所述内存使用参数和目标迁移参数从所述第一内存空间中筛选出第四内存空间,包括:
    按照使用参数由高至低将多个内存单元划分为第一部分和第二部分,其中,所述第一部分与所述第二部分呈目标比例,所述多个内存单元是所述目标内存空间中已存储数据的内存单元,所述内存使用参数包括具有对应关系的内存单元和使用参数,所述使用参数用于指示对应的内存单元被使用的频率;
    按照所述使用参数由高至低筛除所述第二部分中满足第四占比的内存单元得到第四内存单元集合,其中,所述目标迁移参数包括所述目标比例和所述第四占比;
    从所述第四内存单元集合中提取属于所述第一内存空间的内存单元得到所述第四内存空间。
  13. 根据权利要求12所述的方法,其特征在于,在所述按照使用参数由高至低将多个内存单元划分为第一部分和第二部分之前,所述方法还包括:
    获取所述目标内存空间的所述分配参数;
    根据所述分配参数确定所述目标比例和所述第四占比。
  14. 根据权利要求13所述的方法,其特征在于,
    所述获取所述目标内存空间的所述分配参数,包括:获取所述第一内存空间在所述目标内存空间中的第一分配占比,和所述第二内存空间在所述目标内存空间中的第二分配占比,其中,所述分配参数包括所述第一分配占比和所述第二分配占比;
    所述根据所述分配参数确定所述目标比例和所述第四占比,包括:将所述第一分配占比与所述第二分配占比确定为所述目标比例;获取第四预设占比作为所述第四占比。
  15. 根据权利要求1所述的方法,其特征在于,所述将所述第三内存空间中所存储的第一内存数据迁移至所述第一内存空间,包括:
    检测所述第一内存空间的参考剩余容量;
    在所述第一内存数据的数据量大于所述参考剩余容量的情况下,从所述第一内存数据中全部被使用的内存单元中筛选数据量满足所述参考剩余容量的目标内存数据;
    将所述目标内存数据迁移至所述第一内存空间。
  16. 根据权利要求1所述的方法,其特征在于,所述检测目标内存空间的内存使用参数,包括:
    采集目标内存单元在一个或者多个时间段中每个时间段内的使用频次,得到一个或者多个使用频次,其中,所述目标内存单元是所述目标内存空间中已存储数据的多个内存单元中的每个内存单元;
    根据所述一个或者多个使用频次计算所述目标内存单元所对应的使用参数,得到具有对应关系的内存单元和使用参数作为所述内存使用参数。
  17. 根据权利要求16所述的方法,其特征在于,所述根据所述一个或者多个使用频次计算所述目标内存单元所对应的使用参数,包括:
    根据所述每个时间段与当前时间的距离为所述每个时间段分配权重,其中,所述每个时间段与当前时间的距离越小,所述每个时间段所对应的权重越大;
    计算所述一个或者多个使用频次的加权和作为所述目标内存单元所对应的使用参数。
  18. 根据权利要求1所述的方法,其特征在于,在所述检测目标内存空间的内存使用参数之前,所述方法还包括:
    检测为所述目标应用所分配的初始内存空间的使用率;
    根据所述使用率与使用率阈值之间的关系,调整所述初始内存空间,得到所述目标内存空间。
  19. 根据权利要求18所述的方法,其特征在于,所述根据所述使用率与使用率阈值之间的关系,调整所述初始内存空间,得到所述目标内存空间,包括:
    在所述使用率大于或者等于第一使用率阈值的情况下,为所述目标应用分配第一容量的附加内存空间,得到所述目标内存空间,其中,所述目标内存空间包括所述初始内存空间和所述附加内存空间;
    在所述使用率小于或者等于第二使用率阈值的情况下,从所述目标内存空间中释放第二容量的内存空间,得到所述目标内存空间。
  20. 根据权利要求18所述的方法,其特征在于,所述调整所述初始内存空间,包括:
    检测所述目标应用所使用的处理器核心的目标数量;
    在所述目标数量大于或者等于数量阈值的情况下,使用第一内存工具调整所述初始内存空间;
    在所述目标数量小于所述数量阈值的情况下,使用第二内存工具调整所述初始内存空间,其中,所述第二内存工具对内存空间的调整效率低于所述第一内存工具对内存空间的调整效率,但所述第二内存工具的运行负载小于所述第一内存工具的运行负载。
  21. 一种内存数据的迁移装置,其特征在于,包括:
    第一检测模块,被设置为检测目标内存空间的内存使用参数,其中,所述目标内存空间是为待迁移内存数据的目标应用所分配的内存空间,所述目标内存空间中包括属于第一类型的第一内存空间和属于第二类型的第二内存空间,所述内存使用参数用于指示所述目标内存空间被使用的频率分布,所述目标应用对所述第一内存空间的操作效率高于所述目标应用对所述第二内存空间的操作效率;
    第一筛选模块,被设置为根据所述内存使用参数和目标迁移参数从所述第二内存空间中筛选出第三内存空间,其中,所述目标迁移参数是根据所述目标内存空间的分配参数确定的,所述分配参数用于指示所述第一内存空间和所述第二内存空间在所述目标内存空间中的占比,所述目标迁移参数用于指示所述目标内存空间中允许迁移内存数据的内存空间的占比,所述第三内存空间是所述第二内存空间中被使用的频率高于第一频率阈值的内存空间;
    第一迁移模块,被设置为将所述第三内存空间中所存储的第一内存数据迁移至所述第一内存空间。
  22. 一种计算机非易失性可读存储介质,其特征在于,所述计算机非易失性可读存储介质中存储有计算机程序,其中,所述计算机程序被处理器执行时实现所述权利要求1至20任一项中所述的方法的步骤。
  23. 一种电子设备,包括存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现所述权利要求1至20任一项中所述的方法的步骤。
PCT/CN2024/070674 2023-06-21 2024-01-04 内存数据的迁移方法、装置、非易失性可读存储介质及电子装置 Ceased WO2024259951A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US19/137,733 US20260111137A1 (en) 2023-06-21 2024-01-04 Method and Apparatus of Migrating Memory Data, and Non-Volatile Readable Storage Medium and Electronic Apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310743575.1A CN116483740B (zh) 2023-06-21 2023-06-21 内存数据的迁移方法、装置、存储介质及电子装置
CN202310743575.1 2023-06-21

Publications (1)

Publication Number Publication Date
WO2024259951A1 true WO2024259951A1 (zh) 2024-12-26

Family

ID=87218129

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/070674 Ceased WO2024259951A1 (zh) 2023-06-21 2024-01-04 内存数据的迁移方法、装置、非易失性可读存储介质及电子装置

Country Status (3)

Country Link
US (1) US20260111137A1 (zh)
CN (1) CN116483740B (zh)
WO (1) WO2024259951A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116483740B (zh) * 2023-06-21 2023-09-05 苏州浪潮智能科技有限公司 内存数据的迁移方法、装置、存储介质及电子装置
CN120849376A (zh) * 2024-04-28 2025-10-28 云智能资产控股(新加坡)私人股份有限公司 数据迁移方法、系统及装置
CN118132274B (zh) * 2024-04-30 2024-08-16 苏州元脑智能科技有限公司 内存分配方法、计算机设备、存储介质及产品

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107203477A (zh) * 2017-06-16 2017-09-26 深圳市万普拉斯科技有限公司 内存分配方法、装置、电子设备及可读存储介质
CN110673785A (zh) * 2019-08-23 2020-01-10 上海科技发展有限公司 一种数据存储方法及系统
US20210132944A1 (en) * 2019-11-04 2021-05-06 International Business Machines Corporation Optimization of low-level memory operations in a numa environment
CN112783656A (zh) * 2021-01-29 2021-05-11 网易(杭州)网络有限公司 内存管理方法、介质、装置和计算设备
CN113050874A (zh) * 2019-12-26 2021-06-29 华为技术有限公司 一种内存设置方法以及装置
CN115421924A (zh) * 2022-09-29 2022-12-02 阿里巴巴(中国)有限公司 一种内存分配方法、装置及设备
CN115904212A (zh) * 2021-09-30 2023-04-04 华为技术有限公司 数据处理的方法、装置、处理器和混合内存系统
CN116483740A (zh) * 2023-06-21 2023-07-25 苏州浪潮智能科技有限公司 内存数据的迁移方法、装置、存储介质及电子装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111930305A (zh) * 2020-07-24 2020-11-13 北京金山云网络技术有限公司 数据的存储方法和装置、存储介质、电子装置
CN114168490B (zh) * 2020-09-10 2025-09-12 华为技术有限公司 确定内存回收阈值的方法及相关设备
CN114442910B (zh) * 2020-11-06 2024-08-16 伊姆西Ip控股有限责任公司 管理存储系统的方法、电子设备和计算机可读介质
CN115469813A (zh) * 2022-10-18 2022-12-13 上海芯赛云计算科技有限公司 数据处理方法及装置、存储介质、电子装置
CN115599304B (zh) * 2022-10-26 2025-07-22 建信金融科技有限责任公司 应用于存储节点的数据处理方法、装置、设备及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107203477A (zh) * 2017-06-16 2017-09-26 深圳市万普拉斯科技有限公司 内存分配方法、装置、电子设备及可读存储介质
CN110673785A (zh) * 2019-08-23 2020-01-10 上海科技发展有限公司 一种数据存储方法及系统
US20210132944A1 (en) * 2019-11-04 2021-05-06 International Business Machines Corporation Optimization of low-level memory operations in a numa environment
CN113050874A (zh) * 2019-12-26 2021-06-29 华为技术有限公司 一种内存设置方法以及装置
CN112783656A (zh) * 2021-01-29 2021-05-11 网易(杭州)网络有限公司 内存管理方法、介质、装置和计算设备
CN115904212A (zh) * 2021-09-30 2023-04-04 华为技术有限公司 数据处理的方法、装置、处理器和混合内存系统
CN115421924A (zh) * 2022-09-29 2022-12-02 阿里巴巴(中国)有限公司 一种内存分配方法、装置及设备
CN116483740A (zh) * 2023-06-21 2023-07-25 苏州浪潮智能科技有限公司 内存数据的迁移方法、装置、存储介质及电子装置

Also Published As

Publication number Publication date
US20260111137A1 (en) 2026-04-23
CN116483740B (zh) 2023-09-05
CN116483740A (zh) 2023-07-25

Similar Documents

Publication Publication Date Title
WO2024259951A1 (zh) 内存数据的迁移方法、装置、非易失性可读存储介质及电子装置
JP7467593B2 (ja) リソース割振り方法、記憶デバイス、および記憶システム
US20250077075A1 (en) Memory device and host device
CN105190567B (zh) 用于管理存储系统快照的系统和方法
US20170017524A1 (en) Quality of service implementation in a networked storage system with hierarchical schedulers
JP2019008729A (ja) メモリシステムおよび制御方法
CN102428452A (zh) 分割快闪存储器数据存储装置
CN103970680A (zh) 内存管理方法、装置及嵌入式系统
CN110869916B (zh) 用于两层写时复制的方法和装置
WO2025112885A1 (zh) 资源调整方法、装置、电子设备、存储介质及训练平台
CN111104219A (zh) 虚拟核心与物理核心的绑定方法、装置、设备及存储介质
CN116521608B (zh) 数据迁移方法及计算设备
CN103631537B (zh) 一种虚拟磁盘的管理方法及装置
WO2025227946A1 (zh) 虚拟资源分配方法、装置、计算机设备、计算机可读存储介质及计算机程序产品
CN115904689A (zh) 控制内存带宽的方法、装置、处理器及计算设备
US20250199950A1 (en) Internal memory reclaiming method and apparatus, device, medium, and product
CN119576472A (zh) 操作电子系统的方法
WO2025066043A1 (zh) 一种数据访问方法、cxl存储设备及cxl控制器
US11989420B2 (en) Memory allocation method and apparatus, electronic device, and storage medium
WO2024152714A1 (zh) 内存回收方法、计算机设备、介质和程序产品
CN112306927A (zh) 一种io请求的处理方法、装置及系统
CN116627339A (zh) 一种基于固态硬盘的vf动态调度方法、装置及介质
CN115794305A (zh) 虚拟机内存的调整方法和装置、存储介质及电子装置
CN112199044A (zh) 面向多租户的ftl设置方法、系统、计算机程序及存储介质
CN118732960B (zh) 数据的写入方法和装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24824855

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE