CN104102460A - Cloud computing-based memory management method and device - Google Patents

Cloud computing-based memory management method and device Download PDF

Info

Publication number
CN104102460A
CN104102460A CN201410352763.2A CN201410352763A CN104102460A CN 104102460 A CN104102460 A CN 104102460A CN 201410352763 A CN201410352763 A CN 201410352763A CN 104102460 A CN104102460 A CN 104102460A
Authority
CN
China
Prior art keywords
address space
long
free memory
range free
logical extension
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201410352763.2A
Other languages
Chinese (zh)
Inventor
侯德龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Beijing Electronic Information Industry Co Ltd
Original Assignee
Inspur Beijing Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Beijing Electronic Information Industry Co Ltd filed Critical Inspur Beijing Electronic Information Industry Co Ltd
Priority to CN201410352763.2A priority Critical patent/CN104102460A/en
Publication of CN104102460A publication Critical patent/CN104102460A/en
Pending legal-status Critical Current

Links

Landscapes

  • Storage Device Security (AREA)

Abstract

The invention provides a cloud computing-based memory management method and device which is applied to a cloud computing platform. The method includes adopting logical extension address space to extend local physical address space; adopting global extension address space to record a remote free memory which is available locally; and according to predetermined rules, subjecting the global extension address space and the logical extension address space to bidirectional mapping. By the cloud computing-based memory management method and device, utilization of memory resources in the cloud computing is optimized.

Description

A kind of EMS memory management process and device based on cloud computing
Technical field
The present invention relates to computer realm, be specifically related to plant EMS memory management process and the device based on cloud computing.
Background technology
The rise of cloud computing is little by little changing whole computer industry circle and academia, this using computing machine as a kind of usefulness (utility) instrument, using software as a kind of serve come computed mode be that computer software and hardware design have all been brought huge challenge.On the structure and ways of distribution of software, SOA (service-oriented software configuration) and SaaS (software serve) become software building under cloud computing mode and the core mode of distribution just more and more.And the bottom computing basic facility moving for supporting, challenge mainly concentrates on two aspects: system software and hardware.Wherein, adaptivity resource distribution is the ultimate challenge that system software faces, and how Resources allocation is again the most crucial problem of this challenge.Carry out static allocation resource according to peak load, can bring the serious waste of resource.According to statistics, in current data center, the utilization factor of cpu resource is on average between 5%~20%, and the utilization factor of memory source is also no more than 60%.Clearly, current resource allocation mode has very high wasting of resources degree, has increased user's the total cost of ownership (TCO).Carry out static allocation resource taking average load as benchmark and can make service quality in the situation that of peak load decline, and finally affect cloud user's potential profit.
Summary of the invention
The technical problem to be solved in the present invention is how to optimize the utilization of memory source in cloud computing.
In order to address the above problem, the invention provides a kind of EMS memory management process based on cloud computing, be applied in cloud computing platform, comprising:
Adopt logical extension address space to expand local physical address space;
Adopt overall expanded address space record by the local long-range free memory using;
Carry out the two-way mapping between described overall expanded address space and described logical extension address space according to pre-defined rule.
Alternatively, described overall expanded address space comprises remote server mark and the address field of long-range free memory in this remote server at long-range free memory place.
Alternatively, while carrying out the two-way mapping between overall expanded address space and logical extension address space according to pre-defined rule, be that the long-range free memory of multiple remote servers is mapped to a logical extension address space.
Alternatively, described method also comprises:
In the time that long-range free memory exits, data in this long-range free memory are copied in the long-range free memory that other remote server provides page by page, the long-range free memory that this other remote server is provided adds in described overall expanded address space, and is mapped on original same section of the shone upon logical extension address space of exited long-range free memory.
Alternatively, described method also comprises:
The order that described long-range free memory is read and write is sent by Xen by the shared buffer between Xen and Domain0, is processed by the Sw ap finger daemon in Domain0.
A memory management device based on cloud computing, is applied in cloud computing platform, comprising:
Monitor of virtual machine, for adopting logical extension address space to expand local physical address space;
Control desk, for adopting overall expanded address space record by the local long-range free memory using;
Address mapping module, for carrying out the two-way mapping between described overall expanded address space and described logical extension address space according to pre-defined rule.
Alternatively, described overall expanded address space comprises remote server mark and the address field of long-range free memory in this remote server at long-range free memory place.
Alternatively, when described address mapping module is carried out the two-way mapping between overall expanded address space and logical extension address space according to pre-defined rule, be that the long-range free memory of multiple remote servers is mapped to a logical extension address space.
Alternatively, described address mapping module is also in the time that long-range free memory exits, data in this long-range free memory are copied in the long-range free memory that other remote server provides page by page, the long-range free memory that this other remote server is provided adds in described overall expanded address space, and is mapped on original same section of the shone upon logical extension address space of exited long-range free memory.
The present invention is directed to the internal memory optimization needs under virtual cloud computing platform, built a kind of mechanism of utilizing the seamless polymerization freememory of Intel Virtualization Technology resource; On virtual cloud computing infrastructure, introduce the double-deck Address space mappinD mechanism of a kind of logic-based expanded address space and overall expanded address space; Utilize this mechanism, reconstruct memory layers aggregated(particle) structure under virtual cloud platform, the grow area that has built the high reliability based on redundancy backup mechanism carrys out memory optimization utilization; By EVM(extended virtual machine) watch-dog, realize a kind ofly to transparent idle (and utilization factor the is not high) page reclaim mechanism of virtual machine, this mechanism is carried out free memory recovery situation for being not suitable for retouching operation system code has special significance.The present invention, by crossing over the addressing technique of physical platform, realizes the access for long-range free memory resource, the internal memory of poor efficiency can be allocated to the virtual machine use high to memory usage, thereby has optimized the utilization of whole memory source.
Brief description of the drawings
Fig. 1 is the system schematic that has used the method for embodiment mono-;
Fig. 2 is mapping schematic diagram;
Fig. 3 is inner server page status transition diagram;
Fig. 4 is the Xen system schematic of expansion;
Fig. 5 is the schematic diagram of the device of embodiment bis-.
Embodiment
Below in conjunction with drawings and Examples, technical scheme of the present invention is described in detail.
It should be noted that, if do not conflicted, each feature in the embodiment of the present invention and embodiment can mutually combine, all within protection scope of the present invention.In addition, although there is shown logical order in flow process, in some cases, can carry out shown or described step with the order being different from herein.
Embodiment mono-, a kind of EMS memory management process based on cloud computing, comprising:
Adopt logical extension address space to expand local physical address space;
Adopt overall expanded address space record by the local long-range free memory using;
Carry out the two-way mapping between described overall expanded address space and described logical extension address space according to pre-defined rule.
In order to utilize the free memory resource being distributed on each physical server of whole platform, consider the characteristic of distributed platform, the present embodiment has been introduced a distributed memory pond being made up of double-deck Address space mappinD.The structure of double-deck address space has been taken into account efficiency and reliability, provides maximum memory source abstract.As shown in Figure 1, each node in cloud computing platform (node 1, node 2 ..., node n) in, the internal memory (such as the internal memory of VM#1 in node in Fig. 11 and node 2) of the inner poor efficiency of virtual machine (VM) focuses on idling-resource pond, then offers virtual machine (such as the internal memory of VM#k in node in Fig. 12 and the node n) use at high usage internal memory place.
In an embodiment of the present embodiment, described overall expanded address space comprises remote server mark and the address field of long-range free memory in this remote server at long-range free memory place.
In present embodiment, described overall expanded address space can adopt < server ID, and the address that address field > indicates long-range free memory forms; Be used for the overall expanded address space of the multiple long-range free memories that record same remote server, remote server mark is identical, and address field is different.
In present embodiment, can change and dynamically adjust overall expanded address space according to free memory.After the free memory of remote server adopts overall expanded address space to indicate, then be mapped to logical extension address space.
In an embodiment of the present embodiment, while carrying out the two-way mapping between overall expanded address space and logical extension address space according to pre-defined rule, be that the long-range free memory of multiple remote servers is mapped to a logical extension address space; Can realize like this redundant storage of data, improve the reliability of long-distance inner.
Present embodiment as shown in Figure 2, in logical extension address space, can shine upon mutually from the long-range free memory in different server, different internal memory respectively by overall expanded address space by same sector address space.
The design of long-distance inner system need to be considered the requirement of two aspects: (1) shortens memory access latency as far as possible, raises the efficiency; (2) integrity problem of distributed system, provides the reliable and stable stores service based on internal memory.Present embodiment has taken into full account the requirement of these two aspects, reaches the object of high efficient and reliable by overall expanded address space to the many-one mapping mechanism of logical extension address space.
In an embodiment of the present embodiment, adopt address map component that the two-way mapping between overall expanded address space and logical extension address space is provided.
In present embodiment, by safeguarding that relevant mapping relations provide the long-distance inner storage of high efficient and reliable abstract.Address map component is supported the data manipulation that two classes are basic: data access and data are synchronous.Wherein, data access is basic long-distance inner operation, realizes reading and writing for teledata.Data synchronous operation is that exiting of long-range free memory provides support.
In an embodiment of the present embodiment, described method can also comprise:
In the time that long-range free memory exits, data in this long-range free memory are copied in the long-range free memory that other remote server provides page by page, the long-range free memory that this other remote server is provided adds in described overall expanded address space, and is mapped on original same section of the shone upon logical extension address space of exited long-range free memory.
In present embodiment, when a remote server, because self load increase need to regain shared internal memory or remote server and cannot normally provide memory service time, system log-on data synchronous operation actively or passively carrys out migration data and backups on other remote server.Adopt method of data synchronization from the server migration data that exit free memory to other remote server, can realize the high reliability of data storage, ensure the validity of data.
The state of inner server page is changed as shown in Figure 3, and the page of no initializtion is carried out after remote write or synchronous write, and page status becomes dirty; The page of no initializtion is synchronously read or dirty page is carried out after long-range reading, and page status becomes the free time; The page of idle condition is carried out after synchronous write to state still for idle, carry out becoming after remote write dirty; Dirty page is synchronously read to rear state still for dirty.
In an embodiment of the present embodiment, described long-range free memory is except comprising that the internal memory that is not assigned with or virtual machine are initiatively shared internal memory, can also comprise that virtual machine inside is because the existing part of restriction of data access patterns has been distributed use, but the extremely low memory headroom of access frequency.
In order farthest to increase the sharing degree of internal memory, present embodiment has been introduced a kind of to the transparent free page recovery technology of virtual machine.In order to realize the transparent recovery for the page, present embodiment expansion Xen (a kind of open source code virtual machine monitor) monitor of virtual machine is to realize dynamic " stealing " runtime system based on memory pages visit information.
The Xen monitor of virtual machine of expansion as shown in Figure 4, completes two Core Features: (1) has introduced logical extension address space to support long-distance inner access; (2) provide transparent addressing support to the page at local internal memory not.
In order to support logical extension address space, present embodiment has been revised the mapping from false physical address to machine address (p2m mechanism) part in Xen monitor of virtual machine, has increased new address style and has indicated logical extension address space.
In the time of virtual machine access logic expanded address space, or can trigger page fault one time, or trigger logical extension address space addressing mechanism.While processing with instruction operation rollback operation by page fault, virtual machine is suspended carries out to wait for that the page is loaded into local internal memory again; In the time that the page is reloaded into local internal memory, p2m is reset to point to the memory pages newly packing into.If half virtual situation, the treatment scheme of page fault can continue.And when fully virtualized, the abnormal mechanism of replaying will play a role to reschedule virtual machine and carry out, recover abnormal processing procedure.
Instruction operation rollback runtime system intercepts the access to logical extension address space in instruction simulation implementation, trigger the abnormal also up layer functions transmission step by step of logical extension address space access, until actuating logic arrival virtual machine is trapped in the entrance of Xen.There, virtual machine is suspended, and the operational order of logical extension address space is passed to Domain0 by shared memory space.After data are returned, instruction simulation operation or abnormality processing operation are continued to carry out, and virtual machine is scheduled again.
In present embodiment, the order that long-range free memory is read and write is sent by Xen by the shared buffer between Xen and Domain0, and the Sw ap finger daemon in Domain0 is responsible for concrete processing.The reason of doing is like this that Xen itself lacks network protocol stack, and such design can improve system resilience largely.
In Domain0, present embodiment has realized flexible overall expanded address space administrative mechanism.Wherein, every section of continuous address space (address field) represents by an address space descriptor and spatial dimension, and each remote server can comprise one or more address spaces.By shining upon multiple address spaces to same section of logical extension address space, can realize the redundant storage of data.By the form of module, present embodiment is supported the exterior storage of two kinds of forms: long-range free memory and local internal memory.Local internal memory and long-distance inner stack mapping, can obtain there is high reliability, high performance extra storage system.
Present embodiment can by the mode of mmap realize based on share page and Xen between information communication, Xen asks to Domain0 by shared message channel transmission.In Domain0, finger daemon KXenSw ap kernel thread is created the control command of sending from Xen with corresponding, as the read-write operation of long-distance inner.When KXenSw ap has completed after related command by driving, the called notice of a new Hypercall Xen completes the follow-up process such as the amendment of p2m table, page table renewal to recover the scheduled for executing of virtual machine.
Embodiment bis-, a kind of memory management device based on cloud computing, be applied in cloud computing platform, as shown in Figure 5, comprising:
Monitor of virtual machine, for adopting logical extension address space to expand local physical address space;
Control desk, for adopting overall expanded address space record by the local long-range free memory using;
Address mapping module, for carrying out the two-way mapping between described overall expanded address space and described logical extension address space according to pre-defined rule.
In the present embodiment, described address mapping module also can be used as address map component and is integrated in (as shown in Figure 5) in described control desk; The memory headroom of virtual machine, except deriving from local internal memory, also derives from the long-range free memory that logical extension address space shines upon by overall expanded address space; Local internal memory can also provide memory headroom for control desk.
In an embodiment of the present embodiment, described overall expanded address space can comprise remote server mark and the address field of long-range free memory in this remote server at long-range free memory place.
In an embodiment of the present embodiment, when described address mapping module is carried out the two-way mapping between overall expanded address space and logical extension address space according to pre-defined rule, be that the long-range free memory of multiple remote servers is mapped to a logical extension address space.
In an embodiment of the present embodiment, described address mapping module is also in the time that long-range free memory exits, data in this long-range free memory are copied in the long-range free memory that other remote server provides page by page, the long-range free memory that this other remote server is provided adds in described overall expanded address space, and is mapped on original same section of the shone upon logical extension address space of exited long-range free memory.
One of ordinary skill in the art will appreciate that all or part of step in said method can carry out instruction related hardware by program and complete, described program can be stored in computer-readable recording medium, as ROM (read-only memory), disk or CD etc.Alternatively, all or part of step of above-described embodiment also can realize with one or more integrated circuit.Correspondingly, the each module/unit in above-described embodiment can adopt the form of hardware to realize, and also can adopt the form of software function module to realize.The present invention is not restricted to the combination of the hardware and software of any particular form.
Certainly; the present invention also can have other various embodiments; in the situation that not deviating from spirit of the present invention and essence thereof; those of ordinary skill in the art are when making according to the present invention various corresponding changes and distortion, but these corresponding changes and distortion all should belong to the protection domain of claim of the present invention.

Claims (9)

1. the EMS memory management process based on cloud computing, is applied in cloud computing platform, comprising:
Adopt logical extension address space to expand local physical address space;
Adopt overall expanded address space record by the local long-range free memory using;
Carry out the two-way mapping between described overall expanded address space and described logical extension address space according to pre-defined rule.
2. the method for claim 1, is characterized in that:
Described overall expanded address space comprises remote server mark and the address field of long-range free memory in this remote server at long-range free memory place.
3. the method for claim 1, is characterized in that:
While carrying out the two-way mapping between overall expanded address space and logical extension address space according to pre-defined rule, be that the long-range free memory of multiple remote servers is mapped to a logical extension address space.
4. the method for claim 1, is characterized in that, also comprises:
In the time that long-range free memory exits, data in this long-range free memory are copied in the long-range free memory that other remote server provides page by page, the long-range free memory that this other remote server is provided adds in described overall expanded address space, and is mapped on original same section of the shone upon logical extension address space of exited long-range free memory.
5. the method for claim 1, is characterized in that, also comprises:
The order that described long-range free memory is read and write is sent by Xen by the shared buffer between Xen and Domain0, is processed by the Sw ap finger daemon in Domain0.
6. the memory management device based on cloud computing, is applied in cloud computing platform, it is characterized in that, comprising:
Monitor of virtual machine, for adopting logical extension address space to expand local physical address space;
Control desk, for adopting overall expanded address space record by the local long-range free memory using;
Address mapping module, for carrying out the two-way mapping between described overall expanded address space and described logical extension address space according to pre-defined rule.
7. device as claimed in claim 6, is characterized in that:
Described overall expanded address space comprises remote server mark and the address field of long-range free memory in this remote server at long-range free memory place.
8. device as claimed in claim 6, is characterized in that:
When described address mapping module is carried out the two-way mapping between overall expanded address space and logical extension address space according to pre-defined rule, be that the long-range free memory of multiple remote servers is mapped to a logical extension address space.
9. device as claimed in claim 6, is characterized in that:
Described address mapping module is also in the time that long-range free memory exits, data in this long-range free memory are copied in the long-range free memory that other remote server provides page by page, the long-range free memory that this other remote server is provided adds in described overall expanded address space, and is mapped on original same section of the shone upon logical extension address space of exited long-range free memory.
CN201410352763.2A 2014-07-23 2014-07-23 Cloud computing-based memory management method and device Pending CN104102460A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410352763.2A CN104102460A (en) 2014-07-23 2014-07-23 Cloud computing-based memory management method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410352763.2A CN104102460A (en) 2014-07-23 2014-07-23 Cloud computing-based memory management method and device

Publications (1)

Publication Number Publication Date
CN104102460A true CN104102460A (en) 2014-10-15

Family

ID=51670642

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410352763.2A Pending CN104102460A (en) 2014-07-23 2014-07-23 Cloud computing-based memory management method and device

Country Status (1)

Country Link
CN (1) CN104102460A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104536905A (en) * 2015-01-14 2015-04-22 浪潮(北京)电子信息产业有限公司 Memory management method and system
CN104657198A (en) * 2015-01-24 2015-05-27 深圳职业技术学院 Memory access optimization method and memory access optimization system for NUMA (Non-Uniform Memory Access) architecture system in virtual machine environment
CN105278880A (en) * 2015-10-19 2016-01-27 浪潮电子信息产业股份有限公司 Cloud computing virtualization-based memory optimization device and method
CN105988871A (en) * 2015-01-27 2016-10-05 华为技术有限公司 Remote memory allocation method, device and system
CN106445691A (en) * 2016-11-25 2017-02-22 郑州云海信息技术有限公司 Memory optimization method oriented to virtual cloud computing platform
CN107786599A (en) * 2016-08-26 2018-03-09 北京计算机技术及应用研究所 Internal memory cloud system
WO2021218038A1 (en) * 2020-04-28 2021-11-04 华为技术有限公司 Storage system, memory management method, and management node

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104536905A (en) * 2015-01-14 2015-04-22 浪潮(北京)电子信息产业有限公司 Memory management method and system
CN104536905B (en) * 2015-01-14 2018-02-02 浪潮(北京)电子信息产业有限公司 A kind of EMS memory management process and system
CN104657198A (en) * 2015-01-24 2015-05-27 深圳职业技术学院 Memory access optimization method and memory access optimization system for NUMA (Non-Uniform Memory Access) architecture system in virtual machine environment
CN104657198B (en) * 2015-01-24 2017-12-26 深圳职业技术学院 NUMA architecture system memory access optimization method in a virtual machine environment and system
CN105988871A (en) * 2015-01-27 2016-10-05 华为技术有限公司 Remote memory allocation method, device and system
CN105278880A (en) * 2015-10-19 2016-01-27 浪潮电子信息产业股份有限公司 Cloud computing virtualization-based memory optimization device and method
CN107786599A (en) * 2016-08-26 2018-03-09 北京计算机技术及应用研究所 Internal memory cloud system
CN107786599B (en) * 2016-08-26 2020-07-17 北京计算机技术及应用研究所 Memory cloud system
CN106445691A (en) * 2016-11-25 2017-02-22 郑州云海信息技术有限公司 Memory optimization method oriented to virtual cloud computing platform
WO2021218038A1 (en) * 2020-04-28 2021-11-04 华为技术有限公司 Storage system, memory management method, and management node
US11861204B2 (en) 2020-04-28 2024-01-02 Huawei Technologies Co., Ltd. Storage system, memory management method, and management node

Similar Documents

Publication Publication Date Title
CN104102460A (en) Cloud computing-based memory management method and device
US10909072B2 (en) Key value store snapshot in a distributed memory object architecture
Kannan et al. Optimizing checkpoints using nvm as virtual memory
US11243706B2 (en) Fragment management method and fragment management apparatus
AU2015211305B2 (en) Prioritizing data reconstruction in distributed storage systems
CN102467408B (en) Method and device for accessing data of virtual machine
CN103597451B (en) Memory image and redundancy generation for high availability
CN108292235B (en) Network attached storage using selective resource migration
CN104111897A (en) Data processing method, data processing device and computer system
US20140181042A1 (en) Information processor, distributed database system, and backup method
US9448934B2 (en) Affinity group access to global data
CN105051695A (en) Immutable shareable zero-copy data and streaming
CN108694076A (en) Multinode storage operation
CN105103136A (en) Shared and managed memory unified access
US8943291B2 (en) Reorganization of software images based on predicted use thereof
CN113569508B (en) Database model construction method and device for data indexing and access based on ID
CN111727425A (en) Trace logging by logging the inflow to an upper level shared cache and cache coherence protocol translation between lower level caches
CN105468296A (en) No-sharing storage management method based on virtualization platform
CN102597972B (en) virtual computer system, area management method
CN106445691A (en) Memory optimization method oriented to virtual cloud computing platform
CN105378673A (en) Zero-copy caching
CN107577492A (en) The NVM block device drives method and system of accelerating file system read-write
Bayati et al. Exploring benefits of nvme ssds for bigdata processing in enterprise data centers
CN105278880A (en) Cloud computing virtualization-based memory optimization device and method
CN104050093A (en) Fault buffer for resolving page faults in unified virtual memory system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20141015