WO2022120992A1 - Virtual-environment-based memory sharing system and method - Google Patents

Virtual-environment-based memory sharing system and method Download PDF

Info

Publication number
WO2022120992A1
WO2022120992A1 PCT/CN2020/139462 CN2020139462W WO2022120992A1 WO 2022120992 A1 WO2022120992 A1 WO 2022120992A1 CN 2020139462 W CN2020139462 W CN 2020139462W WO 2022120992 A1 WO2022120992 A1 WO 2022120992A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
coordinator
module
access
data
Prior art date
Application number
PCT/CN2020/139462
Other languages
French (fr)
Chinese (zh)
Inventor
李永康
王洋
须成忠
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2022120992A1 publication Critical patent/WO2022120992A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/544Buffers; Shared memory; Pipes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45583Memory management, e.g. access or allocation
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the invention relates to the technical field of memory sharing, in particular to a memory sharing system and method based on a virtual environment.
  • the remote direct memory access (RDMA module) technology offloads the processing of the protocol stack to the network card, and the network card directly uses the DMA technology to access the memory data after parsing the data packet, avoiding multiple copies of the data.
  • the CPU is freed from the network data transmission, which improves the efficiency of CPU usage.
  • GAM implements a memory sharing model on a physically distributed cluster and can manage free memory distributed among multiple nodes to provide a unified memory model.
  • GAM provides a set of user-friendly APIs for memory operations.
  • GAM implements the cache of data on each host, and deeply thinks about cache coherence, distinguishes the different states of memory pages when they are shared, and designs data when reading and writing data. And consistency handling when data fails.
  • GAM also adopts the method of mixing communication primitives in the use of the RDMA module, and only uses the unilateral primitive WRITE in data transmission.
  • the present invention provides a memory sharing system and method based on a virtual environment, so as to solve the problem that the existing memory sharing method cannot efficiently realize memory sharing on a virtual cluster.
  • the present invention provides a memory sharing system based on a virtual environment, which includes: at least two hosts, each host includes at least one virtual machine, a coordinator and a memory module, a virtual machine, a coordinator,
  • the memory modules are electrically connected to each other, and the coordinator of the local host and the coordinator of the remote host are connected by remote communication through the RDMA module;
  • the virtual machine is used to receive the access request initiated by the application and send the access request to the coordinator;
  • the coordinator queries the memory Whether the memory data requested by the access request exists in the module; if so, return the memory data to the application; if not, the coordinator sends an access request to the coordinator of the remote host that stores the memory data through the RDMA module.
  • the memory module includes shared memory, cache, and address space allocated by the operating system and application programs.
  • the cache is used to cache the memory data of the remote host. After the coordinator receives the access request, it queries whether there is memory data in the cache. .
  • an API interface, a library function and a virtual PCI device are provided in the virtual machine, and the application program calls the library function through the API interface to operate the virtual PCI device to access the memory module.
  • the coordinator includes a communication module, a monitoring module and a message processing module;
  • the communication module is used to implement communication between different virtual machines on the same host or between different hosts;
  • the monitoring module is used to monitor the activities of the virtual machines, And by analyzing the access log of the memory module to distinguish the access type of each memory page, and configure the corresponding cache policy for the memory pages of different access types according to the preset policy rules;
  • the message processing module is used for the message received by the communication module. Parse processing and distribute pre-configured message handlers for different types of messages.
  • the communication modules between different hosts implement the data transmission process through the SEND/RECV bilateral primitive, the WRITE unilateral primitive or the WRITE_WITH_IMM primitive of the RDMA module.
  • the monitoring module analyzes the access log of the memory module to distinguish the access type of each memory page, and configures corresponding cache policies for memory pages of different access types according to preset policy rules, including; When the number of accesses by the target remote host reaches the first preset condition, the access type of the memory page is divided into exclusive access, and the memory page is migrated to the memory module of the target remote host; when the record of read data on the memory page reaches the second In the preset condition, the access type of the memory page is divided into read sharing, and the memory page is stored on the remote host with the highest frequency of accessing the memory page, and the permission is set to read-only; when the record of written data on the memory page reaches In the third preset condition, the access type of the memory page is divided into write sharing, and synchronization of the memory page to other remote hosts is prohibited.
  • the monitoring module generates an access log when monitoring that the remote host accesses the memory module of the local host, and the access log records the access host, the access address and the access type.
  • the communication between the virtual machine and the coordinator is implemented by way of process communication.
  • the present invention also provides a virtual environment-based memory sharing method, which is applied to one of the above-mentioned virtual environment-based memory sharing systems.
  • the method includes: when the virtual machine receives an access request initiated by an application program.
  • the coordinator sends the access request to the coordinator; the coordinator analyzes the access request and determines whether the access address of the access request is the local host or the remote host; if it is the local host, it obtains the data requested by the access request from the local host and feeds it back to the application ; If it is a remote host, judge whether the data requested by the access request is cached in the memory module; when the data requested by the access request is cached in the memory module, the data requested by the cached access request is fed back to the application; When the data requested by the access request does not exist in the module, the coordinator sends an access request to the coordinator of the remote host through the RDMA module, and feeds back the data obtained by the access to the application program.
  • the memory sharing system based on the virtual environment of the present invention sets up a coordinator in the host, and realizes data communication between different virtual machines in the host through the coordinator to realize memory sharing, and aims at different hosts. Between them, it uses the RDMA module to realize the communication between the coordinators of different hosts, so as to realize the communication between different hosts, and finally realize the memory sharing of the entire virtual cluster, and the use of the RDMA module effectively reduces the data interaction between different hosts. Delay.
  • FIG. 1 is a schematic structural diagram of a memory sharing system based on a virtual environment according to an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of a memory module of a memory sharing system based on a virtual environment according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a virtual machine of a memory sharing system based on a virtual environment according to an embodiment of the present invention
  • FIG. 4 is a schematic structural diagram of a coordinator of a virtual environment-based memory sharing system according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of an operation flow that needs to be performed when a cached memory page of a memory sharing system based on a virtual environment is modified according to an embodiment of the present invention
  • FIG. 6 is a schematic flowchart of a memory sharing method based on a virtual environment according to an embodiment of the present invention.
  • first”, “second” and “third” in the present invention are only used for description purposes, and should not be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature defined as “first”, “second”, “third” may expressly or implicitly include at least one of that feature.
  • "a plurality of” means at least two, such as two, three, etc., unless otherwise expressly and specifically defined. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship between various components under a certain posture (as shown in the accompanying drawings).
  • FIG. 1 is a schematic structural diagram of a memory sharing system based on a virtual environment according to an embodiment of the present invention.
  • the virtual environment-based memory sharing system 100 includes: at least two hosts 1, each host 1 includes at least one virtual machine 2, a coordinator 3 and a memory module 4, the virtual machine 2, the coordinator 3 , the memory modules 4 are electrically connected to each other, and the coordinator 3 of the local host and the coordinator 3 of the remote host are connected by remote communication through the RDMA module 5; the virtual machine 2 is used to receive the access request initiated by the application program, and send the access request to the coordinator.
  • Coordinator 3 inquires whether there is memory data requested by the access request in memory module 4; if so, returns the memory data to the application; The coordinator 3 of the remote host issues an access request.
  • the virtual environment-based memory sharing system 100 is developed based on KVM/QEMU, wherein the virtual machine 2 runs on KVM.
  • the last simulated virtual machine of QEMU is used to simulate the shared memory on the cluster.
  • Multiple virtual machines 2 can be built on the same host 1. It should be noted that when an application initiates an access request to the virtual machine of host A, host A is the local host, and when the memory data accessed by the application is in the memory module of host B, then host B is the remote host.
  • the virtual machine 2 when the virtual machine 2 receives the access request initiated by the application, it sends the access request to the coordinator 3; the coordinator 3 analyzes the access request, and judges whether the access address of the access request is the local host or the remote host; if it is the local host, Then obtain the memory data requested by the access request from the local host and feed it back to the application; if it is a remote host, then determine whether the memory data requested by the access request is cached in the memory module 4; When the memory data requested, the memory data requested by the cached access request is fed back to the application program; when the memory data requested by the access request does not exist in the memory module 4, the coordinator 3 sends the data to the coordinator of the remote host through the RDMA module 5 3 Send an access request, and feed back the memory data obtained by the access to the application.
  • the memory sharing system based on the virtual environment of this embodiment sets the coordinator 3 in the host, and realizes the data communication between different virtual machines 2 in the host 1 through the coordinator 3 to realize memory sharing. During the time, it realizes the communication between the coordinators 3 of different hosts 1 by using the RDMA module 5, so as to realize the communication between different hosts 1, and finally realize the memory sharing of the whole virtual cluster, and the RDMA module 5 effectively reduces the different Latency of data exchange between hosts 1.
  • the memory module 4 includes a shared memory 41, a cache 42 and an address space 43 allocated by the operating system and application programs.
  • the cache 42 is used to cache the memory data of the remote host. After the coordinator 3 receives the access request, , query whether the memory data exists in the cache 42 .
  • the size of the shared memory 41 of each host 1 is indicated by the command line during system initialization, and when the shared memory 41 is actually allocated, a fixed-size cache 42 needs to be allocated, and the cache 42 is used to cache remote Host memory data. It should be noted that the memory data in the cache is invisible to the virtual machine 2. Therefore, when the virtual machine sends an access request to the coordinator 3, the coordinator 3 queries the cache 42 whether there is the memory data to be queried. , if it exists, directly return the memory data to the application, if not, send the access request to the remote host.
  • the virtual machine 2 is provided with an API interface 21 , a library function 22 and a virtual PCI device 23 , and the application calls the library function 22 through the API interface 21 to operate the virtual PCI device 23 to access the memory module 4 .
  • the virtual PCI device 23 is established based on QEMU, and the number of registers and message processing functions are expanded.
  • the library function 22 includes a library function provided by QEMU and a preconfigured library function for the response message.
  • the coordinator 3 includes a communication module 31, a monitoring module 32 and a message processing module 33;
  • the communication module 31 is used to implement communication between different virtual machines 2 on the same host 1 or between different hosts;
  • monitoring The module 32 is used to monitor the activity of the virtual machine 2, and to distinguish the access type of each memory page by analyzing the access log of the memory module 4, and configure the corresponding cache policy for the memory pages of different access types according to preset policy rules;
  • message The processing module 33 is configured to parse and process the messages received by the communication module, and distribute pre-configured message processing functions for different types of messages.
  • the coordinator 2 is a loadable module responsible for tasks such as maintaining the virtual machine information of the host 1, communicating with the virtual machine 2, maintaining the shared memory on the host 1, and communicating with the remote host.
  • the communication module 31 is responsible for completing the communication between the virtual machines 2. Since the same host and different hosts have great differences in communication, in this embodiment, the communication is divided into two levels, one is the same host 1 The communication between different virtual machines 2 and the virtual machine 2 and the coordinator 3, and the second is the communication between different hosts 1. The communication between the virtual machine 2 and the coordinator 3 is implemented in a process manner, and does not involve data transmission on the network, so the transmission efficiency is higher.
  • the communication module 31 between different hosts 1 implements the data transmission process through the SEND/RECV bilateral primitive, the WRITE unilateral primitive or the WRITE_WITH_IMM primitive of the RDMA module 5 .
  • the communication module 31 transmits an access request or a control message of memory page failure
  • data transmission is realized through the SEND/RECV bilateral primitive; when a large amount of data is transmitted between the communication modules 31, the WRITE unilateral primitive is used.
  • the data transmission is realized by the WRITE_WITH_IMM primitive; when the response message is transmitted between the communication modules 31, the data transmission is realized through the WRITE_WITH_IMM primitive.
  • the memory page invalidation refers to the memory page of the remote host stored in the cache in the local host, and the data transmission change of the memory page of the remote host is that the memory page stored in the cache is invalidated.
  • connections between different hosts 1 are based on RDMA.
  • the idea of software-defined networking is used to separate the control path and the data path, and different data transmission sources are used on these two paths.
  • the transmission of the control path and the data path is divided into two connections, specifically:
  • the data they carry is usually small.
  • the SEND/RECV bilateral primitive in RDMA module 5 can be used. In this way, the participation of the CPU is required.
  • the receiver needs to use RECV to prepare for receiving data, and then the sender uses SEND to send data to ensure the reception. Fang received the message.
  • the host 1 needs to return a response packet after receiving the control message to determine the received message. For this type of message, it is usually only necessary to carry the data in the message header. Therefore, the WRITE_WITH_IMM primitive is used To transmit such data, the WRITE_WITH_IMM primitive does not carry payload during data transmission, reducing the resources consumed by data transmission.
  • the monitoring module 32 is responsible for monitoring the opening, closing and other activities of the virtual machine 2, and is responsible for analyzing the access log of the memory module 4, thereby distinguishing the access type of each memory page in the memory module 4, and according to the preset policy rules for different access types
  • the corresponding cache policy is configured for the memory page.
  • the monitoring module 32 generates an access log when monitoring that the remote host accesses the memory module 4 of the local host, and the access log records the contents of the access host, the access address, and the access type.
  • the message processing module 33 is responsible for analyzing and processing the messages received by the communication module 31. For different types of messages, the message types are confirmed according to the message structure, and then the messages are distributed to different message processing functions for processing. Among them, the type of the message can be divided according to the access object, such as the message of accessing the local host or the message of accessing the remote host, and can also broadcast the message of the newly created virtual machine 2 to the virtual cluster when the newly created virtual machine 2 is initialized, or It is a message or the like of the size of the shared memory of each host 1 .
  • the message processing function is preset.
  • the monitoring module 31 analyzes the access log of the memory module 4 to distinguish the access type of each memory page, and according to the preset policy rule is:
  • the cache policies corresponding to the memory page configurations of different access types are as follows:
  • the access type of the memory page is divided into exclusive access, and the memory page is migrated to the memory module 4 of the target remote host.
  • the first preset condition is preset.
  • the access type at this time is divided into exclusive access.
  • the memory page is migrated to the remote host, and the coordinator 3 of the remote host confirms the actual storage address in the memory module 4. If other remote hosts occasionally access the memory page, a notification is sent to the remote host. to confirm that the remote host can access the memory page.
  • the access type of the memory page is divided into read sharing, and the memory page is stored on the remote host with the highest frequency of accessing the memory page, and the permissions are set. is read-only.
  • the second preset condition is preset.
  • FIG. 5 shows the operation process that needs to be done when the cached memory page is modified.
  • node0 issues a write request
  • node1 is the node that retains the original memory page
  • node2 is the node that has cached data
  • node0 sends a write request
  • node1 will reply to the request to node0 and process data access.
  • node1 will send a notification of memory page invalidation to node2 and other nodes that have memory page cache, and node2, etc. will modify the flag of the cached page and reply the response packet.
  • node2 When subsequent applications need to access again, the data needs to be re-cached to the local host.
  • the third preset condition is preset.
  • the access type of the memory page is divided into write sharing, because the data of the memory page keeps changing when the write operation occurs.
  • the memory pages are cached on multiple remote hosts, considering the maintenance of data consistency, information needs to be exchanged frequently between multiple remote hosts, requiring a lot of synchronization operations, and even involving a lot of lock operations, which greatly reduces the system performance. performance, resulting in a significant increase in the delay of data transmission. Therefore, the shared memory page is not considered, and the application program still sends a request, and the coordinator 3 takes over the operation and accesses data to the remote host.
  • FIG. 6 shows the virtual environment-based memory sharing method of the present invention, which is applied to one of the above-mentioned virtual environment-based memory sharing systems, and the method includes:
  • Step S1 when the virtual machine receives the access request initiated by the application, it sends the access request to the coordinator;
  • Step S2 The coordinator analyzes the access request, and determines whether the access address of the access request is a local host or a remote host; if it is a local host, go to step S3; if it is a remote host, go to step S4.
  • Step S3 obtain the data requested by the access request from the local host and feed it back to the application;
  • Step S4 judging whether the data requested by the access request is cached in the memory module; when the data requested by the access request is cached in the memory module, execute step S5; when the data requested by the access request does not exist in the memory module, execute Step S6.
  • Step S5 feedback the data requested by the cached access request to the application
  • Step S6 The coordinator sends an access request to the coordinator of the remote host through the RDMA module, and feeds back the data obtained by the access to the application program.
  • a coordinator is set in the host, and the coordinator realizes the data communication of different virtual machines in the host to realize memory sharing, and for different hosts, the RDMA module is used to realize the communication between the coordinators of different hosts. Therefore, the communication between different hosts is realized, and the memory sharing of the entire virtual cluster is finally realized, and the RDMA module is used to effectively reduce the delay of data interaction between different hosts.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

Disclosed are a virtual-environment-based memory sharing system and method. The system comprises at least two hosts, wherein each host comprises at least one virtual machine, a coordinator and a memory module; the virtual machine, the coordinator and the memory module are electrically connected to each other; the coordinator of a local host is in remote communication connection with the coordinator of a remote host by means of an RDMA module; the virtual machine is used for receiving an access request initiated by an application program, and sending the access request to the coordinator; and the coordinator queries whether memory cache data requested by the access request is present in the memory module, and if so, the coordinator returns the memory cache data to the application program, and if not, the coordinator sends, by means of the RDMA module, the access request to the coordinator of the remote host in which memory data is stored. By means of the method in the present invention, a memory can be shared between virtual machines of the same host and between virtual machines across hosts.

Description

基于虚拟环境的内存共享系统及方法Memory sharing system and method based on virtual environment 技术领域technical field
本发明涉及内存共享技术领域,特别是涉及一种基于虚拟环境的内存共享系统及方法。The invention relates to the technical field of memory sharing, in particular to a memory sharing system and method based on a virtual environment.
背景技术Background technique
随着DRAM价格的不断下降,构建具有数百GB DRAM容量的商用服务器具有较好的成本效益。然而现在DRAM面临集成度低的问题,虽然大的集群中能达到TB级的主存,但是单节点的内存最大也就几百GB,很难满足大型应用的存储需求。在这一背景驱动下,通过内存语义访问远程数据的共享内存模型使统一的全局内存抽象对分布式计算非常有吸引力。然而在涉及到远程主机之间的通信时,传统的TCP/IP网络由于需要经过内核协议栈的处理,会引起非常大的开销,并不能很好地满足这种共享内存模型在低延迟上的需求。As DRAM prices continue to drop, it is cost-effective to build commodity servers with hundreds of gigabytes of DRAM capacity. However, DRAM now faces the problem of low integration. Although large clusters can reach TB-level main memory, the maximum memory of a single node is only a few hundred GB, which is difficult to meet the storage requirements of large-scale applications. Driven by this background, the shared memory model of accessing remote data through memory semantics makes a unified global memory abstraction very attractive for distributed computing. However, when it comes to the communication between remote hosts, the traditional TCP/IP network will cause a very large overhead due to the processing of the kernel protocol stack, and it cannot well meet the low-latency requirements of this shared memory model. need.
远程直接内存访问(RDMA模块)技术将协议栈的处理卸载到网卡上,网卡在解析数据包后直接利用DMA技术访问内存数据,避免了数据的多次拷贝。同时通过数据处理的卸载,将CPU从网络数据传输中解脱出来,提高了CPU使用的效率。通过这种零拷贝、绕过内核的特点,实现了数据传输过程中的低延迟、高吞吐量和高性能。The remote direct memory access (RDMA module) technology offloads the processing of the protocol stack to the network card, and the network card directly uses the DMA technology to access the memory data after parsing the data packet, avoiding multiple copies of the data. At the same time, through the offloading of data processing, the CPU is freed from the network data transmission, which improves the efficiency of CPU usage. Through this zero-copy, kernel-bypassing feature, low latency, high throughput, and high performance during data transfer are achieved.
针对同主机虚拟机之间的内存共享,已经有过不少深入的研究。但是从目前云计算的发展趋势来看,应用程序运行在虚拟环境中已经是不可阻挡的趋势,因此单纯地在物理主机上的内存共享已经不具有吸引力。我们需要考虑在虚拟机、容器等虚拟环境下,针对多个物理主机基础上的整个集群环境中的内存共享。There has been a lot of in-depth research on memory sharing between virtual machines on the same host. However, judging from the current development trend of cloud computing, it is an irresistible trend for applications to run in a virtual environment, so memory sharing on physical hosts is no longer attractive. We need to consider memory sharing in the entire cluster environment based on multiple physical hosts in virtual environments such as virtual machines and containers.
新加坡国立大学与加州大学的研究人员曾经提出了一个使用RDMA模块和数据缓存的分布式内存管理系统GAM。GAM在物理分布式集群上实现了一个内存共享的模型,可以管理分布在多个节点之间的空闲内存,以提供统一的内存模型。同时GAM提供了一组用户友好的API以进行内存操作。为了提高数据读取的性能,GAM实现了在每个主机上对数据的缓存,并对缓存一致性进行了深入的思考,区分了内存页面在共享时的不同状态,并设计了数据在读、写以及数据失效时的一致性处理。同时GAM在RDMA模块的使用上也采用了混合使用通信原语的方法,在数据传输中只固定使用单边原语WRITE。Researchers from the National University of Singapore and the University of California have proposed a distributed memory management system GAM using RDMA modules and data caching. GAM implements a memory sharing model on a physically distributed cluster and can manage free memory distributed among multiple nodes to provide a unified memory model. At the same time, GAM provides a set of user-friendly APIs for memory operations. In order to improve the performance of data reading, GAM implements the cache of data on each host, and deeply thinks about cache coherence, distinguishes the different states of memory pages when they are shared, and designs data when reading and writing data. And consistency handling when data fails. At the same time, GAM also adopts the method of mixing communication primitives in the use of the RDMA module, and only uses the unilateral primitive WRITE in data transmission.
国内外现有的内存共享的系统,大多数致力于同主机上不同虚拟机之间,或者不同物理主机之间的内存共享。对于整个虚拟集群上的内存共享的研究并不多。Most of the existing memory sharing systems at home and abroad are dedicated to memory sharing between different virtual machines on the same host, or between different physical hosts. There is not much research on memory sharing across virtual clusters.
技术问题technical problem
本发明提供一种基于虚拟环境的内存共享系统及方法,以解决现有内存共享方法无法高效实现虚拟集群上内存共享的问题。The present invention provides a memory sharing system and method based on a virtual environment, so as to solve the problem that the existing memory sharing method cannot efficiently realize memory sharing on a virtual cluster.
技术解决方案technical solutions
为解决上述技术问题,本发明提供了一种基于虚拟环境的内存共享系统,其包括:至少两台主机,每台主机内包括至少一个虚拟机、协调器和内存模块,虚拟机、协调器、内存模块相互电性连接,本地主机的协调器与远程主机的协调器通过RDMA模块进行远程通信连接;虚拟机用于接收应用程序发起的访问请求,并发送访问请求至协调器;协调器查询内存模块中是否存在访问请求所请求的内存数据;若存在,则将内存数据返回给应用程序;若不存在,则协调器经RDMA模块向存储有内存数据的远程主机的协调器发出访问请求。In order to solve the above technical problems, the present invention provides a memory sharing system based on a virtual environment, which includes: at least two hosts, each host includes at least one virtual machine, a coordinator and a memory module, a virtual machine, a coordinator, The memory modules are electrically connected to each other, and the coordinator of the local host and the coordinator of the remote host are connected by remote communication through the RDMA module; the virtual machine is used to receive the access request initiated by the application and send the access request to the coordinator; the coordinator queries the memory Whether the memory data requested by the access request exists in the module; if so, return the memory data to the application; if not, the coordinator sends an access request to the coordinator of the remote host that stores the memory data through the RDMA module.
作为本申请的进一步改进,内存模块包括共享内存、缓存和操作系统及应用程序分配的地址空间,缓存用于缓存远程主机的内存数据,协调器接收到访问请求后,查询缓存中是否存在内存数据。As a further improvement of this application, the memory module includes shared memory, cache, and address space allocated by the operating system and application programs. The cache is used to cache the memory data of the remote host. After the coordinator receives the access request, it queries whether there is memory data in the cache. .
作为本申请的进一步改进,虚拟机内设置有API接口、库函数和虚拟PCI设备,应用程序通过API接口调用库函数以操作虚拟PCI设备访问内存模块。As a further improvement of the present application, an API interface, a library function and a virtual PCI device are provided in the virtual machine, and the application program calls the library function through the API interface to operate the virtual PCI device to access the memory module.
作为本申请的进一步改进,协调器包括通信模块、监测模块和消息处理模块;通信模块用于实现同一主机上不同虚拟机之间或不同主机之间的通信;监测模块用于监测虚拟机的活动,以及通过分析内存模块的访问日志以区分每张内存页的访问类型,并按照预设策略规则为不同访问类型的内存页配置对应的缓存策略;消息处理模块用于对通信模块接收到的消息进行解析处理,并为不同类型的消息分发预先配置好的消息处理函数。As a further improvement of the present application, the coordinator includes a communication module, a monitoring module and a message processing module; the communication module is used to implement communication between different virtual machines on the same host or between different hosts; the monitoring module is used to monitor the activities of the virtual machines, And by analyzing the access log of the memory module to distinguish the access type of each memory page, and configure the corresponding cache policy for the memory pages of different access types according to the preset policy rules; the message processing module is used for the message received by the communication module. Parse processing and distribute pre-configured message handlers for different types of messages.
作为本申请的进一步改进,不同主机之间的通信模块通过RDMA模块的SEND/RECV双边原语、WRITE单边原语或WRITE_WITH_IMM原语实现数据传输过程。As a further improvement of the present application, the communication modules between different hosts implement the data transmission process through the SEND/RECV bilateral primitive, the WRITE unilateral primitive or the WRITE_WITH_IMM primitive of the RDMA module.
作为本申请的进一步改进,当通信模块之间传输访问请求或内存页失效的控制类消息时,通过SEND/RECV双边原语实现数据传输;当通信模块之间进行传输大量数据时,通过WRITE单边原语实现数据传输;当通信模块之间传输应答类消息时,通过WRITE_WITH_IMM原语实现数据传输。As a further improvement of this application, when an access request or a memory page failure control message is transmitted between communication modules, data transmission is realized through the SEND/RECV bilateral primitive; when a large amount of data is transmitted between communication modules, the WRITE single The edge primitive realizes data transmission; when the response message is transmitted between communication modules, the data transmission is realized through the WRITE_WITH_IMM primitive.
作为本申请的进一步改进,监测模块分析内存模块的访问日志以区分每张内存页的访问类型,并按照预设策略规则为不同访问类型的内存页配置对应的缓存策略,包括;当内存页被目标远程主机访问的次数达到第一预设条件时,将内存页的访问类型划分为独占访问,并将内存页迁移至目标远程主机的内存模块;当内存页上读取数据的记录达到第二预设条件时,将内存页的访问类型划分为读共享,并将内存页存储至访问内存页频率最高的远程主机上,且将权限设置为只读;当内存页上写入数据的记录达到第三预设条件时,将内存页的访问类型划分为写共享,并禁止同步内存页至其他远程主机。As a further improvement of the present application, the monitoring module analyzes the access log of the memory module to distinguish the access type of each memory page, and configures corresponding cache policies for memory pages of different access types according to preset policy rules, including; When the number of accesses by the target remote host reaches the first preset condition, the access type of the memory page is divided into exclusive access, and the memory page is migrated to the memory module of the target remote host; when the record of read data on the memory page reaches the second In the preset condition, the access type of the memory page is divided into read sharing, and the memory page is stored on the remote host with the highest frequency of accessing the memory page, and the permission is set to read-only; when the record of written data on the memory page reaches In the third preset condition, the access type of the memory page is divided into write sharing, and synchronization of the memory page to other remote hosts is prohibited.
作为本申请的进一步改进,监测模块在监测到远程主机访问本地主机的内存模块时,生成访问日志,访问日志记录了访问主机、访问地址和访问类型。As a further improvement of the present application, the monitoring module generates an access log when monitoring that the remote host accesses the memory module of the local host, and the access log records the access host, the access address and the access type.
作为本申请的进一步改进,虚拟机与协调器之间的通信以进程通信的方式实现。As a further improvement of the present application, the communication between the virtual machine and the coordinator is implemented by way of process communication.
为解决上述技术问题,本发明还提供了一种基于虚拟环境的内存共享方法,其应用于上述之一的基于虚拟环境的内存共享系统,方法包括:虚拟机接收到应用程序发起的访问请求时,将访问请求发送至协调器;协调器分析访问请求,并判断访问请求的访问地址是本地主机还是远程主机;若是本地主机,则从本地主机中获取访问请求所请求的数据并反馈至应用程序;若是远程主机,则判断内存模块中是否缓存有访问请求所请求的数据;当内存模块中缓存有访问请求所请求的数据时,将缓存的访问请求所请求的数据反馈至应用程序;当内存模块中不存在访问请求所请求的数据时,协调器通过RDMA模块向远程主机的协调器发送访问请求,并将访问得到的数据反馈至应用程序。In order to solve the above technical problems, the present invention also provides a virtual environment-based memory sharing method, which is applied to one of the above-mentioned virtual environment-based memory sharing systems. The method includes: when the virtual machine receives an access request initiated by an application program. , send the access request to the coordinator; the coordinator analyzes the access request and determines whether the access address of the access request is the local host or the remote host; if it is the local host, it obtains the data requested by the access request from the local host and feeds it back to the application ; If it is a remote host, judge whether the data requested by the access request is cached in the memory module; when the data requested by the access request is cached in the memory module, the data requested by the cached access request is fed back to the application; When the data requested by the access request does not exist in the module, the coordinator sends an access request to the coordinator of the remote host through the RDMA module, and feeds back the data obtained by the access to the application program.
有益效果beneficial effect
本发明的有益效果是:本发明的基于虚拟环境的内存共享系统通过在主机内设置协调器,通过协调器实现主机内不同虚拟机之间的数据通信以实现内存共享,而针对于不同的主机之间,其通过利用RDMA模块实现不同主机的协调器之间的通信,从而实现不同主机之间的通信,最终实现整个虚拟集群的内存共享,且利用RDMA模块有效的降低了不同主机间数据交互的延迟。The beneficial effects of the present invention are: the memory sharing system based on the virtual environment of the present invention sets up a coordinator in the host, and realizes data communication between different virtual machines in the host through the coordinator to realize memory sharing, and aims at different hosts. Between them, it uses the RDMA module to realize the communication between the coordinators of different hosts, so as to realize the communication between different hosts, and finally realize the memory sharing of the entire virtual cluster, and the use of the RDMA module effectively reduces the data interaction between different hosts. Delay.
附图说明Description of drawings
图1是本发明实施例基于虚拟环境的内存共享系统的结构示意图;1 is a schematic structural diagram of a memory sharing system based on a virtual environment according to an embodiment of the present invention;
图2是本发明实施例基于虚拟环境的内存共享系统的内存模块的结构示意图;2 is a schematic structural diagram of a memory module of a memory sharing system based on a virtual environment according to an embodiment of the present invention;
图3是本发明实施例基于虚拟环境的内存共享系统的虚拟机的结构示意图;3 is a schematic structural diagram of a virtual machine of a memory sharing system based on a virtual environment according to an embodiment of the present invention;
图4是本发明实施例基于虚拟环境的内存共享系统的协调器的结构示意图;4 is a schematic structural diagram of a coordinator of a virtual environment-based memory sharing system according to an embodiment of the present invention;
图5是本发明实施例基于虚拟环境的内存共享系统的缓存的内存页被修改时需要做出的操作流程示意图;5 is a schematic diagram of an operation flow that needs to be performed when a cached memory page of a memory sharing system based on a virtual environment is modified according to an embodiment of the present invention;
图6是本发明实施例基于虚拟环境的内存共享方法的流程示意图。FIG. 6 is a schematic flowchart of a memory sharing method based on a virtual environment according to an embodiment of the present invention.
本发明的最佳实施方式BEST MODE FOR CARRYING OUT THE INVENTION
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅是本发明的一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
本发明中的术语“第一”、“第二”、“第三”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”、“第三”的特征可以明示或者隐含地包括至少一个该特征。本发明的描述中,“多个”的含义是至少两个,例如两个,三个等,除非另有明确具体的限定。本发明实施例中所有方向性指示(诸如上、下、左、右、前、后……)仅用于解释在某一特定姿态(如附图所示)下各部件之间的相对位置关系、运动情况等,如果该特定姿态发生改变时,则该方向性指示也相应地随之改变。此外,术语“包括”和“具有”以及它们任何变形,意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元,而是可选地还包括没有列出的步骤或单元,或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。The terms "first", "second" and "third" in the present invention are only used for description purposes, and should not be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature defined as "first", "second", "third" may expressly or implicitly include at least one of that feature. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship between various components under a certain posture (as shown in the accompanying drawings). , motion situation, etc., if the specific posture changes, the directional indication also changes accordingly. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.
在本文中提及“实施例”意味着,结合实施例描述的特定特征、结构或特性可以包含在本发明的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例,也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是,本文所描述的实施例可以与其它实施例相结合。Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
图1是本发明实施例的基于虚拟环境的内存共享系统的结构示意图。如图1所示,该基于虚拟环境的内存共享系统100包括:至少两台主机1,每台主机1内包括至少一个虚拟机2、协调器3和内存模块4,虚拟机2、协调器3、内存模块4相互电性连接,本地主机的协调器3与远程主机的协调器3通过RDMA模块5进行远程通信连接;虚拟机2用于接收应用程序发起的访问请求,并发送访问请求至协调器3;协调器3查询内存模块4中是否存在访问请求所请求的内存数据;若存在,则将内存数据返回给应用程序;若不存在,则协调器3经RDMA模块5向存储有内存数据的远程主机的协调器3发出访问请求。FIG. 1 is a schematic structural diagram of a memory sharing system based on a virtual environment according to an embodiment of the present invention. As shown in FIG. 1, the virtual environment-based memory sharing system 100 includes: at least two hosts 1, each host 1 includes at least one virtual machine 2, a coordinator 3 and a memory module 4, the virtual machine 2, the coordinator 3 , the memory modules 4 are electrically connected to each other, and the coordinator 3 of the local host and the coordinator 3 of the remote host are connected by remote communication through the RDMA module 5; the virtual machine 2 is used to receive the access request initiated by the application program, and send the access request to the coordinator. Coordinator 3; Coordinator 3 inquires whether there is memory data requested by the access request in memory module 4; if so, returns the memory data to the application; The coordinator 3 of the remote host issues an access request.
需要说明的是,该基于虚拟环境的内存共享系统100基于KVM/QEMU开发,其中,虚拟机2在KVM上运行,本实施例中,利用QEMU上个模拟虚拟机来模拟集群上的共享内存。同一个主机1中可以构建多个虚拟机2。需要说明的是,当应用程序向A主机的虚拟机发起访问请求时,即以A主机为本地主机,而当应用程序访问的内存数据在B主机的内存模块时,则B为远程主机。It should be noted that the virtual environment-based memory sharing system 100 is developed based on KVM/QEMU, wherein the virtual machine 2 runs on KVM. In this embodiment, the last simulated virtual machine of QEMU is used to simulate the shared memory on the cluster. Multiple virtual machines 2 can be built on the same host 1. It should be noted that when an application initiates an access request to the virtual machine of host A, host A is the local host, and when the memory data accessed by the application is in the memory module of host B, then host B is the remote host.
具体地,虚拟机2接收到应用程序发起的访问请求时,将访问请求发送至协调器3;协调器3分析访问请求,并判断访问请求的访问地址是本地主机还是远程主机;若是本地主机,则从本地主机中获取访问请求所请求的内存数据并反馈至应用程序;若是远程主机,则判断内存模块4中是否缓存有访问请求所请求的内存数据;当内存模块4中缓存有访问请求所请求的内存数据时,将缓存的访问请求所请求的内存数据反馈至应用程序;当内存模块4中不存在访问请求所请求的内存数据时,协调器3通过RDMA模块5向远程主机的协调器3发送访问请求,并将访问得到的内存数据反馈至应用程序。Specifically, when the virtual machine 2 receives the access request initiated by the application, it sends the access request to the coordinator 3; the coordinator 3 analyzes the access request, and judges whether the access address of the access request is the local host or the remote host; if it is the local host, Then obtain the memory data requested by the access request from the local host and feed it back to the application; if it is a remote host, then determine whether the memory data requested by the access request is cached in the memory module 4; When the memory data requested, the memory data requested by the cached access request is fed back to the application program; when the memory data requested by the access request does not exist in the memory module 4, the coordinator 3 sends the data to the coordinator of the remote host through the RDMA module 5 3 Send an access request, and feed back the memory data obtained by the access to the application.
本实施例的基于虚拟环境的内存共享系统通过在主机内设置协调器3,通过协调器3实现主机1内不同虚拟机2之间的数据通信以实现内存共享,而针对于不同的主机1之间,其通过利用RDMA模块5实现不同主机1的协调器3之间的通信,从而实现不同主机1之间的通信,最终实现整个虚拟集群的内存共享,且利用RDMA模块5有效的降低了不同主机1间数据交互的延迟。The memory sharing system based on the virtual environment of this embodiment sets the coordinator 3 in the host, and realizes the data communication between different virtual machines 2 in the host 1 through the coordinator 3 to realize memory sharing. During the time, it realizes the communication between the coordinators 3 of different hosts 1 by using the RDMA module 5, so as to realize the communication between different hosts 1, and finally realize the memory sharing of the whole virtual cluster, and the RDMA module 5 effectively reduces the different Latency of data exchange between hosts 1.
进一步的,如图2所示,内存模块4包括共享内存41、缓存42和操作系统及应用程序分配的地址空间43,缓存42用于缓存远程主机的内存数据,协调器3接收到访问请求后,查询缓存42中是否存在内存数据。Further, as shown in FIG. 2, the memory module 4 includes a shared memory 41, a cache 42 and an address space 43 allocated by the operating system and application programs. The cache 42 is used to cache the memory data of the remote host. After the coordinator 3 receives the access request, , query whether the memory data exists in the cache 42 .
具体地,每个主机1的共享内存41的大小在进行系统初始化时由命令行指出,而在实际分配该共享内存41时,还需分配一固定大小的缓存42,该缓存42用于缓存远程主机的内存数据。需要说明的时,缓存中的内存数据对虚拟机2是不可见的,因此,当虚拟机将访问请求发送给协调器3时,由协调器3查询缓存42中是否存在想要查询的内存数据,若存在,则直接将内存数据返回给应用程序,若不存在,则向远程主机发送该访问请求。Specifically, the size of the shared memory 41 of each host 1 is indicated by the command line during system initialization, and when the shared memory 41 is actually allocated, a fixed-size cache 42 needs to be allocated, and the cache 42 is used to cache remote Host memory data. It should be noted that the memory data in the cache is invisible to the virtual machine 2. Therefore, when the virtual machine sends an access request to the coordinator 3, the coordinator 3 queries the cache 42 whether there is the memory data to be queried. , if it exists, directly return the memory data to the application, if not, send the access request to the remote host.
进一步的,如图3所示,虚拟机2内设置有API接口21、库函数22和虚拟PCI设备23,应用程序通过API接口21调用库函数22以操作虚拟PCI设备23访问内存模块4。Further, as shown in FIG. 3 , the virtual machine 2 is provided with an API interface 21 , a library function 22 and a virtual PCI device 23 , and the application calls the library function 22 through the API interface 21 to operate the virtual PCI device 23 to access the memory module 4 .
具体地,该虚拟PCI设备23是基于QEMU建立,并进行了寄存器数量和消息处理函数扩充。其中,库函数22包括有QEMU自带的库函数以及预先配置的针对于应答消息的库函数。Specifically, the virtual PCI device 23 is established based on QEMU, and the number of registers and message processing functions are expanded. Among them, the library function 22 includes a library function provided by QEMU and a preconfigured library function for the response message.
进一步的,如图4所示,协调器3包括通信模块31、监测模块32和消息处理模块33;通信模块31用于实现同一主机1上不同虚拟机2之间或不同主机之间的通信;监测模块32用于监测虚拟机2的活动,以及通过分析内存模块4的访问日志以区分每张内存页的访问类型,并按照预设策略规则为不同访问类型的内存页配置对应的缓存策略;消息处理模块33用于对通信模块接收到的消息进行解析处理,并为不同类型的消息分发预先配置好的消息处理函数。Further, as shown in FIG. 4 , the coordinator 3 includes a communication module 31, a monitoring module 32 and a message processing module 33; the communication module 31 is used to implement communication between different virtual machines 2 on the same host 1 or between different hosts; monitoring The module 32 is used to monitor the activity of the virtual machine 2, and to distinguish the access type of each memory page by analyzing the access log of the memory module 4, and configure the corresponding cache policy for the memory pages of different access types according to preset policy rules; message The processing module 33 is configured to parse and process the messages received by the communication module, and distribute pre-configured message processing functions for different types of messages.
具体地,协调器2时一个可装载模块,负责维护主机1的虚拟机信息、与虚拟机2通信、维护主机1上的共享内存、与远程主机通信等任务。Specifically, the coordinator 2 is a loadable module responsible for tasks such as maintaining the virtual machine information of the host 1, communicating with the virtual machine 2, maintaining the shared memory on the host 1, and communicating with the remote host.
本实施例中,通信模块31负责完成虚拟机2之间的通信,由于同一主机和不同主机在通信上存在很大的不同,本实施例中将通信分为两个层次,一是同一主机1上不同虚拟机2以及虚拟机2与协调器3的通信,二是不同主机1之间的通信。虚拟机2与协调器3之间的通信以进程的方式实现,不涉及网络上的数据传输,因此,传输效率更高。In this embodiment, the communication module 31 is responsible for completing the communication between the virtual machines 2. Since the same host and different hosts have great differences in communication, in this embodiment, the communication is divided into two levels, one is the same host 1 The communication between different virtual machines 2 and the virtual machine 2 and the coordinator 3, and the second is the communication between different hosts 1. The communication between the virtual machine 2 and the coordinator 3 is implemented in a process manner, and does not involve data transmission on the network, so the transmission efficiency is higher.
进一步的,不同主机1之间的通信模块31通过RDMA模块5的SEND/RECV双边原语、WRITE单边原语或WRITE_WITH_IMM原语实现数据传输过程。具体地,当通信模块31之间传输访问请求或内存页失效的控制类消息时,通过SEND/RECV双边原语实现数据传输;当通信模块31之间进行传输大量数据时,通过WRITE单边原语实现数据传输;当通信模块31之间传输应答类消息时,通过WRITE_WITH_IMM原语实现数据传输。其中,内存页失效是指本地主机中缓存存储的远程主机的内存页,在远程主机的内存页的数据发送变化是,该缓存中存储的内存页失效。Further, the communication module 31 between different hosts 1 implements the data transmission process through the SEND/RECV bilateral primitive, the WRITE unilateral primitive or the WRITE_WITH_IMM primitive of the RDMA module 5 . Specifically, when the communication module 31 transmits an access request or a control message of memory page failure, data transmission is realized through the SEND/RECV bilateral primitive; when a large amount of data is transmitted between the communication modules 31, the WRITE unilateral primitive is used. The data transmission is realized by the WRITE_WITH_IMM primitive; when the response message is transmitted between the communication modules 31, the data transmission is realized through the WRITE_WITH_IMM primitive. The memory page invalidation refers to the memory page of the remote host stored in the cache in the local host, and the data transmission change of the memory page of the remote host is that the memory page stored in the cache is invalidated.
具体地,不同主机1之间的连接均是以RDMA为基础,在数据传输过程中,利用软件定义网络的思想,将控制路径和数据路径分离,而这两种路径上使用不同的数据传输原语,并且,为了避免两者的干扰,将控制路径和数据路径的传输分为两个连接,具体为:Specifically, the connections between different hosts 1 are based on RDMA. In the process of data transmission, the idea of software-defined networking is used to separate the control path and the data path, and different data transmission sources are used on these two paths. In addition, in order to avoid the interference of the two, the transmission of the control path and the data path is divided into two connections, specifically:
1、对于访问请求类的消息以及一些内存页失效等的控制类消息,其携带的数据通常较小,同时为了能够保证控制命令的有效传递,确保另一端可以接收到,且具有一定的可靠性,则可使用RDMA模块5中的SEND/RECV双边原语,在此种方式下,需要CPU的参与,接收方需要先使用RECV准备好接收数据,再由发送方使用SEND发送数据,从而保证接收方收到了消息。1. For access request messages and control messages such as some memory page failures, the data they carry is usually small. At the same time, in order to ensure the effective transmission of control commands, ensure that the other end can receive them, and has certain reliability. , the SEND/RECV bilateral primitive in RDMA module 5 can be used. In this way, the participation of the CPU is required. The receiver needs to use RECV to prepare for receiving data, and then the sender uses SEND to send data to ensure the reception. Fang received the message.
2、对于一般单纯的数据传输,数据量大,对延迟的要求比较高。对此我们使用RDMA模块5的WRITE单边原语,在此种方式下,只要两端建立起了连接,发送方可以直接将数据写到接收方的内存中,而不需要接收方CPU的参与,其通过直接访问远程内存的形式,进一步降低了系统的延迟。2. For general simple data transmission, the amount of data is large, and the requirements for delay are relatively high. For this, we use the WRITE unilateral primitive of RDMA module 5. In this way, as long as the two ends establish a connection, the sender can directly write the data to the receiver's memory without the participation of the receiver's CPU. , which further reduces system latency in the form of direct access to remote memory.
3、数据传输中部分数据是主机1在接收到控制类消息后需要返回应答包以确定收到消息,对于此类消息,通常只需在消息头部携带数据即可,因此,采用WRITE_WITH_IMM原语来传输此类数据,该WRITE_WITH_IMM原语在数据传输时不携带有效载荷,降低数据传输所消耗的资源。3. Part of the data in the data transmission is that the host 1 needs to return a response packet after receiving the control message to determine the received message. For this type of message, it is usually only necessary to carry the data in the message header. Therefore, the WRITE_WITH_IMM primitive is used To transmit such data, the WRITE_WITH_IMM primitive does not carry payload during data transmission, reducing the resources consumed by data transmission.
监测模块32负责监听虚拟机2的开启、关闭等活动,以及负责通过分析内存模块4的访问日志,从而区分内存模块4中每张内存页的访问类型,并按照预设策略规则为不同访问类型的内存页配置对应的缓存策略。The monitoring module 32 is responsible for monitoring the opening, closing and other activities of the virtual machine 2, and is responsible for analyzing the access log of the memory module 4, thereby distinguishing the access type of each memory page in the memory module 4, and according to the preset policy rules for different access types The corresponding cache policy is configured for the memory page.
需要说明的是,监测模块32在监测到远程主机访问本地主机的内存模块4时,生成访问日志,访问日志记录了访问主机、访问地址和访问类型等内容。It should be noted that the monitoring module 32 generates an access log when monitoring that the remote host accesses the memory module 4 of the local host, and the access log records the contents of the access host, the access address, and the access type.
消息处理模块33负责对通信模块31接收到的消息进行解析处理,对于不同类型的消息,根据消息结构确认消息类型,再将消息按类型分发给不同的消息处理函数进行处理。其中,消息的类型可以根据访问对象划分,如访问本地主机的消息还是访问远程主机的消息,还可以为新建立的虚拟机2初始化时,向虚拟集群广播该新建的虚拟机2的消息,或者是各个主机1的共享内存的大小的消息等。消息处理函数预先进行设定。The message processing module 33 is responsible for analyzing and processing the messages received by the communication module 31. For different types of messages, the message types are confirmed according to the message structure, and then the messages are distributed to different message processing functions for processing. Among them, the type of the message can be divided according to the access object, such as the message of accessing the local host or the message of accessing the remote host, and can also broadcast the message of the newly created virtual machine 2 to the virtual cluster when the newly created virtual machine 2 is initialized, or It is a message or the like of the size of the shared memory of each host 1 . The message processing function is preset.
进一步的,为了能够加快对远程数据的访问速度,降低系统的延迟,本实施例中,监测模块31分析内存模块4的访问日志以区分每张内存页的访问类型,并按照预设策略规则为不同访问类型的内存页配置对应的缓存策略具体为:Further, in order to speed up the access speed to the remote data and reduce the delay of the system, in this embodiment, the monitoring module 31 analyzes the access log of the memory module 4 to distinguish the access type of each memory page, and according to the preset policy rule is: The cache policies corresponding to the memory page configurations of different access types are as follows:
1、当内存页被目标远程主机访问的次数达到第一预设条件时,将内存页的访问类型划分为独占访问,并将内存页迁移至目标远程主机的内存模块4。1. When the number of times the memory page is accessed by the target remote host reaches the first preset condition, the access type of the memory page is divided into exclusive access, and the memory page is migrated to the memory module 4 of the target remote host.
需要说明的是,该第一预设条件预先设置。It should be noted that the first preset condition is preset.
具体地,当本地主机的内存页被某个远程主机的访问次数占总访问次数的90%以上时,将此时的访问类型划分为独占访问,此时,考虑到数据传输的延迟过高,为了降低延迟,将该内存页迁移至远程主机,并由远程主机的协调器3确认在内存模块4中的设实际存放地址,若有其他远程主机偶尔访问该内存页,向该远程主机发出通知以确认该远程主机可以访问到该内存页。Specifically, when the number of accesses of the memory page of the local host by a remote host accounts for more than 90% of the total number of accesses, the access type at this time is divided into exclusive access. At this time, considering that the delay of data transmission is too high, In order to reduce the delay, the memory page is migrated to the remote host, and the coordinator 3 of the remote host confirms the actual storage address in the memory module 4. If other remote hosts occasionally access the memory page, a notification is sent to the remote host. to confirm that the remote host can access the memory page.
2、当内存页上读取数据的记录达到第二预设条件时,将内存页的访问类型划分为读共享,并将内存页存储至访问内存页频率最高的远程主机上,且将权限设置为只读。2. When the record of read data on the memory page reaches the second preset condition, the access type of the memory page is divided into read sharing, and the memory page is stored on the remote host with the highest frequency of accessing the memory page, and the permissions are set. is read-only.
需要说明的是,该第二预设条件预先设置。It should be noted that the second preset condition is preset.
具体地,当检测到内存页有80%以上的记录是读取数据时,将内存页的访问类型划分为读共享,此时,需要将内存页缓存到对该内存页访问频率最高的远程主机上,并在缓存映射表上做出记录,在远程主机上,由协调器3将数据放置在预留的缓存空间中以备使用,并将权限设置为只读。例如,图5所示为当缓存的内存页被修改时需要做出的操作流程,其中,node0发出写请求,node1是保留有原内存页的节点,node2是有缓存数据的节点,当node0发出写请求时,node1会向node0回复请求并处理数据访问,同时,node1会向node2等有内存页缓存的节点发出内存页失效的通知,node2等会修改缓存页的标志位,并回复应答包,当后续应用程序需要再次访问时,需要重新将数据缓存到本地主机。Specifically, when it is detected that more than 80% of the records of the memory page are read data, the access type of the memory page is divided into read sharing. At this time, the memory page needs to be cached to the remote host with the highest access frequency to the memory page. On the remote host, the coordinator 3 places the data in the reserved cache space for use, and sets the permission to read-only. For example, Figure 5 shows the operation process that needs to be done when the cached memory page is modified. Among them, node0 issues a write request, node1 is the node that retains the original memory page, node2 is the node that has cached data, and when node0 sends a write request When writing a request, node1 will reply to the request to node0 and process data access. At the same time, node1 will send a notification of memory page invalidation to node2 and other nodes that have memory page cache, and node2, etc. will modify the flag of the cached page and reply the response packet. When subsequent applications need to access again, the data needs to be re-cached to the local host.
3、当内存页上写入数据的记录达到第三预设条件时,将内存页的访问类型划分为写共享,并禁止同步内存页至其他远程主机。3. When the record of written data on the memory page reaches the third preset condition, the access type of the memory page is divided into write sharing, and synchronization of the memory page to other remote hosts is prohibited.
需要说明的是,该第三预设条件预先设置。It should be noted that the third preset condition is preset.
具体地,考虑到写数据的特殊性,当内存页中有50%以上的记录是写入数据时,将内存页的访问类型划分为写共享,由于写操作发生时内存页的数据一直在改变,如果将内存页缓存到多个远程主机上,考虑到数据一致性的维护,多个远程主机之间需要频繁地交换信息,需要大量的同步操作,甚至涉及到很多锁操作,大大降低系统的性能,造成数据传输的延迟大幅度上升,因此,写共享的内存页不做考虑,后续仍然是应用程序发出请求,由协调器3接管操作并向远程主机访问数据。Specifically, considering the particularity of write data, when more than 50% of the records in the memory page are written data, the access type of the memory page is divided into write sharing, because the data of the memory page keeps changing when the write operation occurs. , if the memory pages are cached on multiple remote hosts, considering the maintenance of data consistency, information needs to be exchanged frequently between multiple remote hosts, requiring a lot of synchronization operations, and even involving a lot of lock operations, which greatly reduces the system performance. performance, resulting in a significant increase in the delay of data transmission. Therefore, the shared memory page is not considered, and the application program still sends a request, and the coordinator 3 takes over the operation and accesses data to the remote host.
图6展示了本发明基于虚拟环境的内存共享方法,其应用于上述之一的基于虚拟环境的内存共享系统,所述方法包括:6 shows the virtual environment-based memory sharing method of the present invention, which is applied to one of the above-mentioned virtual environment-based memory sharing systems, and the method includes:
步骤S1:虚拟机接收到应用程序发起的访问请求时,将访问请求发送至协调器;Step S1: when the virtual machine receives the access request initiated by the application, it sends the access request to the coordinator;
步骤S2:协调器分析访问请求,并判断访问请求的访问地址是本地主机还是远程主机;若是本地主机,则执行步骤S3;若是远程主机,则执行步骤S4。Step S2: The coordinator analyzes the access request, and determines whether the access address of the access request is a local host or a remote host; if it is a local host, go to step S3; if it is a remote host, go to step S4.
步骤S3:从本地主机中获取访问请求所请求的数据并反馈至应用程序;Step S3: obtain the data requested by the access request from the local host and feed it back to the application;
步骤S4:判断内存模块中是否缓存有访问请求所请求的数据;当内存模块中缓存有访问请求所请求的数据时,执行步骤S5;当内存模块中不存在访问请求所请求的数据时,执行步骤S6。Step S4: judging whether the data requested by the access request is cached in the memory module; when the data requested by the access request is cached in the memory module, execute step S5; when the data requested by the access request does not exist in the memory module, execute Step S6.
步骤S5:将缓存的访问请求所请求的数据反馈至应用程序;Step S5: feedback the data requested by the cached access request to the application;
步骤S6:协调器通过RDMA模块向远程主机的协调器发送访问请求,并将访问得到的数据反馈至应用程序。Step S6: The coordinator sends an access request to the coordinator of the remote host through the RDMA module, and feeds back the data obtained by the access to the application program.
本实施例通过在主机内设置协调器,通过协调器实现主机内不同虚拟机知己的数据通信以实现内存共享,而针对于不同的主机之间,其通过利用RDMA模块实现不同主机的协调器之间的通信,从而实现不同主机之间的通信,最终实现整个虚拟集群的内存共享,且利用RDMA模块有效的降低了不同主机间数据交互的延迟。In this embodiment, a coordinator is set in the host, and the coordinator realizes the data communication of different virtual machines in the host to realize memory sharing, and for different hosts, the RDMA module is used to realize the communication between the coordinators of different hosts. Therefore, the communication between different hosts is realized, and the memory sharing of the entire virtual cluster is finally realized, and the RDMA module is used to effectively reduce the delay of data interaction between different hosts.
以上对发明的具体实施方式进行了详细说明,但其只作为范例,本发明并不限制于以上描述的具体实施方式。对于本领域的技术人员而言,任何对该发明进行的等同修改或替代也都在本发明的范畴之中,因此,在不脱离本发明的精神和原则范围下所作的均等变换和修改、改进等,都应涵盖在本发明的范围内。The specific embodiments of the invention have been described in detail above, but they are only used as examples, and the present invention is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present invention. Therefore, equivalent changes, modifications and improvements made without departing from the spirit and principle scope of the present invention etc., should be included in the scope of the present invention.

Claims (10)

  1. 一种基于虚拟环境的内存共享系统,其特征在于,其包括: A memory sharing system based on a virtual environment, characterized in that it includes:
    至少两台主机,每台所述主机内包括至少一个虚拟机、协调器和内存模块,所述虚拟机、所述协调器、所述内存模块相互电性连接,本地主机的协调器与远程主机的协调器通过RDMA模块进行远程通信连接;At least two hosts, each of which includes at least one virtual machine, a coordinator, and a memory module, the virtual machine, the coordinator, and the memory module are electrically connected to each other, and the coordinator of the local host is connected to the remote host. The coordinator is connected by remote communication through the RDMA module;
    所述虚拟机用于接收应用程序发起的访问请求,并发送所述访问请求至所述协调器;The virtual machine is configured to receive an access request initiated by an application, and send the access request to the coordinator;
    所述协调器查询所述内存模块中是否存在所述访问请求的内存数据;若存在,则将所述内存数据返回给所述应用程序;若不存在,则所述协调器经所述RDMA模块向存储有所述内存数据的远程主机的协调器发出所述访问请求。The coordinator inquires whether the memory data of the access request exists in the memory module; if so, returns the memory data to the application; if not, the coordinator passes the RDMA module The access request is issued to the coordinator of the remote host where the memory data is stored.
  2. 根据权利要求1所述的基于虚拟环境的内存共享系统,其特征在于,所述内存模块包括共享内存、缓存和操作系统及应用程序分配的地址空间,所述缓存用于缓存远程主机的内存数据,所述协调器接收到所述访问请求后,查询所述缓存中是否存在所述内存数据。 The memory sharing system based on a virtual environment according to claim 1, wherein the memory module comprises a shared memory, a cache and an address space allocated by an operating system and an application program, and the cache is used to cache memory data of a remote host , after receiving the access request, the coordinator queries whether the memory data exists in the cache.
  3. 根据权利要求1所述的基于虚拟环境的内存共享系统,其特征在于,所述虚拟机内设置有API接口、库函数和虚拟PCI设备,所述应用程序通过所述API接口调用所述库函数以操作所述虚拟PCI设备访问所述内存模块。 The memory sharing system based on a virtual environment according to claim 1, wherein the virtual machine is provided with an API interface, a library function and a virtual PCI device, and the application calls the library function through the API interface The memory module is accessed by operating the virtual PCI device.
  4. 根据权利要求1所述的基于虚拟环境的内存共享系统,其特征在于,所述协调器包括通信模块、监测模块和消息处理模块;所述通信模块用于实现同一主机上不同所述虚拟机之间或不同主机之间的通信;所述监测模块用于监测所述虚拟机的活动,以及通过分析所述内存模块的访问日志以区分每张内存页的访问类型,并按照预设策略规则为不同访问类型的内存页配置对应的缓存策略;所述消息处理模块用于对所述通信模块接收到的消息进行解析处理,并为不同类型的消息分发预先配置好的消息处理函数。 The memory sharing system based on a virtual environment according to claim 1, wherein the coordinator comprises a communication module, a monitoring module and a message processing module; the communication module is used to realize the communication between different virtual machines on the same host. Communication between different hosts; the monitoring module is used to monitor the activity of the virtual machine, and to distinguish the access type of each memory page by analyzing the access log of the memory module, and according to the preset policy rules as different The memory page of the access type is configured with a corresponding cache policy; the message processing module is configured to parse and process the message received by the communication module, and distribute pre-configured message processing functions for different types of messages.
  5. 根据权利要求4所述的基于虚拟环境的内存共享系统,其特征在于,不同主机之间的所述通信模块通过所述RDMA模块的SEND/RECV双边原语、WRITE单边原语或WRITE_WITH_IMM原语实现数据传输过程。 The virtual environment-based memory sharing system according to claim 4, wherein the communication modules between different hosts use the SEND/RECV bilateral primitive, WRITE unilateral primitive or WRITE_WITH_IMM primitive of the RDMA module Implement the data transfer process.
  6. 根据权利要求5所述的基于虚拟环境的内存共享系统,其特征在于,当所述通信模块之间传输所述访问请求或内存页失效的控制类消息时,通过所述SEND/RECV双边原语实现数据传输;当所述通信模块之间进行传输大量数据时,通过所述WRITE单边原语实现数据传输;当所述通信模块之间传输应答类消息时,通过所述WRITE_WITH_IMM原语实现数据传输。 The memory sharing system based on a virtual environment according to claim 5, wherein when the access request or the control message of memory page failure is transmitted between the communication modules, the SEND/RECV bilateral primitive is used. Realize data transmission; when a large amount of data is transmitted between the communication modules, data transmission is realized through the WRITE unilateral primitive; when a response message is transmitted between the communication modules, the data is realized through the WRITE_WITH_IMM primitive transmission.
  7. 根据权利要求4所述的基于虚拟环境的内存共享系统,其特征在于,所述监测模块分析所述内存模块的访问日志以区分每张内存页的访问类型,并按照预设策略规则为不同访问类型的内存页配置对应的缓存策略,包括; The memory sharing system based on a virtual environment according to claim 4, wherein the monitoring module analyzes the access log of the memory module to distinguish the access type of each memory page, and assigns different access types according to preset policy rules. The cache policy corresponding to the type of memory page configuration, including;
    当所述内存页被目标远程主机访问的次数达到第一预设条件时,将所述内存页的访问类型划分为独占访问,并将所述内存页迁移至所述目标远程主机的内存模块;When the number of times the memory page is accessed by the target remote host reaches a first preset condition, dividing the access type of the memory page into exclusive access, and migrating the memory page to the memory module of the target remote host;
    当所述内存页上读取数据的记录达到第二预设条件时,将所述内存页的访问类型划分为读共享,并将所述内存页存储至访问所述内存页频率最高的远程主机上,且将权限设置为只读;When the record of read data on the memory page reaches the second preset condition, the access type of the memory page is divided into read sharing, and the memory page is stored to the remote host with the highest frequency of accessing the memory page , and set the permission to read-only;
    当所述内存页上写入数据的记录达到第三预设条件时,将所述内存页的访问类型划分为写共享,并禁止同步所述内存页至其他远程主机。When the record of the written data on the memory page reaches the third preset condition, the access type of the memory page is divided into write sharing, and synchronization of the memory page to other remote hosts is prohibited.
  8. 根据权利要求4所述的基于虚拟环境的内存共享系统,其特征在于,所述监测模块在监测到所述远程主机访问所述本地主机的内存模块时,生成所述访问日志,所述访问日志记录了访问主机、访问地址和访问类型。 The memory sharing system based on a virtual environment according to claim 4, wherein the monitoring module generates the access log when monitoring that the remote host accesses the memory module of the local host, and the access log The access host, access address and access type are recorded.
  9. 根据权利要求1所述的基于虚拟环境的内存共享系统,其特征在于,所述虚拟机与所述协调器之间的通信以进程的方式实现。 The memory sharing system based on a virtual environment according to claim 1, wherein the communication between the virtual machine and the coordinator is implemented in a process mode.
  10. 一种基于虚拟环境的内存共享方法,其特征在于,其应用于权利要求1-9之一所述的基于虚拟环境的内存共享系统,所述方法包括: A memory sharing method based on a virtual environment, characterized in that it is applied to the memory sharing system based on a virtual environment according to one of claims 1-9, the method comprising:
    虚拟机接收到应用程序发起的访问请求时,将所述访问请求发送至协调器;When the virtual machine receives the access request initiated by the application, it sends the access request to the coordinator;
    所述协调器分析所述访问请求,并判断所述访问请求的访问地址是本地主机还是远程主机;The coordinator analyzes the access request, and determines whether the access address of the access request is a local host or a remote host;
    若是本地主机,则从所述本地主机中获取所述访问请求所请求的数据并反馈至所述应用程序;If it is a local host, obtain the data requested by the access request from the local host and feed it back to the application;
    若是远程主机,则判断内存模块中是否缓存有所述访问请求所请求的数据;If it is a remote host, then determine whether the data requested by the access request is cached in the memory module;
    当内存模块中缓存有所述访问请求所请求的数据时,将所述缓存的所述访问请求所请求的数据反馈至所述应用程序;When the data requested by the access request is cached in the memory module, the cached data requested by the access request is fed back to the application;
    当内存模块中不存在所述访问请求所请求的数据时,所述协调器通过RDMA模块向所述远程主机的协调器发送所述访问请求,并将访问得到的数据反馈至所述应用程序。When the data requested by the access request does not exist in the memory module, the coordinator sends the access request to the coordinator of the remote host through the RDMA module, and feeds back the data obtained by the access to the application program.
PCT/CN2020/139462 2020-12-10 2020-12-25 Virtual-environment-based memory sharing system and method WO2022120992A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011436391.3 2020-12-10
CN202011436391.3A CN112783667B (en) 2020-12-10 2020-12-10 Memory sharing system and method based on virtual environment

Publications (1)

Publication Number Publication Date
WO2022120992A1 true WO2022120992A1 (en) 2022-06-16

Family

ID=75750810

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/139462 WO2022120992A1 (en) 2020-12-10 2020-12-25 Virtual-environment-based memory sharing system and method

Country Status (2)

Country Link
CN (1) CN112783667B (en)
WO (1) WO2022120992A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115098045A (en) * 2022-08-23 2022-09-23 成都止观互娱科技有限公司 Data storage system and network data reading and writing method

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113360293B (en) * 2021-06-02 2023-09-08 奥特酷智能科技(南京)有限公司 Vehicle body electrical network architecture based on remote virtual shared memory mechanism
CN113342465B (en) * 2021-06-18 2022-06-21 上海交通大学 Giant virtual machine based on release consistency memory synchronization
CN113722110B (en) * 2021-11-02 2022-04-15 阿里云计算有限公司 Computer system, memory access method and device
CN114237818B (en) * 2021-12-01 2022-11-11 科东(广州)软件科技有限公司 Method, system, computing device and storage medium for sharing resources among virtual machines
CN116954952B (en) * 2023-09-18 2024-01-09 之江实验室 Self-adaptive hybrid communication method, device, medium and equipment for robot

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120137062A1 (en) * 2010-11-30 2012-05-31 International Business Machines Corporation Leveraging coalesced memory
CN103858111A (en) * 2013-10-08 2014-06-11 华为技术有限公司 Methods, equipment and system for realizing memory sharing in aggregation virtualization
CN104281468A (en) * 2013-07-12 2015-01-14 国际商业机器公司 Method and system for distributed virtual machine image management
CN107623722A (en) * 2017-08-21 2018-01-23 云宏信息科技股份有限公司 A kind of remote data caching method, electronic equipment and storage medium

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7756943B1 (en) * 2006-01-26 2010-07-13 Symantec Operating Corporation Efficient data transfer between computers in a virtual NUMA system using RDMA
CN101163133B (en) * 2006-10-10 2011-06-29 天津中科蓝鲸信息技术有限公司 Communication system and method of implementing resource sharing under multi-machine virtual environment
US9483431B2 (en) * 2013-04-17 2016-11-01 Apeiron Data Systems Method and apparatus for accessing multiple storage devices from multiple hosts without use of remote direct memory access (RDMA)
US9727451B2 (en) * 2014-03-28 2017-08-08 Fortinet, Inc. Virtualization in a multi-host environment
CN105242872B (en) * 2014-06-18 2018-06-12 华中科技大学 A kind of shared memory systems of Virtual cluster
CN107491340B (en) * 2017-07-31 2020-07-14 上海交通大学 Method for realizing huge virtual machine crossing physical machines

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120137062A1 (en) * 2010-11-30 2012-05-31 International Business Machines Corporation Leveraging coalesced memory
CN104281468A (en) * 2013-07-12 2015-01-14 国际商业机器公司 Method and system for distributed virtual machine image management
CN103858111A (en) * 2013-10-08 2014-06-11 华为技术有限公司 Methods, equipment and system for realizing memory sharing in aggregation virtualization
CN107623722A (en) * 2017-08-21 2018-01-23 云宏信息科技股份有限公司 A kind of remote data caching method, electronic equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115098045A (en) * 2022-08-23 2022-09-23 成都止观互娱科技有限公司 Data storage system and network data reading and writing method
CN115098045B (en) * 2022-08-23 2022-11-25 成都止观互娱科技有限公司 Data storage system and network data reading and writing method

Also Published As

Publication number Publication date
CN112783667A (en) 2021-05-11
CN112783667B (en) 2023-08-15

Similar Documents

Publication Publication Date Title
WO2022120992A1 (en) Virtual-environment-based memory sharing system and method
US11841814B2 (en) System with cache-coherent memory and server-linking switch
EP3916566B1 (en) System and method for managing memory resources
CN108268208B (en) RDMA (remote direct memory Access) -based distributed memory file system
CN106874128B (en) Data transmission method and device
US20110004732A1 (en) DMA in Distributed Shared Memory System
US20220188249A1 (en) Memory appliance couplings and operations
JP6514329B2 (en) Memory access method, switch, and multiprocessor system
WO2022001417A1 (en) Data transmission method, processor system, and memory access system
CN103870435A (en) Server and data access method
WO2023093418A1 (en) Data migration method and apparatus, and electronic device
KR20210132348A (en) Computing resource disaggregated collaboration system of interconnected an optical line and, resource disaggregated collaboration method
CN102843435A (en) Access and response method and access and response system of storing medium in cluster system
EP4002139A2 (en) Memory expander, host device using memory expander, and operation method of server system including memory expander
US7089378B2 (en) Shared receive queues
Hines et al. Distributed anemone: Transparent low-latency access to remote memory
CN116028232B (en) Cross-cabinet server memory pooling method, device, equipment, server and medium
US20230071386A1 (en) Application-transparent near-memory processing architecture with memory channel network
CN101441661A (en) System and method for sharing file resource between multiple embedded systems
US20240069755A1 (en) Computer system, memory expansion device and method for use in computer system
WO2024077999A1 (en) Collective communication method and computing cluster
KR101846726B1 (en) System semiconductor MMR chip for enabling communication between processes
CN111638854A (en) Performance optimization method and device for NAS construction and SAN stack block equipment
KR20240007533A (en) Memory disaggregation method,and computing system implementing the method
CN117389938A (en) Data center architecture

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20964921

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20964921

Country of ref document: EP

Kind code of ref document: A1