WO2016188014A1 - 数据存储方法和装置 - Google Patents

数据存储方法和装置 Download PDF

Info

Publication number
WO2016188014A1
WO2016188014A1 PCT/CN2015/091773 CN2015091773W WO2016188014A1 WO 2016188014 A1 WO2016188014 A1 WO 2016188014A1 CN 2015091773 W CN2015091773 W CN 2015091773W WO 2016188014 A1 WO2016188014 A1 WO 2016188014A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
storage
server
operating system
guest operating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/091773
Other languages
English (en)
French (fr)
Inventor
谢广军
王柏生
李志勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to US15/550,851 priority Critical patent/US10469583B2/en
Publication of WO2016188014A1 publication Critical patent/WO2016188014A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0661—Format or protocol conversion arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46—Interconnection of networks
    • H04L12/4633—Interconnection of networks using encapsulation techniques, e.g. tunneling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2212/00—Encapsulation of packets

Definitions

  • the present invention relates to the field of Internet technologies, and in particular, to a data storage method and apparatus.
  • Input Output is critical to the performance of the system. Therefore, moving the storage from the local to the cloud storage on the network requires minimizing the performance loss.
  • the existing technical solutions generally adopt a method of the Internet Small Computer System Interface (ISCSI) to create a Small Computer System Interface (SCSI) device on the local physical machine. Assign this device to the virtual operating system.
  • the SCSI device sends the SCSI command to the ISCSI target through the ISCSI on the bottom layer.
  • the cloud storage server has its own data exchange protocol, so this involves the conversion of the SCSI command and the protocol of the cloud storage server.
  • the ISCSI target can be deployed locally or on a fixed server, or on a cloud storage server.
  • the above solution increases the length of the IO path, increases the load of the central processing unit (CPU), and consumes a large amount of system performance.
  • the object of the present invention is to solve at least one of the technical problems in the related art to some extent.
  • a first object of the present invention is to propose a data storage method. This method can save data on the storage server, shorten the IO path, improve the system's IO throughput, reduce the CPU load, and optimize system performance.
  • a second object of the present invention is to provide a data storage device.
  • a data storage method includes: an analog device receiving data from a guest operating system; and the data being in a user space of the host system according to a protocol used by the storage server Encapsulating; sending the encapsulated data to the storage server for storage.
  • the data storage method of the embodiment of the present invention after the analog device receives the data from the guest operating system, the data is encapsulated in a user space of the host system according to a protocol used by the storage server, and then the encapsulated data is sent to the storage server.
  • Storage which enables data to be stored on the storage server, shortens the IO path, improves system throughput, reduces CPU load, and optimizes system performance.
  • a data storage device configured to an embodiment of the second aspect of the present invention is provided in an analog device, the data storage device comprising: a receiving module, configured to receive data from a guest operating system; and a packaging module, configured to The data received by the receiving module is encapsulated in a user space of the host system according to a protocol used by the storage server, and the sending module is configured to send the encapsulated data to the storage server for storage.
  • the data storage device of the embodiment of the present invention is implemented as an analog device in the virtual machine monitor.
  • the encapsulating module encapsulates the data in a user space of the host system according to a protocol used by the storage server, and then sends the data.
  • the module sends the encapsulated data to the storage server for storage, thereby storing data on the storage server, shortening the IO path, improving the system's IO throughput, reducing the CPU load, and optimizing system performance.
  • a third aspect of the present invention provides an electronic device including: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory,
  • the analog device When executed by the one or more processors, the analog device receives data from the guest operating system; the data is encapsulated in a user space of the host system according to a protocol used by the storage server; Data is sent to the storage server for storage.
  • a fourth aspect of the present invention provides a nonvolatile computer storage medium storing one or more modules, when the one or more modules are executed, proceeding as follows Operation:
  • the analog device receives data from the guest operating system; encapsulates the data in a user space of the host system according to a protocol used by the storage server; and sends the encapsulated data to the storage server for storage.
  • FIG. 1 is a flow chart of an embodiment of a data storage method according to the present invention.
  • FIG. 2 is a schematic diagram of an embodiment of an application scenario of a data storage method according to the present invention.
  • FIG. 3 is a schematic structural diagram of an embodiment of a data storage device according to the present invention.
  • FIG. 1 is a flowchart of an embodiment of a data storage method according to the present invention. As shown in FIG. 1, the data storage method may include:
  • step 101 the analog device receives data from the guest operating system.
  • step 102 the foregoing data is encapsulated in a user space of the host system according to a protocol used by the storage server.
  • Step 103 Send the encapsulated data to the storage server for storage.
  • the foregoing analog device is implemented on the host system based on a virtual machine monitor, such as a Quick Emulator (Qemu), which is a peripheral component interconnect.
  • a virtual machine monitor such as a Quick Emulator (Qemu)
  • Qemu Quick Emulator
  • PCI Peripheral Component Interconnect
  • FIG. 2 is a schematic diagram of an embodiment of an application scenario of the data storage method according to the present invention.
  • the present invention designs and implements an analog device based on the virtual machine monitor Qemu.
  • the analog device is simulated as a block device, and the driver operating system does not need to implement any driving of the analog device, and the block device driver can be directly used;
  • the communication between the guest operating system and the host system, the analog device uses the virtual input and output (VirtIO) standard for data communication; in order to avoid the impact of the storage server upgrade, etc., the analog device and the storage server are decoupled, the present invention
  • a proxy server that implements a storage server is designed on the host system.
  • the method provided by the embodiment shown in FIG. 1 of the present invention can be used, which is very meaningful especially for those user-defined guest operating systems, and the present invention registers the analog device as a PCI interface.
  • the VirtIO block device after the guest operating system discovers that it is a VirtIO block device on the PCI bus, will use a virtual input-output block device (virtio-blk) driver to drive the analog device.
  • virtual input-output block device virtual input-output block device
  • the analog device and the guest operating system share the memory by VirtIO, and the analog device and the guest operating system realize the access of the data by transmitting the physical address of the memory where the data is located. Thereby zero copying of data is achieved.
  • the encapsulated data is sent to the storage server for storage:
  • the device sends the encapsulated data to the storage server for storage through the network communication protocol stack.
  • the analog device after receiving the data, does not pass through the kernel IO stack of the host system, but is encapsulated in the user space of the host system according to the protocol of the storage server, and then directly sent to the remote end through the network communication protocol stack.
  • Storage server In addition, this embodiment optimizes the IO path in Qemu and shortens the IO path.
  • the simulation device sends the encapsulated data to the storage server through the network communication protocol stack for storage: the analog device sends the encapsulated data to the proxy server of the storage server, and the encapsulated data is used by the proxy server. It is sent to the storage server for storage through the network communication protocol stack.
  • the network communication protocol stack may be a Transmission Control Protocol (hereinafter referred to as TCP)/Internet Protocol (IP) protocol stack.
  • TCP Transmission Control Protocol
  • IP Internet Protocol
  • the embodiment of the present invention is not limited thereto, and the network communication protocol is not limited thereto.
  • the stack can also be other network communication protocol stacks as long as network communication between the analog device and the storage server can be implemented.
  • the present invention implements a proxy server of the storage server between the simulation device and the remote storage server, and separates the business logic into the proxy server. .
  • the proxy server and the analog device in Qemu only perform simple data communication, and are not affected by the details of the specific communication protocol. No matter whether it is a protocol upgrade or a defect, Qemu does not need to be changed.
  • the present invention separates the control plane and the data channel, and uses a separate dedicated thread to be responsible for the data channel, thereby avoiding control command occupation.
  • the proxy server adopts an extensible architecture, and each of the analog devices corresponds to one process, so that there are multiple guest operating systems or multiple simulations on the same host. In the case of devices, you can take advantage of multiple CPUs in the host system to improve parallelism and improve overall IO throughput.
  • the proxy server may need to perform the necessary upgrades based on the upgrade of the storage protocol, or an unexpected restart may occur due to a defect. But in either case, if not processed, the performance reflected in the guest operating system is an IO error.
  • the present invention implements a fault tolerance mechanism, and the simulation device senses the state of the proxy server in real time. Whether the proxy server is upgraded or accidentally, after the proxy server is restarted, the analog device will retransmit the outstanding IO request; That is to say, after the above proxy server is restarted, the analog device retransmits the data that was not completed before the proxy server restarted.
  • the present invention designs an IO scheduling algorithm for the guest operating system and the analog device, except that the same block is merged.
  • the request can be sent to the storage server in different blocks, so that the backend can be dispatched to different storage servers to improve the overall concurrency, thereby improving the overall IO throughput.
  • This optimization is optional because it involves changes to the guest operating system. If the guest operating system is not modified, the implementation of other portions of the data storage method provided by the embodiment of FIG. 1 of the present invention will not be affected.
  • the analog device after receiving the data from the guest operating system, the analog device encapsulates the data in a user space of the host system according to a protocol used by the storage server, and then sends the encapsulated data to the storage server for storage. Therefore, the data can be stored on the storage server, the IO path is shortened, the system IO throughput is improved, the CPU load is reduced, and the system performance is optimized.
  • FIG. 3 is a schematic structural diagram of an embodiment of the data storage device of the present invention.
  • the data storage device in the embodiment is implemented as an analog device in the virtual machine monitor, and the process of the embodiment shown in FIG. 1 of the present invention can be implemented.
  • the data storage device may include: a receiving module 31, a packaging module 32, and a sending module 33;
  • the receiving module 31 is configured to receive data from a guest operating system.
  • the encapsulating module 32 is configured to encapsulate the data received by the receiving module 31 in a user space of the host system according to a protocol used by the storage server;
  • the sending module 33 is configured to send the encapsulated data of the encapsulating module 32 to the storage server for storage.
  • the foregoing simulation device is implemented on a host system based on a virtual machine monitor, such as Qemu, and the analog device is a block device of a PCI interface; the client operating system uses a block device driver to drive the analog device, The guest operating system runs on top of the above virtual machine monitor.
  • a virtual machine monitor such as Qemu
  • the analog device is a block device of a PCI interface
  • the client operating system uses a block device driver to drive the analog device
  • the guest operating system runs on top of the above virtual machine monitor.
  • the present invention designs and implements an analog device based on the virtual machine monitor Qemu.
  • the analog device is simulated as a block device, and the guest operating system does not need to implement any driver for the analog device, and the block device driver can be used directly; Efficient communication between the guest operating system and the host system, the analog device uses the VirtIO standard for data communication; in order to avoid the impact of the storage server upgrade, the analog device and the storage server are decoupled, the present invention is on the host system Designed to implement a proxy server for the storage server.
  • the device provided by the embodiment can be used. This is especially meaningful for those user-defined guest operating systems.
  • the present invention registers the analog device as a VirtIO block device of the PCI interface, and the client After the operating system finds that it is a VirtIO block device on the PCI bus, it will use the virtio-blk driver to drive the analog device.
  • the analog device and the guest operating system share the memory through the VirtIO mode, and the analog device and the guest operating system realize the access of the data by transmitting the physical address of the memory where the data is located, thereby realizing zero data. copy.
  • the sending module 33 is specifically configured to send the data encapsulated by the encapsulating module 32 to the storage server for storage through a network communication protocol stack.
  • the receiving module 31 does not pass through the kernel IO stack of the host system, but is encapsulated by the encapsulating module 32 in the user space of the host system according to the protocol of the storage server, and the sending module 33 passes through the network.
  • the communication protocol stack is sent directly to the remote storage server.
  • this embodiment optimizes the IO path in Qemu and shortens the IO path.
  • the sending module 33 is configured to send the encapsulated data of the encapsulating module 32 to the proxy server of the storage server, and the proxy server sends the encapsulated data to the storage server for storage through the network communication protocol stack.
  • the network communication protocol stack may be a TCP/IP protocol stack. The embodiment of the present invention is not limited thereto.
  • the network communication protocol stack may also be another network communication protocol stack, as long as the simulation device and the storage server can be implemented. Network communication is fine.
  • the present invention implements a proxy server of the storage server between the simulation device and the remote storage server, and separates the business logic into the proxy server. .
  • the proxy server and the analog device in Qemu only perform simple data communication, and are not affected by the details of the specific communication protocol. No matter whether it is a protocol upgrade or a defect, Qemu does not need to be changed.
  • the present invention separates the control plane and the data channel, and uses a separate dedicated thread to be responsible for the data channel, thereby avoiding control command occupation.
  • the proxy server adopts an extensible architecture, and each of the simulated devices corresponds to one process, so that for a case where there are multiple guest operating systems or multiple analog devices on the same host system, Make full use of multiple CPUs in the host system to improve parallelism and improve overall IO throughput.
  • the sending module 33 is further configured to retransmit the data that is not completed before the proxy server restarts after the proxy server is restarted. Because the proxy server may need to perform the necessary upgrades based on the upgrade of the storage protocol, or an unexpected restart may occur due to a defect. But in either case, if not processed, the performance reflected in the guest operating system is an IO error. In order to improve reliability, the above data storage device is designed to implement a fault tolerance mechanism to sense the state of the proxy server in real time. Whether the proxy server is upgraded or accidental, after the proxy server is restarted, the sending module 33 will retransmit the unfinished IO operations.
  • the encapsulating module 32 encapsulates the data in the user space of the host system according to the protocol used by the storage server, and then the sending module 33 sends the encapsulated data.
  • the above storage server is stored, so that the data can be stored on the storage server, the IO path is shortened, the system IO throughput is improved, the CPU load is reduced, and the system performance is optimized.
  • An embodiment of the present invention further provides an electronic device, including: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory when the one or more
  • the processor performs the following operations: the analog device receives data from the guest operating system; the data is encapsulated in a user space of the host system according to a protocol used by the storage server; and the encapsulated data is sent to the storage server. storage.
  • An embodiment of the present invention further provides a non-volatile computer storage medium, where the computer storage medium stores one or more modules, and when the one or more modules are executed, performing the following operations: the analog device receives the operation from the client System data; the data is entered in the user space of the host system according to the protocol used by the storage server Row encapsulation; sending the encapsulated data to the storage server for storage.
  • portions of the invention may be implemented in hardware, software, firmware or a combination thereof.
  • multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals.
  • Discrete logic circuit, ASIC with suitable combination logic gate Programmable Gate Array (PGA), Field Programmable Gate Array (FPGA).
  • each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.
  • the above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer And Data Communications (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Stored Programmes (AREA)

Abstract

一种数据存储方法和装置，上述数据存储方法包括：模拟设备接收来自客户操作系统的数据（101）；将所述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装（102）；将封装后的数据发送给所述存储服务器进行存储（103）。该方法可以实现将数据存储在存储服务器上，缩短了IO路径，提高了系统的IO吞吐，减轻了CPU的负载，优化了系统性能。

Description

数据存储方法和装置
相关申请的交叉引用
本申请要求北京百度网讯科技有限公司于2015年5月27日提交的、发明名称为“数据存储方法和装置”的、中国专利申请号“201510278937.X”的优先权。
技术领域
本发明涉及互联网技术领域，尤其涉及一种数据存储方法和装置。
背景技术
对于一个虚拟操作系统来说，为了保证其数据可靠性，需要将数据保存在更可靠的云存储服务器上。另外，将存储保存在另外一个位置，也是虚拟操作系统迁移的充分条件。
但是输入输出(Input Output；以下简称：IO)对系统的性能是至关重要的，因此，将存储从本地移动到网络上的云存储，需要尽量减少由此带来的性能损失。
现有的技术方案一般都是采取互联网小型计算机接口(Internet Small Computer System Interface；以下简称：ISCSI)的方式，在本地物理机上创建一个小型计算机接口(Small Computer System Interface；以下简称：SCSI)设备，将这个设备分配给虚拟操作系统。SCSI设备在底层通过ISCSI将SCSI命令发送给ISCSI目标器(ISCSI target)，一般云存储服务器都有自己的数据交换协议，所以这其中还要涉及SCSI命令和云存储服务器的协议的转换。ISCSI target可以部署在本地或者一个固定的服务器上，或者云存储服务器上。但是上述方案增加了IO路径的长度，增加了中央处理器(Central Processing Unit；以下简称：CPU)的负载，对系统性能的消耗较大。
发明内容
本发明的目的旨在至少在一定程度上解决相关技术中的技术问题之一。
为此，本发明的第一个目的在于提出一种数据存储方法。该方法可以实现将数据存储在存储服务器上，缩短了IO路径，提高了系统的IO吞吐，减轻了CPU的负载，优化了系统性能。
本发明的第二个目的在于提出一种数据存储装置。
为了实现上述目的，本发明第一方面实施例的数据存储方法，包括：模拟设备接收来自客户操作系统的数据；将所述数据在宿主系统的用户空间中按照存储服务器使用的协议 进行封装；将封装后的数据发送给所述存储服务器进行存储。
本发明实施例的数据存储方法，模拟设备接收来自客户操作系统的数据之后，将上述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装，然后将封装后的数据发送给上述存储服务器进行存储，从而可以实现将数据存储在存储服务器上，缩短了IO路径，提高了系统的吞吐，减轻了CPU的负载，优化了系统性能。
为了实现上述目的，本发明第二方面实施例的数据存储装置，设置在模拟设备中，所述数据存储装置包括：接收模块，用于接收来自客户操作系统的数据；封装模块，用于将所述接收模块接收的数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装；发送模块，用于将所述封装模块封装后的数据发送给所述存储服务器进行存储。
本发明实施例的数据存储装置，实现为虚拟机监控器中的一个模拟设备，接收模块接收数据之后，封装模块将上述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装，然后发送模块将封装后的数据发送给上述存储服务器进行存储，从而可以实现将数据存储在存储服务器上，缩短了IO路径，提高了系统的IO吞吐，减轻了CPU的负载，优化了系统性能。
为了实现上述目的，本发明第三方面实施例提供了一种电子设备，包括：一个或者多个处理器；存储器；一个或者多个程序，所述一个或者多个程序存储在所述存储器中，当被所述一个或者多个处理器执行时进行如下操作：模拟设备接收来自客户操作系统的数据；将所述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装；将封装后的数据发送给所述存储服务器进行存储。
为了实现上述目的，本发明第四方面实施例提供了一种非易失性计算机存储介质，所述计算机存储介质存储有一个或者多个模块，当所述一个或者多个模块被执行时进行如下操作：模拟设备接收来自客户操作系统的数据；将所述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装；将封装后的数据发送给所述存储服务器进行存储。
本发明附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本发明的实践了解到。
附图说明
本发明上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为本发明数据存储方法一个实施例的流程图；
图2为本发明数据存储方法的应用场景一个实施例的示意图；
图3为本发明数据存储装置一个实施例的结构示意图。
具体实施方式
下面详细描述本发明的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本发明，而不能理解为对本发明的限制。相反，本发明的实施例包括落入所附加权利要求书的精神和内涵范围内的所有变化、修改和等同物。
图1为本发明数据存储方法一个实施例的流程图，如图1所示，该数据存储方法可以包括：
步骤101，模拟设备接收来自客户操作系统的数据。
步骤102，将上述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装。
步骤103，将封装后的数据发送给上述存储服务器进行存储。
本实施例中，上述模拟设备是在上述宿主系统上基于虚拟机监控器(Virtual Machine Monitor)，例如快速模拟器(Quick Emulator；以下简称：Qemu)实现的，上述模拟设备为外设部件互连标准(Peripheral Component Interconnect；以下简称：PCI)接口的块设备；上述客户操作系统使用块设备驱动来驱动上述模拟设备，上述客户操作系统运行于虚拟机监控器之上。
本发明图1所示实施例提供的数据存储方法可以应用在图2所示的架构中，图2为本发明数据存储方法的应用场景一个实施例的示意图。
参见图2，本发明基于虚拟机监控器Qemu，设计并实现了一个模拟设备。为了避免修改客户操作系统，减少对客户操作系统的影响，该模拟设备被模拟为一个块设备，客户操作系统中无需实现任何对该模拟设备的驱动，直接使用块设备驱动即可；为了更高效的在客户操作系统和宿主系统之间进行通信，模拟设备采用虚拟输入输出(VirtIO)标准实现数据通信；为了避免存储服务器升级等带来的影响，模拟设备与存储服务器进行了解耦，本发明在宿主系统上设计实现了一个存储服务器的代理服务器。
为了使客户操作系统不经修改就可以使用本发明图1所示实施例提供的方法，这一点尤其是对于那些用户自定义的客户操作系统非常有意义，本发明将模拟设备注册为PCI接口的VirtIO块设备,客户操作系统发现是PCI总线上的VirtIO块设备后，将使用虚拟输入输出块设备(virtio-blk)驱动来驱动该模拟设备。
本发明图1所示实施例中，上述模拟设备与上述客户操作系统之间通过VirtIO的方式共享内存，模拟设备与客户操作系统之间通过传递上述数据所在内存的物理地址实现上述数据的访问，从而实现了数据的零复制。
本发明图1所示实施例中，将封装后的数据发送给存储服务器进行存储可以为：模拟 设备将封装后的数据通过网络通信协议栈发送给上述存储服务器进行存储。
本发明中，模拟设备接收到数据之后，不再经过宿主系统的内核IO栈，而是在宿主系统的用户空间中按照存储服务器的协议进行封装后，通过网络通信协议栈直接发送给远端的存储服务器。另外，本实施例对Qemu中的IO路径进行了优化，缩短了IO路径。
具体地，模拟设备将封装后的数据通过网络通信协议栈发送给上述存储服务器进行存储可以为：模拟设备将上述封装后的数据发送给存储服务器的代理服务器，由上述代理服务器将封装后的数据通过网络通信协议栈发送给存储服务器进行存储。其中，上述网络通信协议栈可以为传输控制协议(Transmission Control Protocol；以下简称：TCP)/因特网协议(Internet Protocol；以下简称：IP)协议栈，当然本发明实施例并不仅限于此，网络通信协议栈还可以为其他的网络通信协议栈，只要可以实现模拟设备与存储服务器之间的网络通信即可。
在生产环境上，频繁升级Qemu的代价和风险都是比较大的，因此，本发明在模拟设备和远端存储服务器之间实现了一个存储服务器的代理服务器，将业务逻辑分离到代理服务器中来。而代理服务器与Qemu中的模拟设备之间只是进行简单的数据通信，不受具体通信协议细节变更的影响，无论是协议升级，或者是修复缺陷，Qemu均无需变更。
为了提高IO吞吐，本发明将控制平面和数据通道分开实现，使用一个独立的专用的线程负责数据通道，从而避免控制命令占用。本发明图1所示实施例中，上述代理服务器采用可扩展的架构，上述模拟设备中的每个模拟设备对应一个进程，这样，对于同一个宿主机上有多个客户操作系统或者多个模拟设备的情形，可以充分利用宿主系统的多颗CPU，提高并行性，提高整体的IO吞吐。
因为代理服务器可能需要根据存储协议的升级进行必要的升级，或者因为缺陷可能发生意外的重启。但是无论哪种情况，如果不加处理，那么反映到客户操作系统中的表现就是IO错误。为了提高可靠性，本发明设计实现了容错机制，模拟设备实时感知代理服务器的状态，无论代理服务器是因为升级还是因为意外，在代理服务器重启之后，模拟设备将重传尚未完成的IO请求；也就是说，在上述代理服务器重启之后，模拟设备重传在上述代理服务器重启之前未完成传输的数据。
因为对于后端存储服务器，不同的块可能存储在不同的机器上，因此，为了提高总体的并行性，本发明为客户操作系统和模拟设备设计了一种IO调度算法，除了合并同一个块中的相邻请求外，也可以向存储服务器发送位于不同块的请求，这样后端可以调度到不同存储服务器，提高整体的并发，从而提高整体的IO吞吐。这个优化因为涉及到客户操作系统的改动，是可选的，如果不改动客户操作系统，不会影响本发明图1所示实施例提供的数据存储方法其他部分的实现。
上述数据存储方法中，模拟设备接收来自客户操作系统的数据之后，将上述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装，然后将封装后的数据发送给上述存储服务器进行存储，从而可以实现将数据存储在存储服务器上，缩短了IO路径，提高了系统的IO吞吐，减轻了CPU的负载，优化了系统性能。
图3为本发明数据存储装置一个实施例的结构示意图，本实施例中的数据存储装置实现为虚拟机监控器中的一个模拟设备，可以实现本发明图1所示实施例的流程。如图3所示，上述数据存储装置可以包括：接收模块31、封装模块32和发送模块33；
其中，接收模块31，用于接收来自客户操作系统的数据；
封装模块32，用于将接收模块31接收的数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装；
发送模块33，用于将封装模块32封装后的数据发送给上述存储服务器进行存储。
本实施例中，上述模拟设备是在宿主系统上基于虚拟机监控器，例如：Qemu实现的，上述模拟设备为PCI接口的块设备；上述客户操作系统使用块设备驱动来驱动上述模拟设备，上述客户操作系统运行于上述虚拟机监控器之上。
参见图2，本发明基于虚拟机监控器Qemu，设计并实现了一个模拟设备。为了避免客户操作系统的改动，减少对客户操作系统的影响，该模拟设备被模拟为一个块设备，客户操作系统中无需实现任何对该模拟设备的驱动，直接使用块设备驱动即可；为了更高效的在客户操作系统和宿主系统之间进行通信，模拟设备采用VirtIO标准实现数据通信；为了避免存储服务器升级等带来的影响，模拟设备与存储服务器进行了解耦，本发明在宿主系统上设计实现了一个存储服务器的代理服务器。
为了使客户操作系统不经修改就可以使用本实施例提供的装置，这一点尤其是对于那些用户自定义的客户操作系统非常有意义，本发明将模拟设备注册为PCI接口的VirtIO块设备,客户操作系统发现是PCI总线上的VirtIO块设备后，将使用virtio-blk驱动来驱动该模拟设备。
本实施例中，上述模拟设备与客户操作系统之间通过VirtIO的方式共享内存，模拟设备与客户操作系统之间通过传递上述数据所在内存的物理地址实现上述数据的访问，从而实现了数据的零复制。
上述数据存储装置中，发送模块33，具体用于将封装模块32封装后的数据通过网络通信协议栈发送给上述存储服务器进行存储。本实施例中，接收模块31接收到数据之后，不再经过宿主系统的内核IO栈，而是由封装模块32在宿主系统的用户空间中按照存储服务器的协议进行封装后，发送模块33通过网络通信协议栈直接发送给远端的存储服务器。另外，本实施例对Qemu中的IO路径进行了优化，缩短了IO路径。
更具体地，发送模块33，用于将封装模块32封装后的数据发送给上述存储服务器的代理服务器，由代理服务器将上述封装后的数据通过网络通信协议栈发送给存储服务器进行存储。其中，上述网络通信协议栈可以为TCP/IP协议栈，当然本发明实施例并不仅限于此，网络通信协议栈还可以为其他的网络通信协议栈，只要可以实现模拟设备与存储服务器之间的网络通信即可。
在生产环境上，频繁升级Qemu的代价和风险都是比较大的，因此，本发明在模拟设备和远端存储服务器之间实现了一个存储服务器的代理服务器，将业务逻辑分离到代理服务器中来。而代理服务器与Qemu中的模拟设备之间只是进行简单的数据通信，不受具体通信协议细节变更的影响，无论是协议升级，或者是修复缺陷，Qemu均无需变更。
为了提高IO吞吐，本发明将控制平面和数据通道分开实现，使用一个独立的专用的线程负责数据通道，从而避免控制命令占用。本实施例中，上述代理服务器采用可扩展的架构，上述模拟设备中的每个模拟设备对应一个进程，这样，对于同一个宿主系统上有多个客户操作系统或者多个模拟设备的情形，可以充分利用宿主系统的多颗CPU，提高并行性，提高整体的IO吞吐。
上述数据存储装置中，发送模块33，还用于在代理服务器重启之后，重传在代理服务器重启之前未完成传输的数据。因为代理服务器可能需要根据存储协议的升级进行必要的升级，或者因为缺陷可能发生意外的重启。但是无论哪种情况，如果不加处理，那么反映到客户操作系统中的表现就是IO错误。为了提高可靠性，上述数据存储装置设计实现了容错机制，实时感知代理服务器的状态，无论代理服务器是因为升级还是因为意外，在代理服务器重启之后，发送模块33将重传尚未完成的IO操作。
上述数据存储装置中，接收模块31接收来自客户操作系统的数据之后，封装模块32将上述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装，然后发送模块33将封装后的数据发送给上述存储服务器进行存储，从而可以实现将数据存储在存储服务器上，缩短了IO路径，提高了系统的IO吞吐，减轻了CPU的负载，优化了系统性能。
本发明实施例还提供一种电子设备，包括：一个或者多个处理器；存储器；一个或者多个程序，所述一个或者多个程序存储在所述存储器中，当被所述一个或者多个处理器执行时进行如下操作：模拟设备接收来自客户操作系统的数据；将所述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装；将封装后的数据发送给所述存储服务器进行存储。
本发明实施例还提供一种非易失性计算机存储介质，所述计算机存储介质存储有一个或者多个模块，当所述一个或者多个模块被执行时进行如下操作：模拟设备接收来自客户操作系统的数据；将所述数据在宿主系统的用户空间中按照存储服务器使用的协议进 行封装；将封装后的数据发送给所述存储服务器进行存储。
需要说明的是，在本发明的描述中，术语“第一”、“第二”等仅用于描述目的，而不能理解为指示或暗示相对重要性。此外，在本发明的描述中，除非另有说明，“多个”的含义是两个或两个以上。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本发明的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本发明的实施例所属技术领域的技术人员所理解。
应当理解，本发明的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(Programmable Gate Array；以下简称：PGA)，现场可编程门阵列(Field Programmable Gate Array；以下简称：FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。
此外，本发明各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
尽管上面已经示出和描述了本发明的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本发明的限制，本领域的普通技术人员在本发明的范围内可以对上述实施例 进行变化、修改、替换和变型。

Claims (16)

  1. 一种数据存储方法，其特征在于，包括：
    模拟设备接收来自客户操作系统的数据；
    将所述数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装；
    将封装后的数据发送给所述存储服务器进行存储。
  2. 根据权利要求1所述的方法，其特征在于，所述模拟设备是在所述宿主系统上基于虚拟机监控器实现的，所述模拟设备为外设部件互连标准PCI接口的块设备；所述客户操作系统使用块设备驱动来驱动所述模拟设备，所述客户操作系统运行于所述虚拟机监控器之上。
  3. 根据权利要求2所述的方法，其特征在于，所述模拟设备与所述客户操作系统之间通过共享内存的方式传递数据；所述模拟设备与所述客户操作系统之间通过传递所述数据所在内存的物理地址实现所述数据的访问。
  4. 根据权利要求1-3任意一项所述的方法，其特征在于，所述将封装后的数据发送给所述存储服务器进行存储包括：
    所述模拟设备将所述封装后的数据通过网络通信协议栈发送给所述存储服务器进行存储。
  5. 根据权利要求4所述的方法，其特征在于，所述模拟设备将所述封装后的数据通过网络通信协议栈发送给所述存储服务器进行存储包括：
    所述模拟设备将所述封装后的数据发送给所述存储服务器的代理服务器，由所述代理服务器将所述封装后的数据通过网络通信协议栈发送给所述存储服务器进行存储。
  6. 根据权利要求5所述的方法，其特征在于，所述代理服务器采用可扩展的架构，每个模拟设备对应一个进程。
  7. 根据权利要求5或6所述的方法，其特征在于，还包括：
    在所述代理服务器重启之后，所述模拟设备重传在所述代理服务器重启之前未完成传输的数据。
  8. 一种数据存储装置，其特征在于，设置在模拟设备中，所述数据存储装置包括：
    接收模块，用于接收来自客户操作系统的数据；
    封装模块，用于将所述接收模块接收的数据在宿主系统的用户空间中按照存储服务器使用的协议进行封装；
    发送模块，用于将所述封装模块封装后的数据发送给所述存储服务器进行存储。
  9. 根据权利要求8所述的装置，其特征在于，所述模拟设备是在所述宿主系统上基于 虚拟机监控器实现的，所述模拟设备为外设部件互连标准PCI接口的块设备；所述客户操作系统使用块设备驱动来驱动所述模拟设备，所述客户操作系统运行于所述虚拟机监控器之上。
  10. 根据权利要求9所述的装置，其特征在于，所述模拟设备与所述客户操作系统之间通过共享内存的方式传递数据；所述模拟设备与所述客户操作系统之间通过传递所述数据所在内存的物理地址实现所述数据的访问。
  11. 根据权利要求8-10任意一项所述的装置，其特征在于，
    所述发送模块，具体用于将所述封装模块封装后的数据通过网络通信协议栈发送给所述存储服务器进行存储。
  12. 根据权利要求11所述的装置，其特征在于，
    所述发送模块，具体用于将所述封装模块封装后的数据发送给所述存储服务器的代理服务器，由所述代理服务器将所述封装后的数据通过网络通信协议栈发送给所述存储服务器进行存储。
  13. 根据权利要求12所述的装置，其特征在于，所述代理服务器采用可扩展的架构，每个模拟设备对应一个进程。
  14. 根据权利要求12或13所述的装置，其特征在于，
    所述发送模块，还用于在所述代理服务器重启之后，重传在所述代理服务器重启之前未完成传输的数据。
  15. 一种电子设备，其特征在于，包括：
    一个或者多个处理器；
    存储器；
    一个或者多个程序，所述一个或者多个程序存储在所述存储器中，当被所述一个或者多个处理器执行时：
    执行如权利要求1-7任一项所述的方法。
  16. 一种非易失性计算机存储介质，其特征在于，所述计算机存储介质存储有一个或者多个模块，当所述一个或者多个模块被执行时：
    执行如权利要求1-7任一项所述的方法。
PCT/CN2015/091773 2015-05-27 2015-10-12 数据存储方法和装置 Ceased WO2016188014A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/550,851 US10469583B2 (en) 2015-05-27 2015-10-12 Data storage method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510278937.X 2015-05-27
CN201510278937.XA CN104881254B (zh) 2015-05-27 2015-05-27 数据存储方法和装置

Publications (1)

Publication Number Publication Date
WO2016188014A1 true WO2016188014A1 (zh) 2016-12-01

Family

ID=53948764

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/091773 Ceased WO2016188014A1 (zh) 2015-05-27 2015-10-12 数据存储方法和装置

Country Status (3)

Country Link
US (1) US10469583B2 (zh)
CN (1) CN104881254B (zh)
WO (1) WO2016188014A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104881254B (zh) 2015-05-27 2018-09-07 北京百度网讯科技有限公司 数据存储方法和装置
CN110351156B (zh) * 2018-04-03 2021-10-01 大唐移动通信设备有限公司 一种测试方法和装置
CN113778611B (zh) * 2021-07-06 2024-08-20 锐捷网络(苏州)有限公司 数据传输方法、装置、系统及存储介质
CN121349599B (zh) * 2025-12-17 2026-03-10 湖北芯擎科技有限公司 Io处理方法、设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110307886A1 (en) * 2010-06-11 2011-12-15 Oracle International Corporation Method and system for migrating the state of a virtual cluster
CN104092759A (zh) * 2014-07-14 2014-10-08 南京斯坦德云科技股份有限公司 一种分布式云存储服务器集群系统及存储方法
CN104636076A (zh) * 2013-11-15 2015-05-20 中国电信股份有限公司 一种用于云存储的分布式块设备驱动方法和系统
CN104881254A (zh) * 2015-05-27 2015-09-02 北京百度网讯科技有限公司 数据存储方法和装置

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002023364A1 (en) * 2000-09-15 2002-03-21 Wonderware Corporation An industrial process control data access server supporting multiple client data exchange protocols
US7555558B1 (en) * 2003-08-15 2009-06-30 Michael Frederick Kenrich Method and system for fault-tolerant transfer of files across a network
US7500071B2 (en) * 2005-08-31 2009-03-03 International Business Machines Corporation Method for out of user space I/O with server authentication
CN102447723B (zh) * 2010-10-12 2015-09-09 运软网络科技(上海)有限公司 客户端虚拟化架构
US9552215B2 (en) * 2011-03-08 2017-01-24 Rackspace Us, Inc. Method and system for transferring a virtual machine
US9110963B2 (en) * 2012-04-10 2015-08-18 Dell Inc Transparent adaptive file transform
US8904081B1 (en) * 2012-05-08 2014-12-02 Vmware, Inc. Composing a virtual disk using application delta disk images
US9063767B2 (en) * 2012-12-20 2015-06-23 Hewlett-Packard Development Company, L.P. Configuration space with pseudo device identifier
US9405566B2 (en) * 2013-05-24 2016-08-02 Dell Products L.P. Access to storage resources using a virtual storage appliance
CN103927489B (zh) * 2014-04-22 2017-01-18 陈幼雷 一种数据可信存储的系统及其方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110307886A1 (en) * 2010-06-11 2011-12-15 Oracle International Corporation Method and system for migrating the state of a virtual cluster
CN104636076A (zh) * 2013-11-15 2015-05-20 中国电信股份有限公司 一种用于云存储的分布式块设备驱动方法和系统
CN104092759A (zh) * 2014-07-14 2014-10-08 南京斯坦德云科技股份有限公司 一种分布式云存储服务器集群系统及存储方法
CN104881254A (zh) * 2015-05-27 2015-09-02 北京百度网讯科技有限公司 数据存储方法和装置

Also Published As

Publication number Publication date
US20180020053A1 (en) 2018-01-18
CN104881254B (zh) 2018-09-07
US10469583B2 (en) 2019-11-05
CN104881254A (zh) 2015-09-02

Similar Documents

Publication Publication Date Title
US11640269B2 (en) Solid-state drive with initiator mode
TWI647573B (zh) 用於支援對經由nvme控制器、通過網路訪問遠端存放設備的虛擬機器的遷移的系統和方法
US9819739B2 (en) Systems and methods for supporting hot plugging of remote storage devices accessed over a network via NVME controller
US9501245B2 (en) Systems and methods for NVMe controller virtualization to support multiple virtual machines running on a host
EP3660686B1 (en) Method and device for transmitting data processing request
US20160077740A1 (en) Systems and methods for enabling local caching for remote storage devices over a network via nvme controller
KR20180117641A (ko) 원격으로 액세스되는 데이터의 효율적인 라이브-이송
US8447894B2 (en) Upgrading an elastic computing cloud system
WO2012094303A1 (en) Upgrading an elastic computing cloud system
US12495024B2 (en) Computing cluster bring-up on any one of a plurality of different public cloud infrastructures
CN104881254B (zh) 数据存储方法和装置
US12461792B2 (en) Computing cluster bring-up on public cloud infrastructure using expressed intents
US20180039518A1 (en) Arbitrating access to a resource that is shared by multiple processors
US11409624B2 (en) Exposing an independent hardware management and monitoring (IHMM) device of a host system to guests thereon
US11321254B2 (en) Computing system for transmitting completion early between serially connected electronic devices
US12536036B2 (en) Programmable data movement processor for collective communication offload
US20210234787A1 (en) Automatic retransmission capability in hypervisor
CN118245071A (zh) 一种系统安装方法、装置、设备及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15893091

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15550851

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15893091

Country of ref document: EP

Kind code of ref document: A1