WO2022267427A1

WO2022267427A1 - Virtual machine migration method and system, and electronic device

Info

Publication number: WO2022267427A1
Application number: PCT/CN2021/143173
Authority: WO
Inventors: 孟亮; 何继文; 潘宗辉; 金博玉; 黄学军; 刘蜀东; 穆森; 朱玥; 刘辉军; 邱桂苹; 杨硕; 俞坚华; 董全
Original assignee: 航天云网科技发展有限责任公司
Priority date: 2021-06-25
Filing date: 2021-12-30
Publication date: 2022-12-29
Also published as: CN113342471A

Abstract

A virtual machine migration method and system and an electronic device, which relate to the technical field of virtual machines. The method comprises: first, establishing a connection with a receiving end and sending a first message to the receiving end, wherein the first message comprises attribute information of a file to be copied (S101); then, determining a hole region of the file and location information thereof, the hole region being a storage block in the file that continuously stores a binary value of zero (S102); next, determining a non-hole region of the file and an offset thereof according to the hole region and the location information thereof; and once the transmission of the non-hole region and the offset thereof is complete, sending to the receiving end a second message and verification information for verifying the integrity of the file. In the described method, offset data of a file hole block may be used in a message to replace the transmission of the file hole block, thereby increasing the migration speed. Moreover, the occupancy rate of a page cache is reduced by means of a write-through disk method, which increases the read-write efficiency, and helps to improve the transfer rate of very large disk files.

Description

Virtual machine migration method, system and electronic device

Cross References to Related Applications

This application claims priority to a Chinese patent application with application number 202110715998.3 and titled "Virtual Machine Migration Method, System, and Electronic Device" filed with the State Intellectual Property Office of China on June 25, 2021, the entire contents of which are hereby incorporated by reference In this application.

technical field

The present application relates to the technical field of virtual machines, in particular to a virtual machine migration method, system and electronic equipment.

Background technique

During the migration process of an existing virtual machine, a system disk image file (referred to as a disk file) will first be created in the source host machine during static migration, and then the disk file will be copied to the destination host machine. Specifically, the copy process on the source host machine is: (1) open the disk file to be migrated; (2) read the file content to the cache; (3) send the read file content to the new host through TCP/UCP (4) cycle (2)-(3) until the file is sent. The copy process on the destination host is: (1) create a new file and open it; (2) read the disk content transmitted by the peer through TCP/UDP; (3) write the read content into (1) Created file; (4) cycle (2)-(3) until the file is received.

Because the disk files generated by the running of the virtual machine are different from ordinary files, there are usually more "empty" data. "Hole" data is the part of the file that continuously stores binary 0. Generally speaking, the hole in the virtual machine occupies about one-third of the total disk space. Disks of different formats have different voids as they are used. Take disks in qcow2 and raw formats as an example. Generally speaking, the ratio of holes in disks in raw format will decrease with use; while the ratio of holes in disks in qcow2 format will increase with use. During the virtual machine migration process, the transmission of these empty data will waste bandwidth and reduce transmission efficiency.

At the same time, the biggest problem of using cached I/O during virtual machine migration is the consumption of page cache. During the virtual machine migration process, a large number of frequent file read and write operations are involved, and a large number of data copy operations are performed between the address space of the application program and the page cache, and between the page cache and the disk. These data copy operations bring The CPU and memory overhead is very large. If the migrated disk file is large, or if there are multiple virtual machine files to be migrated at the same time, the migration efficiency will decrease.

To sum up, the current virtual machine migration solution has the problem of wasting transmission bandwidth caused by "empty" data; and the page cache is occupied during the process of reading and writing files, resulting in low read and write efficiency, and is not conducive to the transmission of large disk files .

Contents of the invention

In view of this, the purpose of this application is to provide a virtual machine migration method, system and electronic equipment, in which the offset data of the file empty block can be used in the message to replace the transmission of the file empty block, thereby increasing the migration speed; and By writing directly to the disk, the occupancy rate of the page cache is reduced, the efficiency of reading and writing is improved, and the transmission rate of very large disk files is improved.

In the first aspect, the embodiment of the present application provides a virtual machine migration method, which is applied to the sending end, including:

Establish a connection with the receiving end, and send the first message to the receiving end; wherein, the first message includes attribute information of the file to be copied;

Determine the empty area and its location information of the file to be copied; wherein, the empty area is a storage block that continuously stores binary values of 0 in the file to be copied;

According to the hole area and its location information, determine the non-hole area and its offset of the file to be copied, and sequentially transmit the non-hole area and its offset from the address space of the sending end to the disk of the receiving end;

After the non-hole area and its offset are transmitted, the second message is sent to the receiving end; wherein, the second message includes verification information for verifying the integrity of the file to be copied.

In some embodiments, the step of establishing a connection with the receiving end and sending the first message to the receiving end includes:

The sending end establishes a long connection with the receiving end, and initializes the request message and the header message message in the first message; wherein, the request message is used to notify the receiving end to perform the copy process; the header message message is used to inform the receiving end Attribute information of the file to be copied;

Send a request message to the receiving end;

After receiving the response message of the request message, send the header message message to the receiving end.

In some embodiments, the step of determining the empty area of the file to be copied and its location information includes:

Scan the file to be copied according to the preset fragmentation strategy, and judge whether the fragmentation of the copied file is a storage block with a continuous binary value of 0;

If so, the location information of the copied file where the empty area is located is obtained, and the storage block is determined as the empty area.

In some implementation manners, sequentially transferring the non-hole area and its offset from the address space of the sending end to the disk of the receiving end includes:

Obtain the data flow of the non-hole area, and transfer the data flow from the address space of the sender to the application buffer;

Control the data flow of the application buffer, bypass the system kernel buffer of the virtual machine, and transmit it directly to the disk at the receiving end.

In the second aspect, the embodiment of the present application provides a virtual machine migration method, which is applied to the receiving end, including:

After the receiving end establishes the connection, receive the first message sent by the sending end; wherein, the first message includes attribute information of the file to be copied;

According to the attribute information of the file to be copied contained in the first message, a temporary file corresponding to the file to be copied is established in the disk of the receiving end;

Receive the data stream and its offset from the sender, and save the data stream to a temporary file according to the offset data;

When the data stream is saved, the second message from the sender is received, and the integrity of the temporary file is verified by using the verification information contained in the second message.

In some embodiments, according to the attribute information of the file to be copied contained in the first message, a temporary file mapped to the file to be copied is established in the disk of the receiving end, including:

Receive a request message from the sender; where the request message is used to notify the receiver to execute the copy process;

The receiving end responds to the request message and transmits the response message to the sending end;

Receive the header message message from the sender; wherein, the header message message is used to obtain the attribute information of the file to be copied;

According to the attribute information of the file to be copied contained in the header message, a temporary file mapped to the copied file is established.

In the third aspect, the embodiment of the present application provides a virtual machine migration system, which is applied to the sending end, including:

The first message sending module is used to establish a connection with the receiving end, and send the first message to the receiving end; wherein, the first message includes attribute information of the file to be copied;

Hollow area determination module, used to determine the empty area of the file to be copied and its location information; wherein, the empty area is a storage block that continuously stores binary values of 0 in the file to be copied;

The data sending module is used to determine the non-hole area and its offset of the file to be copied according to the hole area and its location information, and sequentially transmit the non-hole area and its offset from the address space of the sending end to the disk of the receiving end ;

The second message sending module is used to send the second message to the receiving end after the transmission of the non-hole area and its offset is completed; wherein, the second message includes verification information for verifying the integrity of the file to be copied.

In the fourth aspect, the embodiment of the present application provides a virtual machine migration system, which is applied to the receiving end, including:

The first message receiving module is used to receive the first message sent by the sending end after the sending end establishes the connection; wherein, the first message includes attribute information of the file to be copied;

A temporary file generating module, configured to create a temporary file mapped to the file to be copied in the disk at the receiving end according to the attribute information of the file to be copied contained in the first message;

The data receiving module is used to receive the data flow and its offset from the sending end, and save the data flow to a temporary file according to the offset data;

The data verification module is configured to receive the second message from the sending end when the data stream is saved, and use the verification information contained in the second message to verify the integrity of the temporary file.

In the fifth aspect, the embodiment of the present application also provides an electronic device, including a memory and a processor, and a computer program that can run on the processor is stored in the memory, wherein, when the processor executes the computer program, the above first aspect and the first aspect are realized. The steps of the virtual machine migration method mentioned in the second aspect.

In the sixth aspect, the embodiment of the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code causes the processor to execute the virtual steps in the machine migration method.

The embodiment of the present application brings at least the following beneficial effects:

The present application provides a virtual machine migration method, system and electronic equipment. In the method, the sending end first establishes a connection with the receiving end, and sends a first message to the receiving end; wherein, the first message includes the attribute of the file to be copied information; then determine the hole area and its position information of the file to be copied; wherein, the hole area is a storage block that continuously stores binary values of 0 in the file to be copied; then determine the non-hole of the file to be copied according to the hole area and its position information area and its offset, and transfer the non-hole area and its offset from the address space of the sender to the disk of the receiver in turn; after the non-hole area and its offset are transmitted, the second message is sent to the receiver ; Wherein, the second message contains verification information for verifying the integrity of the file to be copied; and at the receiving end, the method first receives the first message sent by the sending end after the sending end establishes a connection; then according to the first The attribute information of the file to be copied contained in a message, create a temporary file mapped to the file to be copied in the disk of the receiving end, and receive the data stream and its offset from the sending end, and convert the data stream according to the offset data saving to a temporary file; when the data stream is saved, receiving the second message from the sending end, and using the verification information contained in the second message to verify the integrity of the temporary file. In this method, the value of the offset field can be used in the message to replace the transmission of empty blocks in the file, thereby improving the migration speed; and by writing directly to the disk, the occupancy rate of the page cache is reduced, and the read and write efficiency is improved, which is beneficial to the improvement of the ultra-large disk. The file transfer rate.

Other features and advantages of the present application will be described in the following descriptions, or some of the features and advantages can be deduced or unambiguously determined from the descriptions, or can be known by implementing the above-mentioned technologies of the present application.

In order to make the above-mentioned purpose, features and advantages of the present application more comprehensible, preferred implementation modes are specifically cited below, together with the accompanying drawings, which are described in detail as follows.

Description of drawings

In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the description of the specific embodiments or prior art. Obviously, the accompanying drawings in the following description The drawings are some implementations of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

FIG. 1 is a flow chart of a method for migrating a virtual machine applied to a sending end provided in an embodiment of the present application;

FIG. 2 is a flow chart of step S101 in a method for migrating a virtual machine applied to a sender provided in an embodiment of the present application;

FIG. 3 is a flow chart of step S102 in a method for migrating a virtual machine applied to a sender provided in an embodiment of the present application;

FIG. 4 is a flow chart of sequentially transferring the non-hole area and its offset from the address space of the sending end to the disk of the receiving end in a virtual machine migration method applied to the sending end provided by the embodiment of the present application;

FIG. 5 is a flow comparison diagram of bypassing the page cache for data reading and writing in a virtual machine migration method applied to the sending end provided by the embodiment of the present application;

FIG. 6 is a flow chart of a method for migrating a virtual machine applied to a receiving end provided in an embodiment of the present application;

FIG. 7 is a flow chart of step S603 in a method for migrating a virtual machine applied to a receiving end provided by an embodiment of the present application;

FIG. 8 is a signaling diagram of a virtual machine migration method provided in an embodiment of the present application;

FIG. 9 is a signaling diagram of another virtual machine migration method provided by the embodiment of the present application;

Figure 10 is a cache occupancy diagram during traditional static migration;

Figure 11 is a cache occupancy diagram when static migration is adopted in the virtual machine migration method;

FIG. 12 is a schematic structural diagram of a virtual machine migration system applied to the sending end provided by the embodiment of the present application;

FIG. 13 is a schematic structural diagram of a virtual machine migration system applied to a receiving end provided by an embodiment of the present application;

FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

icon:

1210-the first message sending module; 1220-the hollow area determination module; 1230-the data sending module; 1240-the second message sending module; 1310-the first message receiving module; 1320-temporary file generation module; 1330-data Check module; 101-processor; 102-memory; 103-bus; 104-communication interface.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. the embodiment. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

System virtualization refers to virtualizing a physical computer system into one or more virtual computer systems, and each virtual computer system (referred to as a virtual machine) has its own virtual hardware (such as CPU, memory and equipment, etc.), To provide an independent virtual machine execution environment. Through the simulation of the virtual machine monitor (Virtual Machine Monitor, VMM), the operating system task in the virtual machine is still running exclusively on one system. The operating systems in each virtual machine can be completely different, and their execution environments are completely independent. The environment where the virtual monitor runs, that is, the real physical platform, is called the host machine, and the virtualized platform, that is, the virtual machine, is also called the guest machine.

Virtualization technology refers to the technology for realizing virtualization at the software level, and is generally divided into two camps: open source virtualization and commercial virtualization. Typical representatives are: Xen, KVM, VMware, Hyper-V, etc. Among them, VMware and Hyper-V are paid commercial software, and Xen has been removed from the official kernel of Linux to support it. The current mainstream open source virtualization technology is KVM.

Compared with direct use of physical platforms, virtualization has great advantages in effective resource utilization, dynamic allocation and high reliability. Using virtualization, enterprises can build a new information infrastructure without abandoning the existing infrastructure, so as to make full use of the original IT investment. The excellent encapsulation of the virtual machine makes it easier to save the virtual machine, thus playing a greater role in disaster recovery. Virtual machine snapshots and clones make it easier to deploy various software operating environments, making testing and debugging of software development faster and easier. Running multiple virtual machines on one computer can make resource scheduling more optimal. Although different workloads can be consolidated without virtualization, the isolation of virtual machines allows each application to run independently in its own operating system environment without affecting the work in other systems load.

The main purpose of virtual machine migration is to migrate the client from one host to another, and ensure that its various services can be used normally. The main operation is to completely copy all the software (including the operating system) on the client computer to another physical hardware machine. Virtualization removes hardware dependencies, so that virtual machines on different hardware platforms can be migrated.

Virtual machine migration can be divided into static migration and live migration. The difference between the two lies in whether the virtual machine needs to be stopped during the migration process. The migration performed when the virtual machine is shut down or suspended is called static migration, and the migration performed when the virtual machine remains running is called dynamic migration. The content that needs to be migrated when the virtual machine is in the running state includes the disk file, configuration file, status and memory of the virtual machine, while only the first three items need to be migrated when the virtual machine is in the paused state, and only the first two items need to be migrated when the virtual machine is shut down. The static migration method is simple and easy to implement, and is suitable for occasions that do not have strict requirements on service availability, while the dynamic migration process only has a very short and imperceptible downtime, and the user experience is better.

During the migration process of existing virtual machines, there are often two commonly used data transmission methods: one is the transmission mechanism based on VMM, that is, data transmission is performed through the connection between the host machines; the other is the transmission mechanism based on libvirtd, That is, data transfer between two libvirtd processes. VMM-based data transmission transmits raw data, that is, directly transmits the content of the file without any processing, and does not support data encryption and integrity verification; while libvirtd-based data transmission transmits this transmission mode Encryption is supported, and data is transmitted through the RPC protocol built into libvirtd.

No matter which method of virtual machine migration is based on, during static migration, a system disk image file (disk file for short) will first be created on the source host, and then the disk file will be copied to the destination host.

Specifically, the copy process on the source host is:

(1) Open the disk file to be migrated;

(2) Read the content of the file to the cache;

(3) Send the read file content to the new host computer through TCP/UCP;

(4) Loop (2)-(3) until the file is sent.

The copy process on the destination host is:

(1) Create a new file and open it;

(2) Read the disk content transmitted by the peer through TCP/UDP;

(3) Write the read content to the file created in (1);

(4) Loop (2)-(3) until the file is received.

At the same time, the biggest problem of using cached I/O during virtual machine migration is the consumption of page cache. Buffered I/O, also known as standard I/O, is the default I/O operation mode for most file systems. In the cached I/O mechanism of Linux, the operating system will cache the I/O data in the page cache (page cache) of the file system. That is to say, when reading a disk file, the data will be copied to the buffer of the operating system kernel first, and then copied from the buffer of the operating system kernel to the address space of the application program. Cached I/O has the following advantages: Cached I/O uses the kernel buffer of the operating system, which to a certain extent separates the application space and the actual physical device space; cached I/O can reduce the number of disk reads, thereby improving performance. When an application program reads a certain piece of data, if the piece of data is already in the page cache, then this piece of data can be returned to the application program immediately without going through the actual physical disk reading operation. Since the reading speed of the page cache is much higher than that of the disk, and data access is usually continuous, the data acquisition speed can be greatly improved by caching. For write operations, the application will first write the data to the page cache. Whether the data is written to the disk immediately depends on the writing mechanism used by the application: but whether the application uses synchronous writes (synchronous writes), delayed writes Mechanism (deferred writes) or asynchronous writes (asynchronous writes) will pass through the page cache and then be written to the real physical disk.

During the virtual machine migration process, a large number of frequent file read and write operations are involved, and a large number of data copy operations are performed between the address space of the application program and the page cache, and between the page cache and the disk. These data copy operations bring The CPU and memory overhead is very large. If the migrated disk file is large, or if there are multiple virtual machine files to be migrated at the same time, the migration efficiency will decrease.

Based on this, a virtual machine migration method, system, and electronic device provided by the embodiments of the present application can use the offset data of the empty block of the file in the message to replace the transmission of the empty block of the file, thereby increasing the migration speed; and by The write-to-disk mode reduces the occupancy rate of the page cache, improves read and write efficiency, and helps to increase the transmission rate of very large disk files.

To facilitate understanding of this embodiment, a method for migrating a virtual machine disclosed in this embodiment of the present application is firstly introduced in detail.

Referring to the flow chart of a virtual machine migration method shown in Figure 1, the method is applied to the sending end, including:

Step S101, establish a connection with the receiving end, and send a first message to the receiving end; wherein, the first message includes attribute information of the file to be copied.

This step is an initialization step. Because the virtual machine migration process consumes a lot of resources and time, the sending end and the receiving end need to establish a reliable and stable connection, usually using a long-term connection for interconnection. To ensure transmission reliability, you can choose to use TCP connection, and the connection protocol supports bidirectional transmission of files.

After the connection with the receiving end is successful, the first message is sent to it to perform the migration, and the first message includes the attribute information of the file to be copied. The first message may include a request message and a head message, notifying the receiving end that the file will be copied.

During the data sending process, related head files and messages will be involved, and the receiving end will be notified of the basic information of the file through the head message, such as file name and file size. E.g:

1.Base Head, the basic message header. It is used to determine different message types and contains the basic information of the message message, such as the following parameters: Total Size: the length of the entire message; HEAD: the "HEAD" mark; Version: the software version number; HeadSize: the length of the message header; PacketType: message type; PathSize: length of the transmitted file path; CRCCode: CRC check code of all previous contents; FilePath: transmitted file path.

2.Head type message. The details are as follows: Total Size: the entire message length; HEAD: "HEAD" mark; Version: software version number; HeadSize: message header length; PacketType=1; indicates head type message; PathSize: transmitted file path length; CRCCode : CRC check code of all previous contents; FilePath: path of the transferred file; FileSize: file size.

3. Data type message. The details are as follows: Total Size: the entire message length; HEAD: "HEAD" mark; Version: software version number; HeadSize: message header length; PacketType=2; indicates data type message; PathSize: transmitted file path length; CRCCode : CRC check code of all previous content; FilePath: file path to transfer; Offset: offset of the content on the disk, used to solve the problem of data inconsistency caused by the failure of the transmission of the empty slice; Data: file content.

4. End type message. The details are as follows: Total Size: the entire message length; HEAD: "HEAD" mark; Version: software version number; HeadSize: message header length; PacketType=3; indicates end type message; PathSize: transmitted file path length; CRCCode : CRC check code of all previous contents; FilePath: path of the transmitted file; MD5Code: MD5 value of the entire file.

5. Request type message. The details are as follows: Total Size: the entire message length; HEAD: "HEAD" mark; Version: software version number; HeadSize: message header length; PacketType=4; indicates request type message; PathSize: transmitted file path length; CRCCode : CRC check code of all previous content; FilePath: file path to transfer; Operation: operation (send file/receive file); EnableVerify: whether to perform verification.

6. Response type message. The details are as follows: Total Size: the entire message length; HEAD: "HEAD" mark; Version: software version number; HeadSize: message header length; PacketType=5; indicates response type message; PathSize: transmitted file path length; CRCCode : CRC check code of all previous content; FilePath: path of the transmitted file; Response Type: reply message type; Error Code: error code.

Step S102, determining the hole area and its location information of the file to be copied; wherein, the hole area is a memory block in the file to be copied that continuously stores binary values of 0.

The empty area is a storage block that continuously stores binary values of 0 in the file to be copied. By scanning the file to be copied and judging whether it contains an empty area, if it is included, record the position of the empty area. At the same time, after the position of the cavity area is obtained during the scanning process, the position information of the non-cavity area can be obtained at the same time.

Step S103, according to the hole area and its location information, determine the non-hole area and its offset of the file to be copied, and sequentially transmit the non-hole area and its offset from the address space of the sending end to the disk of the receiving end.

After the non-hole area is obtained, the non-hole area of the file to be copied can be obtained, that is, the data to be transmitted. The offset of the non-hole area can be obtained according to the position information of the non-hole area and the position information of the hole area. This offset is used as a copy parameter, and is transmitted to the disk at the receiving end together with the data in the non-hole area. This process does not copy the non-hole area, thereby saving transmission bandwidth.

Step S104, after the non-hole area and its offset are transmitted, a second message is sent to the receiving end; wherein, the second message includes verification information for verifying the integrity of the file to be copied.

After the transmission ends, the sending end sends a second message to the receiving end to inform the receiving end of the completion of the transmission. At the same time, the integrity of the file to be transmitted that has been transmitted is verified through the verification information contained in the second message.

It can be known from the virtual machine migration method provided in the above-mentioned embodiments that this method can optimize the transmission speed of files with "holes"; Send, thereby reducing the number of packets transmitted by the file network, shortening the overall time of file transmission, and improving its transmission efficiency.

In some embodiments, the step S101 of establishing a connection with the receiving end and sending the first message to the receiving end, as shown in FIG. 2 , includes:

Step S201, the sending end establishes a long connection with the receiving end, and initializes the request message and the header message message in the first message; wherein, the request message is used to notify the receiving end to perform the copy process; the header message message is used to send The receiving end notifies the attribute information of the file to be copied.

Specifically, the format of the Request message can refer to the format of the aforementioned Request type message, as follows:

Total Size: the length of the entire message;

HEAD: "HEAD" tag;

Version: software version number;

HeadSize: message header length;

PacketType=4; indicates request type message;

PathSize: the length of the transmitted file path;

CRCCode: CRC check code of all previous content;

FilePath: the path of the transferred file;

Operation: operation (send file/receive file);

EnableVerify: Whether to verify.

The format of the header message message can refer to the aforementioned Head type message, as follows:

Total Size: the length of the entire message;

HEAD: "HEAD" tag;

Version: software version number;

HeadSize: message header length;

PacketType=1; indicates head type message;

PathSize: the length of the transmitted file path;

CRCCode: CRC check code of all previous content;

FilePath: the path of the transferred file;

FileSize: file size.

Step S202, sending a request message to the receiving end.

After establishing a long connection with the receiving end, first send a request message to the receiving end to notify the receiving end that the file will be copied. This step can be used as a notification step of the receiving end, and the transmission of the subsequent header message needs to be performed after the receiving end responds to the request message.

Step S203, after receiving the response message of the request message, sending the header message message to the receiving end.

After receiving the response message from the receiving end, the sending end informs the receiving end of the basic information of the file through the head message, including the file name (path) and file size.

In some implementations, the step S102 of determining the empty area of the file to be copied and its location information, as shown in FIG. 3 , includes:

Step S301, scan the file to be copied according to the preset fragmentation strategy, and judge whether the fragments of the copied file are storage blocks with consecutive binary values of 0.

For the files to be copied, it is necessary to scan them, detect their storage information and read them; specifically, it can be realized according to the preset fragmentation strategy. For example, in the file to be copied, each time the storage area of the specified size is read, it is judged whether these fragments are continuous storage blocks of 0.

Step S302, if yes, obtain the location information of the copied file where the empty area is located, and determine the storage block as the empty area.

After obtaining the hollow areas, determine the positions of these hollow areas, such as the start number, the end number, the length of the hollow area, and so on. These position information can be used to calculate the offset of the non-hole area, and finally perform data offset during the copy process.

The non-hole area is used as the data for real transmission, while the hole area does not perform actual transmission; therefore, in the actual operation process, it is necessary to transfer the file content and offset in the relevant storage block in the non-hole area, for example, the offset field can be used value to replace the transfer of empty blocks in the file. For the scenario of virtual machine migration, the files read in the page cache only need to be sent to the peer end as needed, and there is no possibility of secondary reading; the data written in the page cache does not involve re-modification , the need to delay writes. Based on the above considerations, the improvement of read and write efficiency brought about by the conventional cache I/O method will not play a role in the process of virtual machine migration, but will reduce efficiency. Therefore, in some embodiments, the non-hole area and its offset are sequentially transferred from the address space of the sending end to the disk of the receiving end, as shown in FIG. 4 , including:

In step S401, the data stream of the non-hole area is obtained, and the data stream is transferred from the address space of the sender to the application buffer.

In the cached I/O mechanism of Linux, the operating system will cache the I/O data in the page cache (page cache) of the file system. That is to say, when reading a disk file, the data will be copied to the buffer of the operating system kernel first, and then copied from the buffer of the operating system kernel to the address space of the application program. Cached I/O has the following advantages: Cached I/O uses the kernel buffer of the operating system, which to a certain extent separates the application space and the actual physical device space; cached I/O can reduce the number of disk reads, thereby improving performance. When an application program reads a certain piece of data, if the piece of data is already in the page cache, then this piece of data can be returned to the application program immediately without going through the actual physical disk reading operation. Since the reading speed of the page cache is much higher than that of the disk, and data access is usually continuous, the data acquisition speed can be greatly improved by caching.

Step S402, controlling the data flow of the application buffer, bypassing the system kernel buffer of the virtual machine, and directly transmitting to the disk at the receiving end.

Aiming at the memory overhead and multiple copy efficiency problems caused by cache I/O in virtual machine migration, data transmission is performed by directly reading and writing disks by applications, bypassing the page cache. The traditional data flow needs to be transmitted from the Application buffer (application buffer) to the Page cache (page cache) through the Clib buffer (Clib buffer), and then from the page cache to the Disk cache (disk cache) of the device layer, and finally to the disk. For the scenario of virtual machine migration, the files read in the page cache only need to be sent to the peer end as needed, and there is no possibility of secondary reading; the data written in the page cache does not involve re-modification , the need to delay writes. Based on the above considerations, the improvement of read and write efficiency brought about by the conventional cache I/O method will not play a role in the process of virtual machine migration, but will reduce efficiency.

Therefore, in step S402, by bypassing the page cache and directly reading and writing the disk to perform data read and write operations, it is also possible to transfer very large disk files when the memory is small or occupied by other applications. . A comparison diagram of the above process is shown in FIG. 5 , which will not be repeated here.

The embodiment of the present application provides a virtual machine migration method, which is applied to the receiving end, as shown in Figure 6, including:

In step S601, after the connection is established by the receiving end, the first message sent by the sending end is received; wherein, the first message includes attribute information of the file to be copied.

After sending the first message to the receiving end, the sending end informs the receiving end that it is ready to accept the file. For example, the sending end sends the request message to the receiving end, and after receiving the message, the receiving end sends a response message of the request message to the sending end.

Step S602, according to the attribute information of the file to be copied contained in the first message, a temporary file mapped to the file to be copied is created in the disk of the receiving end.

The first message includes a related head message, and the head message includes related attribute information of the file to be copied, such as file name, path, file size, and the like. The receiving end creates a file to be copied in the local disk of the receiving end according to the content of the head message. During the creation process, it is necessary to create temporary files corresponding to the file names of the files to be copied one by one. The file names of these temporary files are in one-to-one correspondence with the files contained in the file to be copied, but the content of the files is temporarily empty, which needs to be passed through each The data stream of the file to fill.

Step S603, receiving the data stream and its offset from the sender, and saving the data stream into a temporary file according to the offset data.

After receiving the data of the data stream, copy the data according to its offset. Due to the amount of data transmitted from the sender and the removal of the hole data, the actual size of the data amount is much smaller than the conventional data amount containing the hole data. These data streams are sequentially saved to temporary files according to their own offsets.

Step S604, when the data stream is saved, receive the second message from the sender, and verify the integrity of the temporary file by using the verification information included in the second message.

When the data stream is saved, the sending end will send the second message to the receiving end. At this time, the receiving end obtains the second packet and then verifies the integrity of the temporary file. When the temporary file is consistent with the file to be copied, it indicates that the copy process is normal, and the verification result can be sent to the sender at the same time.

In some implementations, according to the attribute information of the file to be copied contained in the first message, the step S603 of creating a temporary file mapped to the file to be copied in the disk of the receiving end, as shown in FIG. 7 , includes:

Step S701, receiving a request message from the sender; wherein, the request message is used to notify the receiver to execute the copying process.

The request message in this step is a request message, which is sent from the sending end to the receiving end, and is used to notify the receiving end to execute file copying.

Step S702, the receiving end responds to the request message, and transmits the response message to the sending end.

The receiving end responds to the request message, and sends a response message to the sending end, informing the sending end that the file can be copied.

Step S703, receiving a header message from the sender; wherein, the header message is used to acquire attribute information of the file to be copied.

After receiving the response message from the receiving end, the sending end sends a head message to the receiving end to inform the receiving end of the file attribute information to be copied, including file name, file path, and file size.

Step S704, according to the attribute information of the file to be copied contained in the header message, create a temporary file mapped to the copied file.

The attribute information of the file to be copied contained in the head message message, such as multiple types of Head type message, Data type message, End type message, Request type message, and Response type message mentioned in the above-mentioned embodiment message, according to the message content contained in these header message messages, create corresponding temporary files in the receiving end disk.

An overall description of the data migration process including the sending end and the receiving end in the above virtual machine migration method is given below in combination with specific embodiments.

Take the sender as the operation initiator and transfer the local file to the receiver as an example:

1. The sender initiates a connection, first sends a request message to notify the receiver that the file will be copied;

2. Inform the receiving end of the basic information of the file to be copied through the head message, including the file name (path) and file size;

3. Scan the files to be copied in turn, read the specified size (protocol fragment size) each time, and judge whether it is a "hole" slice. All "hole" slices are not transmitted; the information carried by non-hole slices must include: the offset of the slice in the disk and the content of the file slice, and the offset is used to solve the problem of data inconsistency caused by the failure of transmission of the hollow slice.

4. After the transmission is over, the sending end sends an end message to the receiving end. The end message is used to notify the receiving end that the file transfer is over, and carries verification information for the receiving end to verify the integrity of the transmitted file;

The receiving end sequentially receives the messages sent by the sending end and performs corresponding processing.

1. After receiving the request message, prepare to receive the file;

2. After receiving the head message, create a file with the specified name in the local specified location according to the content of the message;

3. After receiving the data message, copy the data according to the offset;

4. After receiving the end message, calculate the integrity of the verification content; and return the verification result to the sender.

The above process can be described by a signaling diagram of a virtual machine migration method as shown in FIG. 8 , in which Host1 can be understood as a sending end; Host2 is a receiving end.

1. Host1 sends a Requset message to Host2, informing Host2 that the file will be copied;

2. Host2 responds to the Requset message and sends a Response message to Host1;

3. Host1 sends the head message to Host2, informing Host2 of the basic information of the file to be copied;

4. Host2 responds to the head message and sends a Response message to Host1;

5. Host1 transmits the file to Host2 through the data message, and the transmission process ends with the data and its offset not including the empty area. The data transmission is completed after N times of recirculation. At this time, N+1 transmissions send an end message to Host2, instructing Host2 to complete the file transmission, and verify the data in Host2 through the verification data in the end message.

FIG. 9 is a signaling diagram of another virtual machine migration method, in which Host1 requests Host2 to migrate to Host1. If it is classified by the location of the migrated data, Host2 is the sending end; Host1 is the receiving end. Specifically, the above virtual machine migration process is as follows:

1. Host2 sends a Requset message to Host1, informing Hos1 that the file will be copied;

2. Host1 responds to the Requset message and sends a Response message to Host2;

3. Host2 sends the head message to Host1, informing Host1 of the basic information of the file to be copied;

4. Host1 responds to the head message and sends a Response message to Host2;

5. Host2 transmits the file to Host1 through the data message, and the transmission process reaches the data and its offset that does not include the empty area. The data transmission is completed after N times of recirculation. At this time, N+1 transmissions send an end message to Host1, instructing Host1 to complete the file transmission, and verify the data in Host1 through the verification data in the end message.

According to the virtual machine migration method mentioned in the above embodiment, it can be seen that the transmission speed of files with "holes" can be optimized; by verifying the content of the message fragments, the full "hole" message fragments are not sent , thereby reducing the number of packets transmitted over the file network, shortening the overall time of file transmission, and improving its transmission efficiency. Virtual machine migration can be performed by directly reading and writing disks, which can reduce cache usage and migrate very large files. Fig. 10 is a cache occupancy diagram during traditional static migration, and Fig. 11 is a cache occupancy diagram during static migration using the virtual machine migration method. The horizontal axis in the figure is time (unit s), and the vertical axis is cache usage (unit M), the total cache size of the host is 3500M. It can be seen that during the ordinary static migration process, the cache is quickly filled, but during the write-to-disk virtual machine migration process, the cache hardly increases. It can be seen that this method improves the read-write efficiency and is beneficial to increase the transmission rate of super-large disk files.

Corresponding to the above method embodiment, the embodiment of the present application provides a virtual machine migration system, as shown in Figure 12, the system is applied to the sending end, including:

The first message sending module 1210 is configured to establish a connection with the receiving end, and send the first message to the receiving end; wherein, the first message includes attribute information of the file to be copied;

Hole area determining module 1220, for determining the hole area of the file to be copied and its location information; wherein, the hole area is a storage block that continuously stores binary values of 0 in the file to be copied;

The data sending module 1230 is configured to determine the non-hole area and its offset of the file to be copied according to the hole area and its location information, and sequentially transmit the non-hole area and its offset from the address space of the sending end to the disk of the receiving end middle;

The second message sending module 1240 is configured to send a second message to the receiving end after the non-hole area and its offset are transmitted; wherein, the second message includes verification information for verifying the integrity of the file to be copied .

Corresponding to the above method embodiment, the embodiment of the present application provides a virtual machine migration system, as shown in Figure 13, the system is applied to the receiving end, including:

The first message receiving module 1310 is configured to receive the first message sent by the sending end after the connection is established by the sending end; wherein, the first message includes attribute information of the file to be copied;

The temporary file generation module 1320 is used to create a temporary file mapped to the file to be copied in the disk of the receiving end according to the attribute information of the file to be copied contained in the first message;

The data verification module 1330 is configured to receive the second message from the sender when the data stream is saved, and use the verification information contained in the second message to verify the integrity of the temporary file.

The virtual machine migration system provided in the embodiment of the present application has the same technical features as the virtual machine migration method provided in the above embodiment, so it can also solve the same technical problem and achieve the same technical effect. For brief description, for the parts not mentioned in the embodiments, reference may be made to the corresponding content in the aforementioned embodiments of the virtual machine migration method.

This embodiment also provides an electronic device, the structural diagram of which is shown in Figure 14, the device includes a processor 101 and a memory 102; wherein, the memory 102 is used to store one or more computer instructions, one or more A computer instruction is executed by the processor to implement the above virtual machine migration method.

The electronic device shown in FIG. 14 further includes a bus 103 and a communication interface 104 , and the processor 101 , the communication interface 104 and the memory 102 are connected through the bus 103 .

Wherein, the memory 102 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The bus 103 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one double-headed arrow is used in FIG. 14 , but it does not mean that there is only one bus or one type of bus.

The communication interface 104 is used to connect with at least one user terminal and other network elements through the network interface, and send the encapsulated IPv4 message or the IPv4 message to the user terminal through the network interface.

The processor 101 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 101 or instructions in the form of software. Above-mentioned processor 101 can be general-purpose processor, comprises central processing unit (Central Processing Unit, be called for short CPU), network processor (Network Processor, be called for short NP) etc.; Can also be Digital Signal Processor (Digital Signal Processor, be called for short DSP) ), Application Specific Integrated Circuit (ASIC for short), Field Programmable Gate Array (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the methods disclosed in the embodiments of the present disclosure may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102, and completes the steps of the methods of the foregoing embodiments in combination with its hardware.

The embodiment of the present application also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the methods in the foregoing embodiments are executed.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Finally, it should be noted that: the above-described embodiments are only specific implementations of the application, used to illustrate the technical solutions of the application, rather than limiting it, and the scope of protection of the application is not limited thereto, although referring to the aforementioned The embodiment has described this application in detail, and those of ordinary skill in the art should understand that any person familiar with this technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in this application Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be covered by this application. within the scope of protection. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

A virtual machine migration method, characterized in that the method is applied to the sending end, including:

Establishing a connection with the receiving end, and sending a first message to the receiving end; wherein, the first message includes attribute information of the file to be copied;

Determining the hole area and its location information of the file to be copied; wherein, the hole area is a storage block that continuously stores binary values of 0 in the file to be copied;

According to the hole area and its location information, determine the non-hole area and its offset of the file to be copied, and sequentially transmit the non-hole area and its offset from the address space of the sending end to the In the disk at the receiving end;

Sending a second message to the receiving end after the non-hole area and its offset are transmitted; wherein, the second message includes verification information for verifying the integrity of the file to be copied.
The virtual machine migration method according to claim 1, wherein the step of establishing a connection with the receiving end and sending the first message to the receiving end includes:

The sending end establishes a long connection with the receiving end, and initializes a request message and a header message message in the first message; wherein, the request message is used to notify the receiving end to perform a copy process; The header message is used to inform the receiving end of the attribute information of the file to be copied;

sending the request message to the receiving end;

After receiving the response message of the request message, sending the header message message to the receiving end.
The virtual machine migration method according to claim 1, wherein the step of determining the empty area of the file to be copied and its location information includes:

Scan the file to be copied according to the preset fragmentation strategy, and judge whether the fragmentation of the copied file is a storage block with a continuous binary value of 0;

If yes, acquire the location information of the copied file where the empty area is located, and determine the storage block as the empty area.
The virtual machine migration method according to claim 1, wherein sequentially transferring the non-hole area and its offset from the address space of the sending end to the disk of the receiving end includes:

Obtaining the data stream of the non-hole area, and transferring the data stream from the address space of the sending end to the application buffer;

The data flow of the application buffer is controlled, bypassing the system kernel buffer of the virtual machine, and directly transmitted to the disk of the receiving end.
A virtual machine migration method, characterized in that the method is applied to a receiving end, including:

After the receiving end establishes the connection, receive the first message sent by the sending end; wherein, the first message includes attribute information of the file to be copied;

According to the attribute information of the file to be copied contained in the first message, create a temporary file mapped to the file to be copied in the disk of the receiving end;

receiving the data stream and its offset from the sending end, and saving the data stream into the temporary file according to the offset data;

When the storage of the data stream is completed, the second message from the sending end is received, and the integrity of the temporary file is verified by using the verification information contained in the second message.
The virtual machine migration method according to claim 5, wherein, according to the attribute information of the file to be copied contained in the first message, an Mapped temporary files, including:

receiving a request message from the sending end; wherein, the request message is used to notify the receiving end to perform a copy process;

The receiving end responds to the request message, and transmits the response message to the sending end;

receiving a header message message from the sending end; wherein, the header message message is used to obtain attribute information of the file to be copied;

A temporary file mapped to the copied file is established according to the attribute information of the file to be copied contained in the header message.
A virtual machine migration system, characterized in that the system is applied to the sending end, including:

A first message sending module, configured to establish a connection with the receiving end, and send a first message to the receiving end; wherein, the first message includes attribute information of the file to be copied;

A hole area determination module, configured to determine the hole area of the file to be copied and its location information; wherein, the hole area is a storage block that continuously stores binary values of 0 in the file to be copied;

The data sending module is used to determine the non-hole area and its offset of the file to be copied according to the hole area and its position information, and sequentially transfer the non-hole area and its offset from the sending end The address space is transmitted to the disk at the receiving end;

The second message sending module is configured to send a second message to the receiving end after the transmission of the non-hole area and its offset is completed; wherein, the second message includes complete information about the file to be copied. The verification information for verification.
A virtual machine migration system, characterized in that the system is applied to a receiving end, including:

The first message receiving module is used to receive the first message sent by the sending end after the connection is established by the sending end; wherein, the first message includes attribute information of the file to be copied;

A temporary file generation module, configured to create a temporary file mapped to the file to be copied in the disk of the receiving end according to the attribute information of the file to be copied contained in the first message;

A data receiving module, configured to receive the data stream and its offset from the sending end, and save the data stream into the temporary file according to the offset data;

A data verification module, configured to receive the second message from the sending end when the data stream is saved, and check the integrity of the temporary file by using the verification information contained in the second message check.
An electronic device, characterized in that it comprises: a processor and a storage device; a computer program is stored on the storage device, and when the computer program is executed by the processor, the computer program according to any one of claims 1 to 6 can be realized. Steps of the virtual machine migration method described above.
A computer-readable storage medium, with a computer program stored on the computer-readable storage medium, characterized in that, when the computer program is run by a processor, the virtual machine migration described in any one of claims 1 to 6 is implemented method steps.